JP2024067631A - Apparatus, method and program - Google Patents

Abstract

[Problem] A device is provided that outputs recommended control parameters output from a control model. [Solution] An apparatus is provided that includes a deviation acquisition unit that acquires a deviation between a measured value and a target value of a state related to a controlled object, a control parameter acquisition unit that acquires shifted control parameters obtained by shifting control parameters supplied to the controlled object, a first supply unit that supplies the deviation acquired by the deviation acquisition unit and the shifted control parameters acquired by the control parameter acquisition unit to a control model that outputs recommended control parameters to be supplied to the controlled object in response to input of the deviation and control parameters, and an output unit that outputs the recommended control parameters output from the control model in response to supply to the control model from the first supply unit. [Selected Figure] Figure 1

Description

The present invention relates to an apparatus, a method and a program.

Patent Documents 1 to 4 state, for example, that "an operation amount map is selected based on the target value SV, and the operation amount MV is calculated using the selected operation amount map" (paragraph 0031 of Patent Document 1).
[Prior Art Literature]
[Patent Documents]
[Patent Document 1] JP 2022-156797 A [Patent Document 2] JP 2020-95352 A [Patent Document 3] JP 2021-117699 A [Patent Document 4] JP 2022-014099 A

In a first aspect of the present invention, there is provided an apparatus including: a deviation acquisition unit that acquires the deviation between a measured value and a target value of a state related to a control object; a control parameter acquisition unit that acquires shifted control parameters obtained by shifting control parameters supplied to the control object; a first supply unit that supplies the deviation acquired by the deviation acquisition unit and the shifted control parameters acquired by the control parameter acquisition unit to a control model that outputs recommended control parameters to be supplied to the control object in response to input of the deviation and the control parameters; and an output unit that outputs the recommended control parameters output from the control model in response to supply from the first supply unit to the control model.

The above device may further include a target value acquisition unit that acquires a target value for the state, and the control parameter acquisition unit may shift the control parameters supplied to the control object in response to the target value being changed from a reference target value, and acquire the shifted control parameters.

In the above device, the control parameter acquisition unit may, in response to the target value being changed from the reference target value, shift the control parameter supplied to the control object by a shift amount corresponding to the target value, and acquire the shifted control parameter.

In the above device that shifts the control parameter by a shift amount corresponding to the target value, the control parameter acquisition unit may determine the shift amount by multiplying the difference between the target value and the reference target value by a preset coefficient.

In the above device in which the control parameter is shifted by a shift amount corresponding to the target value, the control parameter acquisition unit may determine the shift amount using a pre-set relational expression that indicates the relationship between a value set as the target value and the value of the control parameter when the measured value stabilizes at that value.

Any of the above devices may further include a first detection unit that detects that the deviation acquired by the deviation acquisition unit is not stable within a reference range after the control object is controlled by the recommended control parameters, and the control parameter acquisition unit may shift the control parameters supplied to the control object and acquire the shifted control parameters in response to the first detection unit detecting that the deviation acquired by the deviation acquisition unit is not stable within the reference range.

The above device further includes a second supply unit that supplies the deviation acquired by the deviation acquisition unit and the rate of change of the deviation to a shift amount output model that outputs a recommended shift amount that recommends a shift of a control parameter by the control parameter acquisition unit in response to input of the deviation and the rate of change of the deviation, and the control parameter acquisition unit may shift the control parameter supplied to the controlled object by the recommended shift amount output from the shift amount output model in response to detection by the first detection unit that the deviation acquired by the deviation acquisition unit is not stable within the reference range and supply from the second supply unit to the shift amount output model to acquire the shifted control parameter.

In the above device that supplies the deviation acquired by the deviation acquisition unit and the rate of change of the deviation to a shift amount output model, the second supply unit may supply the deviation to the shift amount output model at each reference interval.

The above device, which supplies the deviation acquired by the deviation acquisition unit and the rate of change of the deviation to a shift amount output model, may further include a first learning processing unit that performs learning processing of the shift amount output model using learning data including the deviation acquired by the deviation acquisition unit, the rate of change of the deviation, and the shift amount of the control parameter acquired by the control parameter acquisition unit, and outputs the recommended shift amount recommended to increase a reward value determined by a preset reward function in response to input of the deviation and the rate of change of the deviation.

Any of the above devices may further include a second detection unit that detects that the control target has been changed, and the control parameter acquisition unit may shift the control parameters supplied to the control target in response to the second detection unit detecting that the control target has been changed, and acquire the shifted control parameters.

In any of the above devices, the control model may include a change amount output model that outputs a recommended change amount that recommends a change to the control parameter in response to input of the deviation and the control parameter, and an adder that calculates the recommended control parameter by adding the control parameter supplied to the control target and the recommended change amount output from the change amount output model.

The above device may further include a second learning processing unit that performs learning processing of the change amount output model using learning data including the deviation acquired by the deviation acquisition unit and the control parameters acquired by the control parameter acquisition unit, and outputs the recommended change amount recommended to increase a reward value determined by a preset reward function in response to input of the deviation and the control parameters.

In a second aspect of the present invention, a method is provided that includes a deviation acquisition step of acquiring a deviation between a measured value and a target value of a state related to a control object, a control parameter acquisition step of acquiring a shifted control parameter obtained by shifting a control parameter supplied to the control object, a first supply step of supplying the deviation acquired by the deviation acquisition step and the shifted control parameter acquired by the control parameter acquisition step to a control model that outputs a recommended control parameter recommended to be supplied to the control object in response to input of the deviation and the control parameter, and an output step of outputting the recommended control parameter output from the control model in response to supply to the control model by the first supply step.

In a third aspect of the present invention, a program is provided that causes a computer to function as a deviation acquisition unit that acquires the deviation between a measured value and a target value of a state related to a control object, a control parameter acquisition unit that acquires shifted control parameters obtained by shifting control parameters supplied to the control object, a first supply unit that supplies the deviation acquired by the deviation acquisition unit and the shifted control parameters acquired by the control parameter acquisition unit to a control model that outputs recommended control parameters to be supplied to the control object in response to input of the deviation and control parameters, and an output unit that outputs the recommended control parameters output from the control model in response to supply from the first supply unit to the control model.

Note that the above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions.

1 shows a system 1 according to a first embodiment. 2 shows an example of a change amount output model 2061. 2 shows another example of the change amount output model 2061. The effect of shifting the control parameters input to the control model 206 is shown. The fitted curve of the sample data is shown. 2 illustrates the operation of the device 200. 13 shows the transition of the measured value PV and the control parameter when the target value SP is changed. 1 shows a system 1A according to a second embodiment. 2 shows an example of a shift amount output model 213A. 13 shows another example of the shift amount output model 213A. The operation of the device 200A is shown. 13 shows the transition of the measured value PV and the control parameter when a disturbance occurs. 13 shows a system 1B according to a third embodiment. 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part.

The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

<1. First embodiment>
<1.1. System 1>
1 shows a system 1 according to the first embodiment. The system 1 includes a facility 100 and an apparatus 200.

<1.1.1. Equipment 100>
The equipment 100 is a facility or device in which a control target 101 is installed. For example, the equipment 100 may be a plant or a composite device in which multiple devices are combined. In addition to industrial plants such as chemical and biotechnology plants, plants that manage and control wellheads and surrounding areas of gas and oil fields, plants that manage and control hydroelectric, thermal and nuclear power plants, and environmental power plants such as solar and wind power plants. Examples of such plants include those that manage and control water supply, sewage, and dams.

The facility 100 is provided with one or more control objects 101. The control object 101 may be a tool, machine, or device to be controlled, and may be a so-called field device. For example, the control object 101 may be a sensor device such as a pressure gauge, flow meter, or temperature sensor, a valve device such as a flow control valve or an on-off valve, or an actuator device such as a fan or a motor. The control object 101 is controlled from the outside by wire or wirelessly, but may also be controlled manually. The control object 101 may be controlled by the control unit 207 in the device 200. In this embodiment, as an example, the control object 101 may be controlled by supplying an instruction value IV (Instructed Value) for a manipulated variable MV (Manipulated Variable) from the control unit 207.

The facility 100 may be provided with one or more sensors 102. Each sensor 102 may measure a measurement value of the inside and outside state of the facility 100, that is, a measurement value of a physical quantity indicating the inside and outside state. At least one sensor 102 may measure a measurement value PV (Process Variable) of the state of the control object 101. The measurement value PV may be operation data indicating the operation state as a result of controlling the control object 101, and may indicate the control amount to be controlled. As an example, the measurement value PV may indicate the output of the control object 101 itself, or may indicate various values that change depending on the output of the control object 101. As an example, the measurement value PV may indicate pressure, temperature, pH, speed, flow rate, etc. Each sensor 102 may supply the measured measurement value PV to the device 200.

<1.1.2. Apparatus 200>
The device 200 controls the controlled object 101, and may be, for example, a controller for the controlled object 101. The device 200 outputs an instruction value IV for a manipulated variable MV of the controlled object 101 to adjust the temperature, Process controls such as adjusting the surface level or adjusting the flow rate may be performed.

The device 200 may be a computer such as a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, or a general-purpose computer, or may be a computer system to which multiple computers are connected. Such a computer system is also a computer in the broad sense. The device 200 may be implemented by a virtual computer environment in which one or more programs can be executed within a computer. Alternatively, the device 200 may be a dedicated computer designed for AI control, or may be dedicated hardware realized by a dedicated circuit. Furthermore, if the device 200 can be connected to the Internet, the device 200 may be realized by cloud computing.

The device 200 may have a measurement value acquisition unit 201, a target value acquisition unit 202, a deviation acquisition unit 203, a control parameter acquisition unit 204, a first supply unit 205, a control model 206, a control unit 207, and a learning processing unit 208. Note that these blocks are functionally separated functional blocks, and may not necessarily match the actual device configuration. That is, even if shown as one block in this diagram, it is not limited to being configured by one device. Also, even if shown as separate blocks in this diagram, it is not limited to being configured by separate devices.

<1.1.2-1. Measurement value acquisition unit 201>
The measurement value acquiring unit 201 acquires a measurement value PV of a state related to the control target 101. In the present embodiment, as an example, the measurement value acquiring unit 201 acquires a measurement value PV for one physical quantity from one sensor 102, but the measurement value acquiring unit 201 may acquire measurement values PV for each of a plurality of physical quantities from a plurality of sensors 102. The measurement value acquiring unit 201 may supply the acquired measurement value PV to the deviation acquiring unit 203.

<1.1.2-2. Target value acquisition unit 202>
The target value acquisition unit 202 acquires a target value SP (Set Point) of a state related to the control target 101. The target value acquisition unit 202 may acquire the target value SP of the measurement value PV acquired by the measurement value acquisition unit 201. The target value acquisition unit 202 may acquire the target value SP from an operator via an input unit (not shown). When the target value SP is not set by the operator, the target value acquisition unit 202 may acquire a preset reference target value as the target value SP. The target value acquisition unit 202 may supply the acquired target value SP to the deviation acquisition unit 203. In response to the target value SP being set to a value different from the reference target value, the target value acquisition unit 202 may supply a signal indicating the set target value SP (also referred to as a target value change signal) to the control parameter acquisition unit 204.

<1.1.2-3. Deviation acquisition unit 203>
The deviation acquisition unit 203 acquires the deviation between the measurement value PV and the target value SP of the state related to the control target 101. The deviation acquisition unit 203 may acquire the measurement value PV from the measurement value acquisition unit 201 and the target value SP from the target value acquisition unit 202, and calculate the deviation by subtracting the measurement value PV from the target value SP. Alternatively, the deviation acquisition unit 203 may calculate the deviation by subtracting the target value SP from the measurement value PV. The deviation acquisition unit 203 may supply the acquired deviation to the first supply unit 205. The deviation acquisition unit 203 may store the acquired deviation in a storage unit not shown.

<1.1.2-4. Control parameter acquisition unit 204>
The control parameter acquisition unit 204 acquires a shifted control parameter P+Δp by shifting the control parameter P supplied to the control object 101. The control parameter acquisition unit 204 may acquire the control parameter P from the control unit 207 described later, and in the present embodiment, as an example, may acquire the control parameter P every time the control unit 207 supplies the control parameter P to the control object 101. The control parameter P may indicate an instruction value IV for an operation amount MV of the control object 101. When the control object 101 is a valve, the control parameter P may indicate a valve opening degree, as an example. The control parameter acquisition unit 204 may shift the control parameter P acquired from the control unit 207 to generate the shifted control parameter P+Δp.

The control parameter acquisition unit 204 may shift the control parameter P supplied to the control object 101 in response to a change in the target value SP from the above-mentioned reference target value, and acquire a shifted control parameter P+Δp. In response to receiving a target value change signal from the target value acquisition unit 202, the control parameter acquisition unit 204 may shift the most recent control parameter P acquired from the control unit 207, and generate a shifted control parameter P+Δp.

In response to the target value SP being changed from the above-mentioned reference target value, the control parameter acquisition unit 204 may shift the control parameter P supplied to the control target 101 by a shift amount Δp corresponding to the target value SP to acquire a shifted control parameter P+Δp. The control parameter acquisition unit 204 may determine the shift amount Δp based on the target value SP indicated by the target value change signal. The shift amount Δp will be described in detail later.

The control parameter acquisition unit 204 may supply the acquired shifted control parameter P+Δp to the first supply unit 205. Note that, as an example in this embodiment, when the target value SP is a reference target value, the control parameter acquisition unit 204 supplies the control parameter P acquired from the control unit 207 to the first supply unit 205 without shifting it, but the control parameter P may be shifted by 0 and supplied to the first supply unit 205 as a shifted control parameter P+Δp.

<1.1.2-5. First supply unit 205>
The first supply unit 205 supplies the deviation acquired by the deviation acquisition unit 203 and the shifted control parameter P+Δp acquired by the control parameter acquisition unit 204 to the control model 206. A shifted control parameter P+Δp obtained by shifting the control parameter P supplied from the control unit 207 to the controlled object 101 and a deviation indicating an operating state as a result of controlling the controlled object 101 using the control parameter P are supplied to the control model 206. When the unshifted control parameter P is supplied from the control parameter acquisition unit 204, the first supplying unit 205 may supply the control parameter P and the deviation to the control model 206.

<1.1.2-6. Control model 206>
In response to input of the deviation and the control parameter P, the control model 206 outputs a recommended control parameter Pr recommended to be supplied to the controlled object 101. In response to input of the deviation and the control parameter P supplied to one controlled object 101, the control model 206 may output a recommended control parameter Pr recommended to be supplied to the one controlled object 101. The recommended control parameter Pr may indicate a recommended indicated value IV for the manipulated variable MV of the controlled object 101.

The control model 206 may be generated based on a reference value V (0, for example) for the control parameter P, and may output the value of the recommended control parameter Pr based on the reference value V in response to inputting the relative value of the control parameter P with respect to the reference value V together with the deviation. Inputting the deviation and the shifted control parameter P+Δp instead of inputting the deviation and the control parameter P to the control model 206 based on the reference value V may be equivalent to inputting the deviation and the control parameter P to another control model based on the reference value V-Δp. Such another control model may be generated for use in an environment different from that of the control model 206 in the conventional technology as shown in Patent Document 1.

In response to the input of the deviation and the control parameter P, the control model 206 may output a recommended control parameter Pr corresponding to a state in which the target value SP is a reference target value. In other words, the control model 206 may output a recommended control parameter Pr that is recommended in a state in which the target value is a reference target value. Inputting the deviation and the shifted control parameter P+Δp instead of inputting the deviation and the control parameter P to such a control model 206 may be equivalent to inputting the deviation and the control parameter P to another control model that outputs a recommended control parameter Pr that is suitable when the target value SP is a value different from the reference target value.

The control model 206 may output a recommended control parameter Pr to the control unit 207 in response to input of the deviation and the shifted control parameter P+Δp or the control parameter P from the first supply unit 205. In this embodiment, as an example, the deviation of the measurement value PV and the target value SP for one physical quantity is described as being input to the control model 206, but the deviations of the measurement value PV and the target value SP for multiple physical quantities may also be input. The control model 206 may have a change amount output model 2061 and an adder 2062.

<1.1.2-6(1). Change amount output model 2061>
In response to input of the deviation and the control parameter P, the change amount output model 2061 outputs a recommended change amount that recommends a change to the control parameter P. The change amount output model 2061 may supply the recommended change amount to the adder 2062. The recommended change amount may indicate a change amount that is recommended to be changed from the most recent control parameter P supplied to the controlled object 101. In the present embodiment, as an example, the recommended change amount may indicate a recommended change amount for the most recent indicated value IV for the manipulated variable MV. The change amount output model 2061 may be generated by a learning process by the learning processor 208, and may be stored in a storage unit (not shown).

<1.1.2-6(2). Addition unit 2062>
The adder 2062 calculates a recommended control parameter Pr by adding the control parameter P supplied to the control target 101 and the recommended change amount output from the change amount output model 2061. The adder 2062 may calculate the recommended control parameter Pr by adding the most recent control parameter P supplied from the control unit 207 and the recommended change amount supplied from the change amount output model 2061.

The adder 2062 may calculate the recommended control parameter Pr(t) at time t by adding the control parameter P _(t-1) at time t-1 and the recommended change amount Δu _(t) at time t, as shown in the following equation ₍₁₎ .
Pr _(t) = P _(t-1) + Δu _(t) (1)

The adder 2062 may store the control parameter P supplied from the control unit 207 and use it to calculate the recommended control parameter Pr. The adder 2062 may supply the calculated recommended control parameter Pr to the control unit 207.

<1.1.2-7. Control unit 207>
The control unit 207 is an example of an output unit, and outputs a recommended control parameter Pr output from the control model 206 in response to supply from the first supply unit 205 to the control model 206. As an example in the present embodiment, the control unit 207 may output the recommended control parameter Pr to the control object 101 as a control parameter P to control the control object 101. The control unit 207 may output the control parameter P input by an operator to the control object 101 to control the control object 101. The control unit 207 may output the control parameter P to the control object 101 in accordance with the control period of the control object 101.

The control unit 207 may store the control parameter P supplied to the control object 101 in a storage unit (not shown). The control unit 207 may store the control parameter P supplied to the control object 101 in the storage unit in association with the deviation acquired by the deviation acquisition unit 203. The control unit 207 may store the control parameter P supplied to the control object 101 in the storage unit in association with the deviation indicating the operating state resulting from controlling the control object 101 with the control parameter P.

<1.1.2-8. Learning processing unit 208>
The learning processing unit 208 is an example of a second learning processing unit, and performs learning processing of the change amount output model 2061 using learning data including the deviation acquired by the deviation acquisition unit 203 and the control parameter P acquired by the control parameter acquisition unit 204. The deviation and control parameter P included in the learning data may be the deviation and control parameter P acquired when the target value SP is the above-mentioned reference target value, and may be stored in association with each other in a storage unit (not shown). Note that the learning data may be acquired from a simulator (not shown) of the system 1 instead of being acquired from the actual system 1. The simulator may be created using actual measurement data of the facility 100 by any system identification technology.

The learning processing unit 208 may learn the change amount output model 2061 so as to output a recommended change amount recommended for increasing the reward value in response to the input of the deviation and the control parameter P. The recommended change amount may be a change amount recommended for increasing the reward value above a reference reward value (for example, a reward value obtained by inputting a value corresponding to the measurement value PV at that time into the reward function) corresponding to the state of the control object 101 at a predetermined time point (for example, the time point at which the deviation and the control parameter P are acquired) when the reference reward value is set as the reward value. As an example, the learning processing unit 208 may learn using an algorithm of the Kernel Dynamic Policy Programming (KDPP). The reward value may be a value determined by a preset reward function. The reward function may be a function based on the deviation, and as an example, a function in which the smaller the deviation, the larger the reward value. In addition, when the deviation acquisition unit 203 acquires deviations for each of multiple physical quantities, the reward function may be a function based on the sum of the multiple deviations, or may be a function based on the result of weighting and adding the multiple deviations.

According to the above-described device 200, the control model 206 outputs the recommended control parameter Pr in response to the input of the deviation between the measured value PV and the target value SP and the control parameter P, and the shifted control parameter P+Δp is supplied to the control model 206. Therefore, the recommended control parameter Pr output by inputting the control parameter P to another control model in which the reference value of the control parameter P is shifted by the shift amount -Δp from the control model 206 can be obtained from the control model 206. Therefore, by shifting the control parameter P in accordance with the change in the environment (the change in the target value as an example in this embodiment) and inputting it to the control model 206, the recommended control parameter Pr adapted to the changed environment can be obtained. In addition, by shifting the control parameter P, the recommended control parameter Pr output from another control model with a different reference value can be obtained from the control model 206, so that the device 200 can be made smaller than when multiple control models are built into the device 200.

In addition, in the control model 206, a recommended change amount for the control parameter P is output from the change amount output model 2061 according to the deviation and the control parameter P that has already been supplied to the control object 101, and the recommended change amount is added to the supplied control parameter P to calculate the recommended control parameter Pr. Therefore, even when the control model 206 is used as another control model by inputting the control parameter P to the control model 206 as the shifted control parameter P+ΔP, a recommended control parameter Pr based on the supplied control parameter P can be obtained. Therefore, an appropriate recommended control parameter Pr that is suited to the control object 101 can be obtained.

In addition, using learning data including the deviation acquired by the deviation acquisition unit 203 and the control parameter P acquired by the control parameter acquisition unit 204, a learning process is performed on the change amount output model 2061 so as to output a recommended change amount recommended for increasing the reward value determined by the reward function in response to the input of the deviation and the control parameter P. Therefore, an appropriate recommended control parameter Pr can be reliably acquired from the control model 206.

In addition, in response to the deviation and control parameter P being input to the control model 206, a recommended control parameter Pr corresponding to the state in which the target value SP is the reference target value is output from the control model 206, and in response to the target value SP being changed from the reference target value, a shifted control parameter P+Δp obtained by shifting the control parameter P is supplied to the control model 206. Therefore, using a single control model 206, it is possible to obtain a recommended control parameter Pr corresponding to a change in the target value SP.

In addition, in response to the target value SP being changed from the above-mentioned reference target value, the control parameter P supplied to the control target 101 is shifted by a shift amount Δp corresponding to the target value SP to obtain a shifted control parameter P+Δp. Therefore, using a single control model 206, it is possible to obtain a highly accurate recommended control parameter Pr corresponding to the change in the target value SP.

<1.2. Change amount output model 2061>
Fig. 2 shows an example of the change amount output model 2061. In Fig. 2 and Fig. 4 described later, the vertical axis indicates the control parameter P (as an example, the command value IV of the valve opening) and the horizontal axis indicates the deviation.

The change amount output model 2061 may show the correspondence between the combination of the deviation and the control parameter P and the recommended change amount. The change amount output model 2061 in this example may be an operation amount map that maps the correspondence between the combination of the deviation and the control parameter P and the recommended change amount. The operation amount map may be divided into multiple regions that correspond to different recommended change amounts according to the combination of the control parameter P and the deviation, and may output recommended change amounts that correspond to the coordinate position of the input combination of the control parameter P and the deviation. When such a change amount output model 2061 is used, the process becomes stable at a coordinate point where the deviation is 0 and the change amount is 0 (also called the equilibrium point O. In FIG. 2, as an example, the point where the deviation is 0 and the control parameter P is about 50).

The change amount output model 2061 may include information about the entire area of the operation amount map. Alternatively, the change amount output model 2061 may include only information indicating the boundaries of each area (for example, coordinates or function formulas indicating the boundaries) and the recommended change amount corresponding to each area. In this case, the storage area for storing the change amount output model 2061 can be reduced.

Figure 3 shows another example of the change amount output model 2061. As shown in this figure, the change amount output model 2061 may be a table that associates combinations of deviations and control parameters P with recommended change amounts.

1.3. Shift of control parameter P
FIG. 4 illustrates the effect of shifting the control parameter P input to the control model 206.

The manipulated variable map of the change amount output model 2061 according to this embodiment has a coordinate axis for the deviation, and even if the environment of the system 1 changes, for example because the target value SP is changed from the reference target value, the process remains stable on the line of deviation = 0. Therefore, the deviation of the equilibrium point O caused by a change in the environment can be eliminated by shifting the manipulated variable map in the direction of the coordinate axis of the control parameter P (the vertical axis direction in this figure is used as an example).

In the device 200 according to this embodiment, when a point Os different from the original equilibrium point O becomes the actual equilibrium point due to a change in the environment, the control parameter P is shifted by a shift amount Δp and input to the change amount output model 2061 of the control model 206. As a result, the operation amount map of the change amount output model 2061 shown in the upper part of the figure is shifted by -Δp in the coordinate axis direction of the control parameter P to become the operation amount map shown in the lower part of the figure, and a recommended change amount similar to that when the deviation and the control parameter P are input is output. Therefore, by shifting the control parameter P and inputting it to the control model 206, it is possible to obtain an output from another control model with a shifted operation amount map.

1.4. Shift amount
The control parameter acquisition unit 204 may shift the control parameter P supplied to the controlled object 101 by a shift amount Δp according to the set target value SP to generate a shifted control parameter P+Δp. For example, the control parameter acquisition unit 204 may determine the shift amount Δp by multiplying the difference between the target value SP and the reference target value by a preset coefficient. Alternatively or in addition to this, the control parameter acquisition unit 204 may determine the shift amount Δp using a preset relational expression that indicates the relationship between the value set in the target value SP and the value of the control parameter P when the measurement value PV stabilizes at that value (also referred to as the value of the control parameter P at the equilibrium point O).

The measurement value PV being stabilized at the target value SP may mean that after the control parameter P is output from the control unit 207 to the control object 101 and the first reference time has elapsed, the measurement value PV falls within a reference range over a second reference time. The reference range may be a range that includes the target value SP at the center. The first reference time and the second reference time may be set in advance according to the time constant of the control object 101. The first reference time and the second reference time may be different times from each other. The second reference time may be longer than the first reference time and may be longer than the control perimeter of the control object 101 by the control unit 207.

The above-mentioned coefficients and relational expressions used to determine the shift amount may be set from an approximation curve for multiple sample data including the measurement value PV at the equilibrium point O and the control parameter P at the equilibrium point O. If the measurement value PV at the equilibrium point O is taken as the target value SP of the measurement value PV, the approximation curve may indicate the relationship between the target value SP and the value of the control parameter P at the equilibrium point O. The function of the reference time curve may be set in advance as the above-mentioned relational expression used to determine the shift amount. If the approximation curve is a linear function, the ratio of the change in the control parameter P to the change in the target value SP, i.e., a value indicating the slope of the approximation curve, may be set in advance as the above-mentioned coefficient used to determine the shift amount.

Each sample data may include the value of the control parameter P when the step response of the measurement value PV stabilizes to a single value while maintaining the control parameter P supplied to the control object 101 at a constant value, and the value of the measurement value PV. Alternatively, each sample data may include the value of the control parameter P when the measurement value PV stabilizes to the target value SP by setting a target value SP and controlling the control object 101 by PID control, and the target value SP. Each sample data may be obtained by actually operating the control object 101, or may be obtained by a simulator. An approximation curve of the sample data may be calculated by linear regression such as the least squares method, or may be calculated by nonlinear regression such as polynomial approximation or Gaussian process regression.

Figure 5 shows an approximation curve of sample data. The horizontal axis (x-axis) in the figure shows the target value SP (or the measurement value PV at the equilibrium point O), and the vertical axis (y-axis) shows the value of the control parameter P at the equilibrium point. In the example of this figure, the approximation curve may be y = 0.8165x - 0.0002, and the target value SP may be changed from 50 as the reference target value to 80. In this case, the control parameter acquisition unit 204 may multiply 30, which is the difference between the target value SP and the reference target value, by 0.8165, which is a preset coefficient, to determine the shift amount Δp as 24.495. Alternatively, the control parameter acquisition unit 204 may use a preset function formula y = 0.8165x - 0.0002 to calculate the value of the control parameter P as 40.8248 (= 0.8165 x 50) when the target value SP is the reference target value of 50, and the value of the control parameter P as 65.3198 (= 0.8165 x 80) when the target value SP is 80, and determine the shift amount Δp to be 24.495 from the difference between the two. Note that the value of the control parameter P when the target value SP is the reference target value (40.8248 in this example) may be stored in advance in the control parameter acquisition unit 204.

With the device 200 having the control parameter acquisition unit 204 described above, the shift amount is determined by multiplying the difference between the target value SP and the reference target value by a preset coefficient, so that a highly accurate recommended control parameter Pr can be obtained that matches the changed target value SP.

In addition, since the shift amount is determined using a predefined relational equation that indicates the relationship between the value set for the target value SP and the value of the control parameter P when the measurement value PV stabilizes at that value, the shift amount can be determined by inputting the target value SP into the function equation. Therefore, it is possible to easily obtain a highly accurate recommended control parameter Pr that matches the changed target value SP.

<1.5. Operation>
6 shows the operation of the device 200. The device 200 may control the control target 101 by performing the processes of steps S11 to S29. Note that this operation is performed in response to the device 200 being started up. In addition, the learning process of the change amount output model 2061 may be completed at the start of the operation.

In step S11, the target value acquisition unit 202 acquires a target value SP for the state related to the control object 101. The target value acquisition unit 202 may acquire a preset reference target value as the target value SP, or may acquire a value different from the reference target value as the target value SP. When the process of step S11 is executed for the first time, the target value acquisition unit 202 may acquire the reference target value as the target value SP. In response to acquiring a value different from the reference target value as the target value SP, the target value acquisition unit 202 may output a target value change signal indicating the set target value SP.

In step S13, the measurement value acquisition unit 201 acquires the measurement value PV of the state of the control object 101. The target value acquisition unit 202 may acquire the measurement value PV from the sensor 102 of the equipment 100.

In step S15, the deviation acquisition unit 203 acquires the deviation between the target value SP acquired in step S11 and the measurement value PV acquired in step S13.

In step S17, the control parameter acquisition unit 204 acquires the control parameter P supplied to the control object 101. The control parameter acquisition unit 204 may acquire the control parameter P supplied to the control object 101 in the most recent control cycle from the control unit 207. As an example, the control parameter acquisition unit 204 may acquire and temporarily store the control parameter P output from the control unit 207 to the control object 101 in the processing of step S27 described below, and read out the control parameter P in step S17. When step S17 is executed for the first time, that is, when the processing of step S27 has not been executed, the control parameter acquisition unit 204 may acquire the initial value of the control parameter P that has been set in advance.

In step S19, the control parameter acquisition unit 204 determines whether the target value SP acquired in step S11 is a reference target value. The control parameter acquisition unit 204 may make this determination based on whether a target value change signal has been acquired from the target value acquisition unit 202. If it is determined in step S19 that the target value SP is the reference target value (step S19; Yes), the process may proceed to step S25. If it is determined in step S19 that the target value SP is not the reference target value (step S19; No), the process may proceed to step S21.

In step S21, the control parameter acquisition unit 204 determines a shift amount Δp of the control parameter P. The control parameter acquisition unit 204 may determine the shift amount Δp according to the target value SP acquired in step S11.

In step S23, the control parameter acquisition unit 204 shifts the control parameter P acquired in step S17 by the determined shift amount Δp to acquire the shifted control parameter P+Δp.

In step S25, the first supply unit 205 supplies the control parameter P supplied from the control parameter acquisition unit 204 and the deviation supplied from the deviation acquisition unit 203 to the control model 206. If the target value SP is determined to be the reference target value in step S19, the first supply unit 205 may supply the control parameter P acquired by the control parameter acquisition unit 204 in step S17 to the control model 206 as is. If the target value SP is determined to not be the reference target value in step S19, the first supply unit 205 may supply the control parameter P shifted in step S23, i.e., the shifted control parameter P+Δp, to the control model 206.

As a result, a recommended control parameter Pr corresponding to the input control parameter P and deviation is output from the control model 206. As an example in this embodiment, a recommended change amount corresponding to the input control parameter P and deviation is output from the change amount output model 2061, and the recommended change amount and the control parameter P acquired in step S17 are added by the adder 2062 to generate the recommended control parameter Pr.

In step S27, the control unit 207 outputs the recommended control parameters Pr from the control model 206. The control unit 207 may supply the recommended control parameters Pr to the control object 101 as control parameters P to control the control object 101.

In step S29, the target value acquisition unit 202 determines whether the target value SP is changed by the operator. If it is determined that the target value SP is not changed (step S29; No), the process may proceed to step S13. If it is determined that the target value SP is changed (step S29; Yes), the process may proceed to step S11.

<1.6. Operation example>
7 shows the transition of the measurement value PV and the control parameter P when the target value SP is changed. The horizontal axis in the figure indicates time (seconds), and the vertical axis indicates the values of the measurement value PV and the control parameter P. Note that in this figure, as an example, the control parameter P indicates the indication value IV of the valve opening, and the measurement value PV and the control parameter P are normalized within the range of 0 to 100. As shown in this figure, in the device 200 according to this embodiment, even when the target value SP is changed, the controlled object 101 is controlled so that the measurement value PV coincides with the changed target value SP.

<2. Second embodiment>
<2.1. System 1A>
Fig. 8 shows a system 1A according to the second embodiment. Components that are substantially the same as those in the system 1 shown in Fig. 1 are given the same reference numerals, and descriptions thereof will be omitted. The system 1A includes an apparatus 200A. The apparatus 200A may include a deviation acquisition unit 203A, an unstable state detection unit 211A, a second supply unit 212A, a shift amount output model 213A, a control parameter acquisition unit 204A, and a learning processing unit 214A.

<2.1.1. Deviation Acquisition Unit 203A>
The deviation acquisition unit 203A acquires the deviation between the measured value PV and the target value SP of the state of the control target 101 in the same manner as the deviation acquisition unit 203 in the first embodiment described above. For example, each time the deviation is acquired, the deviation acquisition unit 203A calculates the rate of change of the deviation by dividing the amount of change between the most recent two deviations by the interval between the acquisition timings. The deviation acquisition unit 203A may supply the acquired deviation to the first supply unit 205 and the unstable state detection unit 211A. The deviation acquisition section 203A may supply the deviation and the rate of change therein to the supply section 212A. The deviation acquisition section 203A may store the acquired deviation and the rate of change therein in a storage section (not shown).

<2.1.2. Unstable state detection unit 211A>
The unstable state detection unit 211A is an example of a first detection unit, and detects that the deviation acquired by the deviation acquisition unit 203A after the control target 101 is controlled by the recommended control parameter Pr is not stable within the reference range. The reference range may be a range of any size including 0 at the center. The deviation not being stable within the reference range may mean that the deviation is outside the reference range at least at one point within the second reference time after the control parameter P is output from the control unit 207 to the control target 101 and the first reference time has elapsed. The situation in which the deviation is not stable within the reference range may be caused by a change in the external environment of the equipment 100 (also referred to as a disturbance). In response to detecting that the deviation is not stable within the reference range, the unstable state detection unit 211A may supply a signal indicating that fact (also referred to as an unstable state detection signal) to the control parameter acquisition unit 204A.

<2.1.3. Second supply unit 212A>
The second supply unit 212A supplies the deviation acquired by the deviation acquisition unit 203A and the rate of change of the deviation to the shift amount output model 213A. The reference interval may be equal to or longer than the control period of the control target 101 by the control unit 207. The reference interval may be set to a length according to the time constant of the control target 101 (for example, the time The length of the byte may be a constant multiplied by an integer.

<2.1.4. Shift amount output model 213A>
In response to input of the deviation and the rate of change of the deviation, the shift amount output model 213A outputs a recommended shift amount Δpr that recommends a shift of the control parameter P by the control parameter acquisition unit 204. In response to input of the deviation and the rate of change from the second supply unit 212A, the shift amount output model 213A may output the recommended shift amount Δpr to the control parameter acquisition unit 204A.

The shift amount output model 213A may indicate the correspondence between the combination of the deviation and the rate of change of the deviation, and the recommended shift amount Δpr. As an example, the shift amount output model 213A may be a shift amount map that maps the correspondence between the combination of the deviation and the rate of change, and the recommended shift amount Δpr. Alternatively, the shift amount output model 213A may be a table that associates the combination of the deviation and the rate of change with the recommended shift amount Δpr. The shift amount output model 213A may be generated by a learning process by the learning processing unit 214A, and may be stored in a storage unit (not shown).

<2.1.5. Control parameter acquisition unit 204A>
The control parameter acquisition unit 204A acquires the shifted control parameter P+Δp in the same manner as the control parameter acquisition unit 204 in the first embodiment.

In addition, in response to detection by the unstable state detection unit 211A that the deviation acquired by the deviation acquisition unit 203A is not stable within the reference range, the control parameter acquisition unit 204A may shift the control parameter P supplied to the control target 101 (the control parameter P supplied from the control unit 207 in this embodiment as an example) to acquire the shifted control parameter P+Δp.

When the unstable state detection unit 211A detects that the deviation acquired by the deviation acquisition unit 203A is not stable within the reference range, and when a supply is made from the second supply unit 212A to the shift amount output model 213A, the control parameter acquisition unit 204A may shift the control parameter P supplied to the control object 101 by the recommended shift amount Δpr output from the shift amount output model 213A.

The control parameter acquisition unit 204A may supply the acquired shifted control parameter P+Δp to the first supply unit 205. Note that, as an example in this embodiment, when the target value SP is a reference target value and the deviation is stable at the target value SP, the control parameter acquisition unit 204A supplies the control parameter P acquired from the control unit 207 to the first supply unit 205 without shifting it, but the control parameter P may be shifted by 0 to supply it to the first supply unit 205 as a shifted control parameter P+Δp.

<2.1.6. Learning processing unit 214A>
The learning processing unit 214A is an example of a first learning processing unit, and performs learning processing of the shift amount output model 213A using learning data including the deviation acquired by the deviation acquiring unit 203A, the rate of change of the deviation, and the shift amount Δp of the control parameter P acquired by the control parameter acquiring unit 204A. The deviation and the rate of change included in the learning data may be the deviation and the rate of change acquired when the target value SP is a reference target value.

The learning data may be generated by a simulator (not shown) of the system 1, and may include a combination of the deviation acquired by the deviation acquisition unit 203A in a simulation in which an arbitrary disturbance is artificially applied to the equipment 100, the rate of change of the deviation, and the shift amount Δp of the control parameter P shifted by the control parameter acquisition unit 204A. The simulator may be created using actual measurement data of the equipment 100 by any system identification technique. After the shift amount output model 213A is generated, the learning data may include the deviation when the measurement value PV does not stabilize at the target value SP when the equipment 100 is actually operated, the rate of change of the deviation, and the shift amount Δp of the control parameter P by the control parameter acquisition unit 204A.

The learning processing unit 214A may learn the shift amount output model 213A so as to output a recommended shift amount Δpr recommended for increasing the reward value in response to the input of the deviation and the change rate of the deviation. The recommended shift amount Δpr may be a shift amount recommended for increasing the reward value higher than the reference reward value when the reward value (for example, the reward value obtained by inputting a value corresponding to the measurement value PV at that time into the reward function) corresponding to the state of the control object 101 at a predetermined time point (for example, the time point at which the deviation and the change rate are acquired) is set as the reference reward value. As an example, the learning processing unit 214A may learn using an algorithm of the kernel dynamic policy programming method. The reward value may be a value determined by a preset reward function. The reward function may be a function based on the deviation, and as an example, a function in which the smaller the deviation, the larger the reward value. Note that, when the deviation acquisition unit 203A acquires deviations for each of the multiple physical quantities, the reward function may be a function based on the sum of the multiple deviations, or may be a function based on the result of weighted addition of the multiple deviations.

According to the above-described device 200A, in response to detection that the acquired deviation is not stable at the target value SP, the control parameter P supplied to the control object 101 is shifted to acquire the shifted control parameter P+Δp. Therefore, a single control model 206 can be used to acquire the recommended control parameter Pr in response to the occurrence of a disturbance. In addition, since the recommended control parameter Pr can be acquired even when a disturbance occurs, the robustness of the device 200A can be improved.

The control parameter P is shifted by the recommended shift amount Δpr output from the shift amount output model 213A according to the deviation and the rate of change of the deviation. Therefore, by generating the shift amount output model 213A in advance, the recommended control parameter Pr when a disturbance occurs can be easily obtained.

In addition, the deviation and the rate of change of the deviation are supplied to the shift amount output model 213A for each reference interval, so that the shift amount Δp can be changed according to the disturbance situation and an appropriate recommended control parameter Pr can be obtained.

In addition, using learning data including the deviation, the rate of change of the deviation, and the shift amount Δp of the control parameter P, the shift amount output model 213A performs a learning process to output a recommended shift amount Δpr that is recommended to increase the reward value determined by a default reward function in response to the input of the deviation and the rate of change of the deviation. Therefore, an appropriate recommended shift amount Δpr can be reliably obtained from the shift amount output model 213A.

2.2. Shift amount output model 213A
9 shows an example of the shift amount output model 213 A. In FIG. 9, the vertical axis indicates the rate of change of the deviation, and the horizontal axis indicates the deviation.

The shift amount output model 213A may indicate the correspondence between the combination of deviation and change rate and the recommended shift amount. In this example, the shift amount output model 213A may be a shift amount map that maps the correspondence between the combination of deviation and control parameter P and the recommended shift amount Δpr. The shift amount output model 213A may be divided into multiple regions that correspond to different recommended shift amounts Δpr according to the combination of deviation and change rate, and may output the recommended shift amount Δpr that corresponds to the coordinate position of the input combination of deviation and change rate.

The shift amount output model 213A may include information about the entire area of the shift amount map. Alternatively, the shift amount output model 213A may include only information indicating the boundaries of each area (for example, coordinates or a function indicating the boundaries) and the recommended shift amount Δpr corresponding to the area. In this case, the storage area for storing the shift amount output model 213A can be reduced.

Figure 10 shows another example of the shift amount output model 213A. As shown in this figure, the shift amount output model 213A may be a table that associates combinations of deviation and rate of change with the recommended shift amount Δpr.

<2.3. Operation>
11 shows the operation of the device 200A. The device 200A may control the control target 101 by performing the processes of steps S11 to S45. Note that this operation is performed in response to the device 200A being started. At the start of the operation, the learning process of the change amount output model 2061 and the shift amount output model 213A may be completed. The operation of the device 200A according to the second embodiment may be performed in the same manner as in the first embodiment. 10. This differs from the operation of the apparatus 200 in that steps S31 to S45 are performed between steps S27 and S29.

In step S31, the measurement value acquisition unit 201 acquires the measurement value PV of the state of the control object 101. The target value acquisition unit 202 may acquire the measurement value PV in the same manner as in step S13.

In step S33, the deviation acquisition unit 203A acquires the deviation between the target value SP acquired in step S11 and the measurement value PV acquired in step S31. The deviation acquisition unit 203A further acquires the rate of change of the acquired deviation.

In step S35, the unstable state detection unit 211A determines whether the deviation stabilizes within the reference range. After the control parameter P is output from the control unit 207 to the control object 101 by the processing of step S27 and the first reference time has elapsed, the unstable state detection unit 211A may detect that the deviation does not stabilize within the reference range in response to the deviation being outside the reference range at least at one point within the second reference time, and make a determination to that effect.

If it is determined in step S35 that the deviation is stable within the reference range (step S35; Yes), the process may proceed to step S29. If it is determined in step S35 that the deviation is not stable within the reference range (step S35; No), the process may proceed to step S37.

In step S37, the control parameter acquisition unit 204A determines a shift amount Δp of the control parameter P. In response to the deviation and rate of change acquired in step S33 being supplied to the shift amount output model 213A, the control parameter acquisition unit 204A may determine the recommended shift amount Δpr output from the shift amount output model 213A as the shift amount Δp.

In step S39, the control parameter acquisition unit 204A acquires the control parameters P supplied to the control object 101. The control parameter acquisition unit 204A may acquire the control parameters P supplied to the control object 101 in the most recent control cycle from the control unit 207A. As an example, the control parameter acquisition unit 204A may acquire and temporarily store the control parameters P output from the control unit 207 to the control object 101 in the most recently executed process of step S27 or step S45 described below, and read them out in step S39.

In step S41, the control parameter acquisition unit 204A shifts the control parameter P acquired in step S39 by the shift amount Δp determined in step S37 to acquire the shifted control parameter P+Δp.

In step S43, the first supply unit 205 supplies the shifted control parameter P+Δp obtained in step S41 and the deviation obtained in step S33 to the control model 206.

In step S45, the control unit 207 outputs the recommended control parameters Pr from the control model 206. The control unit 207 may supply the recommended control parameters Pr to the control object 101 as control parameters P to control the control object 101. When the processing of step S45 is completed, the processing may proceed to step S31.

<2.4. Operation example>
12 shows the transition of the measurement value PV and the control parameter P when a disturbance occurs. The horizontal axis in the figure indicates time (seconds), and the vertical axis indicates the values of the measurement value PV and the control parameter P. Note that in this figure, as an example, the control parameter P indicates the indication value IV of the valve opening, and the measurement value PV and the control parameter P are normalized within the range of 0 to 100. As shown in this figure, in the device 200A according to this embodiment, even when a disturbance occurs, the controlled object 101 is controlled so that the measurement value PV coincides with the target value SP.

<3. Third embodiment>
<3.1. System 1B>
FIG. 13 shows a system 1B according to the third embodiment. The same reference numerals are used for the parts that are substantially the same as those in the systems 1 and 1A shown in FIG. 1 and FIG. 2, and the description thereof is omitted. The system 1B includes an apparatus 200B. The apparatus 200B according to this embodiment is capable of changing the controlled object 101. The controlled object 101 before and after the change may have similar process dynamic characteristics, and may be a first-order lag system or a second-order lag system. In other words, the controlled object 101 before and after the change may have the same order of the denominator of the transfer function indicating the relationship between the input and output. The apparatus 200B may include an object information acquisition unit 221B, a control unit 207B, and a control parameter acquisition unit 204B.

<3.1.1. Target information acquisition unit 221B>
The target information acquisition unit 221B is an example of a second detection unit, and detects that the control target 101 has been changed. The target information acquisition unit 221B may detect that the control target 101 has been changed by acquiring identification information of the changed control target 101 from the operator every time the control target 101 is changed. In response to the change in the control target 101, the target information acquisition unit 221B may supply a signal indicating the identification information of the changed control target 101 (also referred to as a control target change signal) to the control unit 207B and the control parameter acquisition unit 204B.

<3.1.2. Control unit 207B>
Similar to the control unit 207B in the first embodiment, in response to a supply from the first supply unit 205 to the control model 206, the control unit 207B outputs the recommended control parameters Pr output from the control model 206 as control parameters P. In response to receiving a control target change signal from the target information acquisition unit 221B, the control unit 207B may output the control parameters P to the changed control target 101 to control the changed control target 101.

<3.1.3. Control parameter acquisition unit 204B>
The control parameter acquisition unit 204B acquires the shifted control parameter P+Δp in the same manner as the control parameter acquisition unit 204A in the second embodiment.

In addition, in response to detection by the target information acquisition unit 221B that the control target 101 has been changed, the control parameter acquisition unit 204B may shift the control parameter P supplied to the control target 101 (the control parameter P supplied from the control unit 207 as an example in this embodiment) to acquire a shifted control parameter P+Δp. The control parameter P supplied to the control target 101 may be the control parameter P supplied to the control target 101 before the change, or may be the control parameter P supplied to the control target 101 after the change.

When the control object 101 is changed, the control parameter acquisition unit 204B may determine the shift amount Δp based on the difference between the values of the control parameter P at the equilibrium point of each control object 101 before and after the change.

For example, the control parameter acquisition unit 204B may determine, as the shift amount Δp, the difference between the values of the control parameter P at the equilibrium point O acquired when the target value SP is set to a reference target value in advance and the control object 101 before and after the change are used. In this case, the control parameter acquisition unit 204B may store in advance, for each control object 101, the value of the control parameter P at the equilibrium point O when the target value SP is set to a reference target value.

Alternatively, the control parameter acquisition unit 204B may determine, as the shift amount Δp, the difference between the values of the control parameter P at the equilibrium point O acquired when the target value SP is set in advance to the same value as the current setting value and the control object 101 before and after the change are used. In this case, the control parameter acquisition unit 204B may store in advance, for each control object 101, a relational expression showing the relationship between the target value SP and the control parameter P at the equilibrium point O, and may calculate the value of the control parameter P at the equilibrium point corresponding to the current target value SP from the function expression.

In the above-mentioned step S19, when the target value SP is the reference target value and the control object 101 has been changed, the control parameter acquisition unit 204B may determine a shift amount Δp based on the difference between the values of the control parameter P at the equilibrium point of each control object 101 before and after the change, and supply the shifted control parameter P+Δp to the first supply unit 205. When the target value SP is changed from the reference target value after the control object 101 is changed, the control parameter acquisition unit 204B may determine the shift amount Δp using the above-mentioned relational expression showing the relationship between the target value SP and the value of the control parameter P at the equilibrium point O for the changed control object 101.

In this embodiment, as an example, when the target value SP is a reference target value, the deviation is stable at the target value SP, and the control target 101 is not changed, the control parameter acquisition unit 204B may supply the control parameter P acquired from the control unit 207 to the first supply unit 205 without shifting it. Alternatively, the control parameter acquisition unit 204 may supply the control parameter P to the first supply unit 205 as a shifted control parameter P+Δp, which is obtained by shifting the control parameter P by 0.

According to the above-described device 200B, in response to a change in the control object 101, the control parameter P supplied to the control object 101 is shifted to obtain the shifted control parameter P+Δp. Therefore, since a single control model 206 can be used to obtain the recommended control parameter Pr in response to a change in the control object 101, the versatility of the device 200B can be improved and the device 200B can be made smaller than when a separate control model is used for each control object 101.

In the control model 206, the recommended change amount Δpr of the control parameter P is output from the change amount output model 2061 according to the deviation and the control parameter P already supplied to the control object 101, and the recommended change amount Δpr is added to the already supplied control parameter P to calculate the recommended control parameter Pr. Therefore, even if the control model 206 is used for another control object 101 by inputting the control parameter P to the control model 206 as the shifted control parameter P+ΔP, it is possible to obtain a recommended control parameter Pr based on the already supplied control parameter P. Therefore, regardless of the difference in process gain for each control object 101, it is possible to obtain an appropriate recommended control parameter Pr that is suitable for the changed control object 101.

3. Modifications
In the above first to third embodiments, the control model 206 has been described as having the change amount output model 2061 and the adder 2062, but as long as the recommended control parameter Pr is output in response to the input of the deviation and the control parameter P, the control model 206 may not have these. In this case, the control model 206 may be a learning model generated by an algorithm such as a kernel dynamic policy programming method, deep reinforcement learning, a support vector machine, a logistic regression, a decision tree, or a neural network. The learning processing unit 208 may perform learning processing of the control model 206 using learning data including the deviation acquired by the deviation acquisition unit 203 and the control parameter P acquired by the control parameter acquisition unit 204.

In addition, the change amount output model 2061 and the shift amount output model 213A have been described as maps and tables generated by the learning algorithm of the kernel dynamic policy programming method, but they may be generated by other algorithms such as deep reinforcement learning, support vector machines, logistic regression, decision trees, and neural networks, or may be models of other forms different from maps and tables.

Furthermore, although it has been described that the deviation and the control parameter P are input to the change amount output model 2061, other values may also be input. Similarly, although it has been described that the deviation and the rate of change are input to the shift amount output model 213A, other values may also be input. The other values may be, for example, the differential value or integral value of the measurement value by the sensor 102.

In addition, although the devices 200, 200A, and 200B have been described as having a measurement value acquisition unit 201, a target value acquisition unit 202, and a learning processing unit 208, they may not have any of these. If the devices 200, 200A, and 200B do not have the measurement value acquisition unit 201 and the target value acquisition unit 202, the deviation acquisition units 203 and 203A may acquire a deviation calculated by an external device. If the devices 200, 200A, and 200B do not have the learning processing unit 208, they may have a change amount output model 2061 that has been learned in advance by an external device.

In addition, although the control parameter acquisition units 204, 204A, and 204B have been described as calculating and acquiring the shifted control parameter P+Δp, the shifted control parameter P+Δp calculated outside the devices 200, 200A, and 200B may be acquired.

In addition, in the second and third embodiments, the devices 200A and 200B are described as having the learning processing unit 214A, but they may not have the learning processing unit 214A. In this case, the devices 200A and 200B may have a shift amount output model 213A that has been trained in advance by an external device.

In the second embodiment described above, the control parameter acquisition unit 204 shifts the control parameter P when the target value SP is changed from the reference target value and when the deviation is not stabilized within the reference range. However, it is not necessary to shift the control parameter P when the target value SP is changed from the reference target value.

Similarly, in the above third embodiment, the control parameter acquisition unit 204 has been described as shifting the control parameter P when the target value SP is changed from the reference target value, when the deviation is not stabilized within the reference range, and when the control target 101 is changed, but it is not necessary to shift the control parameter P when at least one of the cases is when the target value SP is changed from the reference target value and when the deviation is not stabilized within the reference range.

Various embodiments of the present invention may also be described with reference to flow charts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of an apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable medium, and/or a processor provided with computer readable instructions stored on a computer readable medium. Dedicated circuitry may include digital and/or analog hardware circuitry and may include integrated circuits (ICs) and/or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

A computer-readable medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, and the like.

The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.

The computer-readable instructions may be provided to a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc., to execute the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

14 shows an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part. A program installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to an embodiment of the present invention, and/or to perform a process or steps of the process according to an embodiment of the present invention. Such a program may be executed by the CPU 2212 to cause the computer 2200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input/output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input/output controller 2220. The computer also includes legacy input/output units such as a ROM 2230 and a keyboard 2242, which are connected to the input/output controller 2220 via an input/output chip 2240.

The CPU 2212 operates according to the programs stored in the ROM 2230 and the RAM 2214, thereby controlling each unit. The graphics controller 2216 retrieves image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into itself, and causes the image data to be displayed on the display device 2218.

The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and/or writes programs and data to an IC card.

ROM 2230 stores therein a boot program, etc., executed by computer 2200 upon activation, and/or a program that depends on the hardware of computer 2200. I/O chip 2240 may also connect various I/O units to I/O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. The information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by realizing the manipulation or processing of information according to the use of the computer 2200.

For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

The CPU 2212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and may undergo information processing. CPU 2212 may perform various types of processing on data read from RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search/replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to RAM 2214. CPU 2212 may also search for information in a file, database, etc. in the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, CPU 2212 may search for an entry that matches a condition in which an attribute value of the first attribute is specified from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

The above-described program or software module may be stored on a computer-readable medium on the computer 2200 or in the vicinity of the computer 2200. Also, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network.

The present invention has been described above using an embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiment. It is clear from the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and it should be noted that the processes may be performed in any order, unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is necessary to perform the processes in this order.

REFERENCE SIGNS LIST 1 System 100 Equipment 101 Control target 102 Sensor 200 Device 201 Measurement value acquisition unit 202 Target value acquisition unit 203 Deviation acquisition unit 204 Control parameter acquisition unit 205 First supply unit 206 Control model 207 Control unit 208 Learning processing unit 211 Unstable state detection unit 212 Second supply unit 213 Shift amount output model 214 Learning processing unit 221 Target information acquisition unit 2061 Change amount output model 2062 Addition unit 2200 Computer 2201 DVD-ROM
2210 host controller 2212 CPU
2214 RAM
2216 Graphic controller 2218 Display device 2220 Input/output controller 2222 Communication interface 2224 Hard disk drive 2226 DVD-ROM drive 2230 ROM
2240 Input/Output Chip 2242 Keyboard

Claims (14)

a deviation acquisition unit for acquiring a deviation between a measured value of a state of a control object and a target value;
a control parameter acquisition unit that acquires shifted control parameters by shifting the control parameters supplied to the control target;
a first supply unit that supplies the deviation acquired by the deviation acquisition unit and the shifted control parameter acquired by the control parameter acquisition unit to a control model that outputs a recommended control parameter to be recommended to be supplied to the controlled object in response to an input of the deviation and the control parameter;
an output unit that outputs the recommended control parameters output from the control model in response to the supply from the first supply unit to the control model;
An apparatus comprising: A target value acquisition unit that acquires a target value of the state,
The device according to claim 1 , wherein the control parameter acquisition unit shifts the control parameters supplied to the controlled object in response to the target value being changed from a reference target value, and acquires the shifted control parameters. The device according to claim 2, wherein the control parameter acquisition unit, in response to the target value being changed from the reference target value, shifts the control parameters supplied to the control target by a shift amount corresponding to the target value, and acquires the shifted control parameters. The device according to claim 3, wherein the control parameter acquisition unit determines the shift amount by multiplying the difference between the target value and the reference target value by a preset coefficient. The device according to claim 3, wherein the control parameter acquisition unit determines the shift amount using a pre-set relational expression that indicates the relationship between a value set for the target value and the value of the control parameter when the measured value stabilizes at that value. a first detection unit that detects that the deviation acquired by the deviation acquisition unit is not stable within a reference range after the control target is controlled by the recommended control parameters,
The device according to any one of claims 1 to 3, wherein the control parameter acquisition unit shifts the control parameter supplied to the controlled object in response to detection by the first detection unit that the deviation acquired by the deviation acquisition unit is not stable within the reference range, and acquires the shifted control parameter. a second supply unit that supplies the deviation acquired by the deviation acquisition unit and the change rate of the deviation to a shift amount output model that outputs a recommended shift amount that recommends a shift of the control parameter by the control parameter acquisition unit in response to input of the deviation and the change rate of the deviation,
7. The device according to claim 6, wherein the control parameter acquisition unit shifts the control parameters supplied to the controlled object by the recommended shift amount output from the shift amount output model in response to the first detection unit detecting that the deviation acquired by the deviation acquisition unit is not stable within the reference range and the second supply unit supplying the shift amount to the shift amount output model, thereby acquiring the shifted control parameters. The device according to claim 7, wherein the second supply unit supplies the shift amount output model for each reference interval. The device according to claim 7, further comprising a first learning processing unit that performs learning processing of the shift amount output model using learning data including the deviation acquired by the deviation acquisition unit, the rate of change of the deviation, and the shift amount of the control parameter acquired by the control parameter acquisition unit, so as to output the recommended shift amount recommended to increase a reward value determined by a preset reward function in response to input of the deviation and the rate of change of the deviation. A second detection unit that detects that the control target has been changed,
The device according to claim 1 , wherein the control parameter acquisition unit shifts the control parameters supplied to the control object in response to detection by the second detection unit that the control object has been changed, and acquires the shifted control parameters. The control model is
a change amount output model that outputs a recommended change amount that recommends a change to the control parameter in response to input of the deviation and the control parameter;
an adder that calculates the recommended control parameter by adding the control parameter supplied to the controlled object and the recommended change amount output from the change amount output model;
The apparatus of claim 1 , further comprising: The device according to claim 11, further comprising a second learning processing unit that performs learning processing of the change amount output model using learning data including the deviation acquired by the deviation acquisition unit and the control parameters acquired by the control parameter acquisition unit, so as to output the recommended change amount recommended for increasing a reward value determined by a preset reward function in response to input of the deviation and the control parameters. A deviation acquisition step of acquiring a deviation between a measured value of a state of a control object and a target value;
a control parameter acquisition step of acquiring shifted control parameters by shifting the control parameters supplied to the control object;
a first supply step of supplying the deviation acquired in the deviation acquisition step and the shifted control parameter acquired in the control parameter acquisition step to a control model that outputs a recommended control parameter to be recommended to be supplied to the controlled object in response to an input of the deviation and the control parameter;
an output step of outputting the recommended control parameters output from the control model in response to the supply to the control model by the first supply step;
A method for providing the above. Computer,
a deviation acquisition unit for acquiring a deviation between a measured value of a state of a control object and a target value;
a control parameter acquisition unit that acquires shifted control parameters by shifting the control parameters supplied to the control target;
a first supply unit that supplies the deviation acquired by the deviation acquisition unit and the shifted control parameter acquired by the control parameter acquisition unit to a control model that outputs a recommended control parameter to be recommended to be supplied to the controlled object in response to an input of the deviation and the control parameter;
an output unit that outputs the recommended control parameters output from the control model in response to the supply from the first supply unit to the control model.

Images