JP7517313B2 - Apparatus, method and program - Google Patents

Description

The present invention relates to an apparatus, a method and a program.

Patent Document 1 states that "in response to input of measurement data, a learning process for a first model is executed to output recommended control parameters indicating a first type of control content recommended for increasing a reward value determined by a preset reward function."
[Prior Art Literature]
[Patent Documents]
[Patent Document 1] JP 2021-086283 A [Patent Document 2] JP 2020-027556 A [Patent Document 3] JP 2019-020885 A [Non-Patent Document]
[Non-Patent Document 1] Go Takami, "Realizing Plant Control AI," Yokogawa Technical Report, Yokogawa Electric Corporation, 2020, Vol. 63, No. 1, pp. 33-36
[Non-Patent Document 2] Takuji Imai, "Yokogawa Electric and NAIST Achieve Reinforcement Learning for Chemical Plants, Advanced Control with Fewer Trials," Nikkei Robotics, Nikkei BP, March 2019 issue

In a first aspect of the present invention, an apparatus is provided. The apparatus may include a supply unit that supplies a state parameter value to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of a state parameter related to the equipment. The apparatus may include a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the state parameter value to the operation model. The apparatus may include an acquisition unit that acquires a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition unit. The apparatus may include an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value corresponding to a result of operating the equipment by manual operation.

The reference evaluation value may be calculated based on the results of manual operations input into an equipment simulator.

The model evaluation value may be calculated based on the results of inputting the recommended values acquired by the control parameter acquisition unit into the equipment simulator.

The model evaluation value may be calculated based on whether the parameters for the equipment operated with the recommended values fall within the target range. The reference evaluation value may be calculated based on whether the parameters for the equipment operated by manual operation fall within the target range.

The device may further include a target range acquisition unit that acquires a target range set by an operator for a selected parameter selected by the operator from among multiple types of parameters related to the equipment.

The device may further include a display control unit that displays the value of a selected parameter from past operations of the equipment in response to the selection of the selected parameter from multiple types of parameters.

The display control unit may display the values of each selected parameter from past operations of the equipment in a coordinate space with each selected parameter as the coordinate axis.

The equipment may be equipment for manufacturing an item. The equipment parameter may be at least one of an index value indicating the quality of the item or the production volume of the item.

The device may further include a learning processing unit that performs learning processing of the operation model using learning data including values of the state parameters and the control parameters.

The learning processing unit may perform learning processing of the operation model using the learning data and a reward value determined by a preset reward function.

In a second aspect of the present invention, a method is provided. The method may include a supply step of supplying a value of a state parameter indicating a state of the equipment to an operation model that outputs a recommended value of a control parameter of the equipment in response to input of the value of the state parameter. The method may include a control parameter acquisition step of acquiring a recommended value of a control parameter output from the operation model in response to supplying the value of the state parameter to the operation model by the supply step. The method may include an acquisition step of acquiring a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition step. The method may include an evaluation step of evaluating the operation model based on the model evaluation value and a reference evaluation value corresponding to a result of operating the equipment by manual operation.

In a third aspect of the present invention, a program is provided. The program may cause a computer to function as a supply unit that supplies a state parameter value to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of a state parameter related to the equipment. The program may cause a computer to function as a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the state parameter value to the operation model. The program may cause a computer to function as an acquisition unit that acquires a model evaluation value corresponding to the result of operating the equipment using the recommended value acquired by the control parameter acquisition unit. The program may cause a computer to function as an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value corresponding to the result of operating the equipment by manual operation.

Note that the above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions.

1 shows a system 1 according to an embodiment. 4 shows the data structure of an operation model 401. 1 shows an action decision table. The learning operation of the operation model 401 is shown. Another learning operation of the operation model 401 is shown. 4 illustrates the learning operation of the goal setting model 414. The evaluation operation of the operation model 401 is shown. The operation of equipment 2 is shown. 13 shows a goal setting model 414A according to a modified example. 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part.

The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[1. System]
FIG. 1 shows a system 1 according to this embodiment. The system 1 includes a facility 2 and an apparatus 4. Note that the blocks in the figure are functionally separated functional blocks, and do not necessarily correspond to the actual apparatus configuration. In other words, even if one block is shown in this figure, it does not necessarily have to be composed of one apparatus. Also, even if separate blocks are shown in this figure, they do not necessarily have to be composed of separate apparatus.

[1.1. Equipment 2]
The facility 2 is a facility or device equipped with one or more devices (not shown). For example, the facility 2 may be a plant, or a composite device in which multiple devices are combined. Examples of the plant include industrial plants such as chemical and bio plants, plants that manage and control wellheads and surrounding areas of gas fields and oil fields, plants that manage and control power generation such as hydroelectric power, thermal power, and nuclear power, plants that manage and control environmental power generation such as solar power and wind power, and plants that manage and control water supply and sewage systems, dams, etc.

Each device is an instrument, machine, or device, and may be, for example, an actuator such as a valve, pump, heater, fan, motor, or switch that controls at least one physical quantity, such as pressure, temperature, pH, speed, or flow rate, in the process of facility 2. Each device may be of a different type, or at least some of two or more devices may be of the same type. Each device may be controlled by device 2 via a wired or wireless connection.

The equipment 2 may be provided with one or more sensors (not shown). Each sensor measures or determines the status of the equipment 2. Each sensor may measure or determine the operating status of the equipment 2, such as the production volume of the equipment 2, the proportion of impurities mixed in, the operating status of each control object, and the occurrence of alarms. The operating status of the equipment may be represented by at least one physical quantity, such as pressure, temperature, pH, speed, and flow rate, which are controlled by the equipment, as an example. Each sensor may supply the measurement or determination results to the device 2.

[1.2. Apparatus 4]
The device 4 may operate the equipment 2 using an operation model 401 for operating the equipment 2, and may have the operation model 401, an operation unit 402, and a parameter acquisition unit 403. The device 4 may also perform a learning process of the operation model 401, and may have an input unit 411, a storage unit 412, a second learning processing unit 413, a target setting model 414, a first supply unit 415, a second acquisition unit 416, and a second supply unit 417. The device 4 may also perform a learning process of the target setting model 414, and may have a first acquisition unit 421 and a first learning processing unit 422. The device 4 may also evaluate the operation model 401, and may have a simulator 431, a display control unit 432, a target range acquisition unit 433, an evaluation value acquisition unit 434, and an evaluation unit 435.

[1.2.1. Operation Model 401]
The operation model 401 is a model for operating the equipment 2. The operation model 401 may output recommended values of control parameters of the equipment 2 in response to input of values of state parameters related to the equipment 2. The operation model 401 may supply the control parameters to the operation section 402.

In response to input of the value of the state parameter, the operation model 401 may output a recommended value of the control parameter that brings the state of the equipment 2 closer to a state corresponding to the contents of the target setting data used to learn the operation model 401. The target setting data may include identification information of a parameter for which a target range is set among the parameters of the equipment 2, and the target range set for that parameter. The target setting data may include only one combination of parameter identification information and target range, or may include multiple combinations.

The parameters related to the equipment 2 may include state parameters related to the equipment 2 and control parameters of the equipment 2. The state parameters related to the equipment 2 may include state parameters of the equipment 2 and state parameters of the product by the operation of the equipment 2 (also referred to as performance parameters). The state parameters of the equipment 2 may be, for example, pressure, flow rate, temperature, pH, speed, power consumption, concentration, etc. The state parameters of the equipment 2 may be parameters related to the energy consumption of the equipment 2, parameters related to the amount of greenhouse gas emissions, or parameters related to the yield. The state parameters of the product may be, for example, an index value indicating quality (also referred to as a quality value) or a production amount. The quality value may be, for example, a value indicating the purity, concentration, composition, viscosity, color, etc. of the product. The control parameter may be, for example, the amount of operation of the valve. In this embodiment, as an example, the parameter for which the target range is set may be a state parameter related to the equipment 2.

[1.2.2. Operation Section 402]
The operation unit 402 operates the equipment 2 using the operation model 401. The operation unit 402 may be an example of a control parameter acquisition unit, and may acquire recommended values of control parameters output from the operation model 401 in response to a parameter acquisition unit 403 described below supplying values of state parameters to the operation model 401. The operation unit 402 may operate the equipment 2 by controlling each device of the equipment 2 using the control parameters output from the operation model 401. The operation unit 402 may supply the recommended values of the control parameters output from the operation model 401 to a simulator 431 to simulate the operation of the equipment 2.

The operation unit 402 may operate the equipment 2 in response to manual operations input via the input unit 411. The operation unit 402 may supply control parameters in response to the manual operations to the simulator 431 to simulate the operation of the equipment 2.

Parameter Acquisition Unit 403
The parameter acquisition unit 403 acquires parameters related to the equipment 2. The parameter acquisition unit 403 may acquire control parameters of the equipment 2 from the operation unit 402. The parameter acquisition unit 403 may acquire state parameters related to the equipment 2 (in the present embodiment, as an example, state parameters of the product and state parameters of the equipment 2) from the equipment 2. However, the parameter acquisition unit 403 may acquire the control parameters of the equipment 2 from the equipment 2, or may acquire the state parameters of the product from the operator. In addition, when a simulation of the equipment 2 is performed by the simulator 431, the parameter acquisition unit 403 may acquire state parameters related to the equipment 2 from the simulator 431, among the parameters related to the simulated equipment 2.

The parameter acquisition unit 403 may store each acquired parameter in the storage unit 412. The parameter acquisition unit 403 may be an example of a supply unit, and may supply the value of the state parameter to the operation model 401.

[1.2.4. Input section 411]
The input unit 411 receives various input operations from an operator. The input unit 411 may receive an input operation of an operation plan for the equipment 2 from the operator. The input unit 411 may also receive an input operation of target setting data from the operator. When the learning process of the target setting model 414 is not completed, the operation plan and the target setting data may be input in association with each other.

Here, the operation plan for facility 2 may indicate at least one of the planned production volume, target quality, and material type of the product produced by facility 2. The operation plan for facility 2 may also indicate other content, such as the energy efficiency, power consumption, yield, and greenhouse gas emissions of facility 2.

The input unit 411 may store the input operation plan and target setting data in the storage unit 412. The input unit 411 may supply the input target setting data to the second learning processing unit 413. The input unit 411 may supply the input operation plan to the first supply unit 415.

[1.2.5. Storage unit 412]
The storage unit 412 stores various data. The storage unit 412 may store each parameter acquired by the parameter acquisition unit 403. The storage unit 412 may also store the target setting data input by the input unit 411. and an operation plan of the facility 2. The target setting data stored in the storage unit 412 may be data used for learning the operation model 401, and may be data set by an experienced operator, for example. In addition, when the target setting data is generated by supplying the operation plan to the target setting model 414 described later, the storage unit 412 further stores the target setting data and the operation plan. The data stored in the storage unit 412 may be used in the learning process by the first learning processing unit 422 and the second learning processing unit 413.

[1.2.6. Second learning processing unit 413]
The second learning processing unit 413 executes a learning process of the operation model 401 to output recommended values of the control parameters in response to input of the state parameter values, using learning data including values of state parameters related to the equipment 2 and values of control parameters of the equipment 2. The types of state parameters and control parameters included in the learning data used by the second learning processing unit 413 may be arbitrarily selected by the operator from the parameters acquired by the parameter acquisition unit 403.

The second learning processing unit 413 may perform the learning process of the operation model 401 by reinforcement learning. For example, the second learning processing unit 413 may execute the learning process of the operation model 401 using the learning data and a reward value determined by a preset reward function.

The second learning processing unit 413 may further use the target setting data to perform learning processing of the operation model 401, and may perform learning processing to output values of control parameters that bring the state of the equipment 2 closer to a state corresponding to the content of the target setting data in response to input of the value of the state parameter. In this case, the second learning processing unit 413 may perform learning processing using a reward value determined by a reward function set based on the content of the target setting data.

For example, the reward function may be a function that sets the reward value to 1 when the value of a state parameter for equipment 2 operated using the control parameters output from operation model 401 satisfies the contents of the target setting data, and sets the reward value to 0 when it does not. Furthermore, the reward function may be a function that changes the reward value depending on the degree to which the value of a state parameter for equipment 2 operated using the control parameters output from operation model 401 deviates from the target range of the target setting data. As an example, the reward function may be a function shown in the following formula (1).

Reward value = a * energy saving index - b * quality value deviation (1)
In addition, "a" and "b" in formula (1) may be coefficients. The "energy saving index" is an index indicating the degree of energy saving of the equipment 2, and may be a value calculated from the state parameters of the equipment 2. The "degree of deviation of the quality value" may be the degree to which the quality value of the product deviates from the target range of the quality value in the target setting data.

1.2.7. Goal Setting Model 414
In response to an input of an operation plan, the target setting model 414 outputs at least one of the parameter identification information and the target range among the target setting data to be used for learning of the operation model 401. As an example in the present embodiment, the target setting model 414 may output both the parameter identification information and the target range. The target setting model 414 may output only one combination of the parameter identification information and the target range, or may output multiple combinations.

[1.2.8. 1st supply section 415]
In response to the input of an operation plan of the facility 2, the first supply unit 415 supplies the operation plan to the target setting model 414. In response to the input of the plan, the new operation plan may be provided to the target setting model 414. As a result, data corresponding to the operation plan is output from the target setting model 414.

[1.2.9. Second acquisition unit 416]
The second acquisition unit 416 acquires output data from the target setting model 414 to which the operation plan is supplied by the first supply unit 415. As an example in the present embodiment, the second acquisition unit 416 acquires output data from the target setting model 414. The second acquiring section 416 may acquire both the identification information and the target range of the parameter as output data, and may supply the acquired output data to the second supplying section 417.

[1.2.10. Second supply section 417]
The second supply unit 417 supplies the second learning processing unit 413 with the target setting data corresponding to the output data acquired by the second acquisition unit 416. A learning process of the operation model 401 is performed using the setting data.

In this embodiment, as an example, the output data from the target setting model 414 includes both parameter identification information and target ranges. Therefore, the second supply unit 417 may supply the output data directly to the operation model 401 as target setting data.

[1.2.11. First acquisition unit 421]
The first acquisition unit 421 acquires the operation plan of the equipment 2 and at least the identification information of the parameters of the target setting data used for learning the operation model 401. The first acquisition section 421 may acquire both the identification information and the target range of the parameter from the target setting data. The first acquisition section 421 may supply the acquired data to the first learning processing section 422.

[1.2.12. First learning processing unit 422]
The first learning processing unit 422 performs learning processing of the target setting model 414 using learning data including the parameter identification information and the operation plan acquired by the first acquisition unit 421 .

The first learning processing unit 422 performs learning processing of the target setting model 414 by supervised learning such as deep learning, but may also perform learning of the target setting model 414 by other machine learning methods. For example, the first learning processing unit 422 may perform learning processing of the target setting model 414 using learning data including identification information of parameters in the target setting data used to learn the operation model 401 and an operation plan input in association with the target setting data. As an example in this embodiment, the first learning processing unit 422 may perform learning processing of the target setting model 414 using learning data including identification information of parameters in the target setting data set by a veteran operator and used to learn the operation model 401 and an operation plan input in association with the target setting data.

The first learning processing unit 422 may perform learning processing of the target setting model 414 so that the contents of the output data from the target setting model 414 approximate the contents of the target setting data used in learning the operation model 401. In addition, the first learning processing unit 422 may perform learning processing to output identification information and target ranges of parameters of the target setting data to be used in learning the operation model 401 in order to achieve an operation plan, in response to an operation plan being input to the target setting model 414.

The first learning processing unit 422 may perform learning processing of the goal setting model 414 using learning data that further includes the target range of the parameter acquired by the first acquisition unit 421. In other words, the first learning processing unit 422 may perform learning processing of the goal setting model 414 using learning data that includes the identification information and target range of the parameter in the goal setting data used in learning the operation model 401. In response to an operation plan being input to the goal setting model 414, the first learning processing unit 422 may perform learning processing to output both the identification information and the target range of the parameter from the target setting data to be used in learning the operation model 401 to achieve the operation plan.

Simulator 431
The simulator 431 simulates the state of the facility 2. The simulator 431 may be a dynamic simulator that dynamically simulates the facility 2 from a steady state to a stop, or may be a static simulator that simulates the steady state of the facility 2.

The simulator 431 may simulate the state of the operated equipment 2 based on the values of the control parameters supplied from the operation unit 402. The values of the control parameters supplied from the operation unit 402 may be recommended values of the control parameters output from the operation model 401, or may be control parameters according to manual operation. The simulator 431 may supply parameters of the equipment 2 operated in the simulation (as an example in this embodiment, state parameters of the equipment 2) to the evaluation value acquisition unit 434 and the parameter acquisition unit 403.

[1.2.14. Display control unit 432]
The display control unit 432 displays various information on a display device (not shown). For example, the display control unit 432 may display identification information and target ranges of parameters acquired by the second acquisition unit 416 from the target setting model 414. The display control unit 432 may read out and display each parameter acquired by the parameter acquisition unit 403 from the storage unit 412. In response to an operator selecting any parameter (also referred to as a selected parameter) from a plurality of types of parameters related to the equipment 2, the display control unit 432 may display the value of the selected parameter in past operations of the equipment 2.

When equipment 2 manufactures goods, the parameters related to equipment 2 in the display control unit 432, the target range acquisition unit 433, the evaluation value acquisition unit 434, and the evaluation unit 435 described below may be state parameters related to the product, and in this embodiment, as an example, may be at least one of an index value indicating the quality of the product or the production volume of the product. In addition to this, or instead of this, the parameters related to equipment 2 may be state parameters of equipment 2 (for example, the energy efficiency and power consumption of equipment 2, etc.) or may be control parameters of equipment 2.

[1.2.15. Target range acquisition unit 433]
The target range acquisition unit 433 acquires a target range (also referred to as an evaluation target range) set by the operator for the selected parameter selected by the operator in order to evaluate the operation model 401. For example, the target range acquisition unit 433 may acquire the evaluation target range set by the operator based on the value in the past operation displayed for each selected parameter by the display control unit 432. The target range acquisition unit 433 may supply the acquired evaluation target range to the evaluation value acquisition unit 434.

The evaluation target range may be the same as or different from the target range in the target setting data used to train the operation model 401. The parameters for which the evaluation target range is set may be the same as or different from the parameters in the target setting data used to train the operation model 401.

[1.2.16. Evaluation value acquisition unit 434]
The evaluation value acquisition unit 434 acquires a model evaluation value according to a result of operating the equipment 2 based on the recommended value acquired from the operation model 401 by the operation unit 402. The evaluation value acquisition unit 434 may supply the acquired model evaluation value to the evaluation unit 435.

The model evaluation value may be an evaluation value for evaluating the operation model 401. In this embodiment, as an example, the model evaluation value may be calculated based on whether or not the parameters related to the equipment 2 operated according to the recommended value output from the operation model 401 fall within the evaluation target range. Note that the model evaluation value may be the same value as the reward value used by the second learning processing unit 413 in the reinforcement learning of the operation model 401, or may be a different value.

The evaluation value acquisition unit 434 may further acquire a reference evaluation value according to the results of operating the equipment 2 through manual operation (for example, operation by an experienced operator). The evaluation value acquisition unit 434 may supply the acquired reference evaluation value to the evaluation unit 435.

The reference evaluation value may be a reference value of the model evaluation value. The reference evaluation value may be calculated in the same manner as the model evaluation value, based on whether the parameters related to the equipment 2 operated by manual operation fall within a target range.

Evaluation Unit 435
The evaluation unit 435 evaluates the operation model 401 based on the model evaluation value and the reference evaluation value. The evaluation unit 435 may evaluate the operation model 401 based on a comparison result between the model evaluation value and the reference evaluation value. For example, when the model evaluation value is a better value than the reference evaluation value, the evaluation unit 435 may evaluate that the operation model 401 is good. The evaluation unit 435 may output the evaluation result to the display control unit 432 or the like.

According to the above-mentioned device 4, the target setting model 414 outputs at least one of the parameter identification information or the target range of the target setting data to be used for learning the operation model 401 in response to the input of the operation plan. In addition, the learning process of the target setting model 414 is performed using the operation plan of the equipment 2 and learning data including at least the parameter identification information of the target setting data used for learning the operation model 401. Therefore, the contents of the output data from the target setting model 414 (here, at least one of the parameter identification information or the target range) can be approximated to the contents of the target setting data used in learning the operation model 401 to achieve the operation plan. Therefore, by performing the learning process of the operation model 401 using the output data from the target setting model 414, it is possible to generate an operation model 401 that performs appropriate operation according to the operation plan.

In addition, using the target setting data, a learning process is performed on the operation model 401 so as to output the value of the control parameter that brings the state of the equipment 2 closer to the state corresponding to the content of the target setting data in response to the input of the state parameter value. Therefore, an operation model 401 that operates the equipment 2 in an appropriate operating state can be generated.

In addition, when a new operation plan is input, the operation plan is supplied to the target setting model 414, and the operation model 401 undergoes a learning process using target setting data corresponding to the output data from the target setting model 414. Therefore, each time the operation plan is changed, the operation model 401 undergoes a learning process using target setting data corresponding to the operation plan, and an operation model 401 that performs appropriate operation according to the operation plan can be generated.

The operation model 401 is evaluated based on a model evaluation value corresponding to the results of operating the equipment 2 using the recommended values of the control parameters output from the operation model 401, and a reference evaluation value corresponding to the results of operating the equipment 2 by manual operation. Therefore, the quality of the operation results obtained by using the operation model 401, and therefore the quality of the operation model 401, can be uniformly determined.

In addition, a model evaluation value is calculated based on whether the parameters of equipment 2 operated according to the recommended values of the control parameters output from the operation model 401 fall within the evaluation target range of the operation model 401, and a reference evaluation value is calculated based on whether the parameters of equipment 2 operated by manual operation fall within the evaluation target range. Therefore, the quality of the operation results by using the operation model 401 can be judged more uniformly.

In addition, the target evaluation range set by the operator is obtained for the selected parameters selected by the operator from among multiple types of parameters related to the equipment 2, so any target evaluation range can be set for any parameter. Therefore, the evaluation criteria for the operation results can be set arbitrarily.

In addition, as a selected parameter is selected from multiple types of parameters, the value of the selected parameter in past operations of equipment 2 is displayed, so that a target range for evaluation can be set based on the past values of the selected parameter.

In addition, in the evaluation of the operation model 401, the parameter related to the equipment 2 is at least one of an index value indicating the quality of the product produced by the equipment 2 or the production volume of the product, so that an evaluation result can be obtained that determines that the operation model 401 that improves the production volume or quality is a good operation model 401. Therefore, by using a highly evaluated operation model 401, the production volume and quality can be improved.

In addition, the second learning processing unit 413 performs learning processing of the operation model 401 using learning data including the values of the state parameters and the values of the control parameters, so that it is possible to perform learning processing on an operation model 401 with a low evaluation to obtain an operation model 401 with a high evaluation.

In addition, the learning process of the operation model 401 is performed using the learning data and a reward value determined by a preset reward function, so that a highly rated operation model 401 can be reliably obtained.

[2. Operation Model 401]
FIG. 2 shows the data structure of the operation model 401. The operation model 401 has a data structure consisting of a combination (s, a) of a state s indicating a set of sampled state data and an action a taken under each state, and a weight w calculated by the reward. Such weights may be determined based on the reward determined by a reward function using goal setting data. In this figure, as an example, a case where state s=(TI001, TI002, TI003, FI001, FI002, VI001) is shown. In this figure, for example, when an action of a=1 is taken under a state of s=(-2.47803, -2.48413, -0.07324, 29.71191, 24.2511, 70), the weight calculated by the reward is w=144.1484. The next action is determined by such an operation model 401.

Figure 3 shows the action decision table. The action decision table is composed of the input state s and the possible actions a. In this figure, as an example, the input state s = (0.1, 0.2, 0.4, 0.3, 0.8, 0.2) and the five possible actions a = (-3, -1, 0, 1, 3) are shown. For example, by inputting such an action decision table into the operation model 401 shown in Figure 4, the next action is decided. This will be explained in detail using a flow chart.

3. Operation of Device 4
[3.1. Learning Operation of Operation Model 401]
4 shows the learning operation of the operation model 401. The device 1 may generate the operation model 401 by the processes of steps S101 to S119.

In step S101, the second learning processing unit 413 acquires goal setting data. In the operation of this figure, the second learning processing unit 413 may acquire goal setting data including identification information of a parameter input via the input unit 411 and a target range of the parameter.

In step S101, the display control unit 432 may display the contents of the target setting data previously used in the learning of the operation model 401. For example, the display control unit 432 may display the target range previously set for the parameter in response to the operator selecting one of the multiple types of parameters related to the equipment 2 as the target range setting target for the parameter. The display control unit 432 may further display at least a part of the area included in the previously set target range as the recommended target range. For example, the display control unit 432 may display the central range of the previously set target range with a pre-specified ratio as the recommended target range. As an example, if the previously set target range for the parameter Pa is 5 to 15, the previously set target range for the parameter Pb is 10 to 30, and the specified ratio is 90%, the display control unit 432 may set the recommended target range for the parameter Pa to 6 to 14 and the recommended target range for the parameter Pb to 12 to 28.

The display control unit 432 may display a previously set target range in a coordinate space having each parameter for which the target range is set as a coordinate axis. The display control unit 432 may also display a recommended target range in the coordinate space. When a previously set target range is displayed in the coordinate space, the second learning processing unit 413 may acquire the specified range as the target range in response to a range specified in the coordinate space by the input unit 411.

In step S103, the second learning processing unit 413 determines a reward function using the goal setting data. The second learning processing unit 413 may determine the reward function such that the reward value becomes high when the state of the equipment 2 operated by the operation model 401 approaches a state corresponding to the contents of the goal setting data. The second learning processing unit 413 may also determine the reward function such that the reward value becomes high when the state parameters of the equipment 2 operated by the operation model 401 satisfy the contents of the goal setting data.

In step S105, the parameter acquisition unit 403 acquires state parameters related to the equipment 2. For example, the parameter acquisition unit 403 may acquire the state parameters from the equipment 2 or the simulator 431.

In step S107, the second learning processing unit 413 determines an action and determines control parameters according to the determined action. For example, the second learning processing unit 413 randomly determines an action. In the above description, the second learning processing unit 413 randomly determines an action, but this is not limited to this. When the second learning processing unit 413 determines an action, a known AI algorithm such as FKDPP (Factorial Kernel Dynamic Policy Programming) may be used. When using such a kernel method, the second learning processing unit 413 generates a vector of state s from the state data. Next, the second learning processing unit 413 generates a combination of the state s and all possible actions a as an action determination table, for example, as shown in FIG. 3. Then, the second learning processing unit 413 inputs the action determination table into the operation model 401, for example, as shown in FIG. 2. In response to this, a kernel calculation is performed between each row of the behavior determination table and each sample data of the operation model 401 excluding the weight column, and the distance between each sample data is calculated. Then, the distance calculated for each sample data is multiplied by the value of each weight column and the products are sequentially added together to calculate the reward expectation value for each behavior. The operation model 401 selects the behavior that has the highest reward expectation value calculated in this way. The second learning processing unit 413 may determine the behavior by selecting the behavior that is determined to have the highest reward expectation value using the operation model being updated in this way, for example. During learning, the second learning processing unit 413 may determine the behavior while appropriately selecting whether to determine the behavior randomly or to determine the behavior using the operation model 401. The second learning processing unit 413 supplies the control parameters according to the determined behavior to the operation unit 402.

In step S109, the operation unit 402 operates the equipment 2 according to the supplied control parameters. The operation unit 402 may cause the simulator 431 to perform a simulation according to the supplied control parameters.

In step S111, the parameter acquisition unit 403 acquires state parameters related to the equipment 2. This acquires state parameters that have changed in response to the equipment 2 being operated according to the determined control parameters. Note that, if a simulation has been performed in step S109, the parameter acquisition unit 403 may acquire the state parameters from the simulator 431.

In step S113, the second learning processing unit 413 calculates a reward value based on the acquired parameters. The second learning processing unit 413 may calculate the reward value using the reward function determined in step S103.

In step S115, the second learning processing unit 413 determines whether the process of acquiring parameters in response to the determination of the control parameters has exceeded a specified number of steps. Note that such a number of steps may be specified in advance by the operator, or may be determined based on a learning period (e.g., 10 days, etc.). If it is determined that the above-mentioned process has not exceeded the specified number of steps (step S115; No), the second learning processing unit 413 returns the process to step S107 and continues the flow. As a result, the process of acquiring state parameters in response to the determination of the control parameters is executed the specified number of steps.

If it is determined in step S115 that the above-mentioned processing has exceeded the specified number of steps (step S115; Yes), the second learning processing unit 413 advances the processing to step S117. In step S117, the second learning processing unit 413 updates the operation model 401. For example, the second learning processing unit 413 overwrites the values of the weight column in the operation model shown in FIG. 2, and adds new sample data that has not been saved to the operation model 401.

In step S119, the second learning processing unit 413 determines whether the update process of the operation model 401 has exceeded a specified number of repetitions. Note that such a number of repetitions may be specified in advance by an operator, or may be determined according to the validity of the operation model 401. If it is determined that the above-mentioned process has not exceeded the specified number of repetitions (step S119; No), the second learning processing unit 413 returns the process to step S105 and continues the flow.

If it is determined in step S119 that the above-mentioned process has exceeded the specified number of repetitions (step S119; Yes), the second learning processing unit 413 ends the flow. In this way, for example, the second learning processing unit 413 can generate an operation model 401 that outputs control parameters according to state parameters related to the equipment 2.

[3.2. Other learning operations of the operation model 401]
5 shows another learning operation of the operation model 401. When the learning process of the target setting model 414 is completed, the device 1 may generate the operation model 401 by the processes of steps S121 to S123 and S103 to S119.

In step S121, the first supply unit 415 acquires a newly input operation plan. The first supply unit 415 may acquire a newly input operation plan to the input unit 411.

In step S123, the first supply unit 415 supplies the acquired operation plan to the target setting model 414. As a result, when a new operation plan is input, the operation plan is supplied to the target setting model 414.

In step S125, the second supply unit 417 acquires output data (as an example in this embodiment, parameter identification information and target range) from the goal setting model, and acquires goal setting data corresponding to the output data. The second supply unit 417 may acquire the output data as it is as the goal setting data.

Alternatively, the second supply unit 417 may cause the display control unit 432 to display the contents of the acquired output data as recommended goal setting data, and acquire the goal setting data input by the operator based on the displayed contents. As an example, if the contents of the output data in which the target range for a certain parameter is 1 to 10 is displayed, and the operator inputs the target range for the parameter as 2 to 9, the second supply unit 417 may acquire the goal setting data in which the target range for the parameter is 2 to 9.

After this, the learning operation of the operation model 401 may be performed in the same manner as steps S103 to S119 described above.

3.3. Learning Operation of Goal Setting Model 414
6 shows the learning operation of the goal setting model 414. The device 1 may generate the goal setting model 414 by the processes of steps S131 to S133.

In step S131, the first acquisition unit 421 acquires the operation plan of the equipment 2 and at least parameter identification information of the target setting data used to learn the operation model 401. As an example in this embodiment, the first acquisition unit 421 may acquire at least parameter identification information of the target setting data used to learn the operation model 401 and the operation plan input by the input unit 411 in association with the target setting data. The first acquisition unit 421 may acquire both the parameter identification information and the target range of the parameter in the target setting data. The first acquisition unit 421 may acquire the parameter identification information in the target setting data, the operation plan, and the like from the storage unit 412.

In step S133, the first learning processing unit 422 performs a learning process for the target setting model 414 using learning data including the parameter identification information and the operation plan acquired by the first acquisition unit 421. The first learning processing unit 422 may perform the learning process for the target setting model 414 by supervised learning such as deep learning, and may perform the learning process for the target setting model 414 so that the content of the output data from the target setting model 414 approximates the content of the target setting data used in the learning of the operation model 401. Also, as an example in this embodiment, the first learning processing unit 422 may perform a learning process for the target setting model 414 so that, in response to an input of an operation plan, the first learning processing unit 422 outputs the content of the target setting data to be used in the learning of the operation model 401 to achieve the operation plan.

[3.4. Evaluation of Operation Model 401]
7 shows an evaluation operation of the operation model 401. The device 1 may evaluate the operation model 401 generated by the processes of steps S141 to S175.

In step S141, in response to the operator selecting one of multiple types of parameters related to equipment 2 as a selected parameter, the display control unit 432 displays the value of the selected parameter in past operations of equipment 2. For example, the display control unit 432 may display the value of each selected parameter in past operations of equipment 2 in a coordinate space with each selected parameter as a coordinate axis. As one example, the display control unit 432 may display the value of each selected parameter in past operations of equipment 2, or may display the maximum and minimum values of each selected parameter in past operations of equipment 2 to display the range of values of each selected parameter.

In step S143, the target range acquisition unit 433 acquires the evaluation target range set by the operator for the selected parameters. The target range acquisition unit 433 may acquire the specified range as the evaluation target range in response to range specification within a coordinate space displayed by the display control unit 432, with each selected parameter as its coordinate axis.

In step S145, the state parameter acquisition unit 403 acquires state parameters related to the equipment 2 from the simulator 431. When the process of step S145 is executed for the first time, the state of the equipment 2 may be a preset initial state.

In step S147, the simulator 431 performs a simulation according to the manual operation. The simulator 431 may simulate the state of the equipment 2 operated based on the control parameters according to the manual operation.

In step S149, the simulator 431 determines whether or not an instruction to end the simulation has been given. For example, the simulator 431 may determine whether or not an instruction to end the simulation has been input via the input unit 411. If it is determined in step S149 that an instruction to end the simulation has not been given (step S149; No), the process may proceed to step S145 described above. If it is determined in step S149 that an instruction to end the simulation has been given (step S149; Yes), the process may proceed to step S151.

In step S151, the evaluation value acquisition unit 434 acquires a reference evaluation value according to the result of operating the equipment 2 by manual operation. In this embodiment, as an example, the reference evaluation value may be calculated based on the result of inputting the manual operation into the simulator 431.

The reference evaluation value may be calculated based on whether the parameters for the equipment 2 operated by the control parameters according to the human operation fall within the target range for evaluation. When a target range for evaluation is set for each of the multiple parameters for the equipment 2, the reference evaluation value may be calculated based on the ratio (i.e., b/a) of the number of parameters (b) that fall within the corresponding target range out of the number of parameters (a) for which the target range for evaluation is set. Note that the reference evaluation value may be calculated by the simulator 431, or may be calculated by the evaluation value acquisition unit 434 that acquires the parameters for the operated equipment 2 from the simulator 431.

In step S153, the simulator 431 ends the simulation in response to the manual operation. Upon completion of the simulation, the simulated state of the equipment 2 may be reset to the initial state.

In step S161, the state parameter acquisition unit 403 acquires state parameters related to the equipment 2 from the simulator 431. Note that when the process of step S161 is executed for the first time, the state of the equipment 2 may be a preset initial state.

In step S163, the parameter acquisition unit 403 supplies the acquired state parameters to the operation model 401. This causes the operation model 401 to output recommended values for the control parameters.

In step S165, the operation unit 402 obtains the recommended values of the control parameters output from the operation model 401.

In step S167, the simulator 431 performs a simulation according to the recommended values of the control parameters from the operation model 401. The simulator 431 may simulate the state of the equipment 2 operated based on the recommended values of the control parameters.

In step S169, the simulator 431 determines whether or not an instruction to end the simulation has been given. If it is determined in step S169 that an instruction to end the simulation has not been given (step S169; No), the process may proceed to step S161 described above. If it is determined in step S169 that an instruction to end the simulation has been given (step S169; Yes), the process may proceed to step S171.

In step S171, the evaluation value acquisition unit 434 acquires a model evaluation value according to the result of operating the equipment 2 with the recommended values of the control parameters. As an example in this embodiment, the model evaluation value may be calculated based on the result of inputting the recommended values acquired by the operation unit 402 to the simulator 431 of the equipment 2. The model evaluation value may be calculated, similar to the reference evaluation value, based on whether the parameters related to the equipment 2 operated with the recommended values of the control parameters fall within the evaluation target range.

In step S173, the simulator 431 ends the simulation according to the recommended values of the control parameters. Upon ending the simulation, the simulated state of the equipment 2 may be reset to the initial state.

In step S175, the evaluation unit 435 evaluates the operation model 401 based on the model evaluation value and the reference evaluation value. As an example in this embodiment, the evaluation unit 435 may evaluate the operation model 401 as being good when the model evaluation value is greater than the reference evaluation value.

According to the above operation, the reference evaluation value is calculated based on the results of manual operations input into the simulator 431 of the equipment 2, so the reference evaluation value can be obtained quickly without actually operating the equipment 2.

In addition, the model evaluation value is calculated based on the results of inputting the recommended values of the control parameters obtained from the operation model 401 into the simulator 431 of the equipment 2, so that the model evaluation value can be obtained quickly without actually operating the equipment 2.

In addition, since the reference evaluation value and the model evaluation value are each calculated based on the results of the simulation, the equipment 2 can be brought into the same state before operation when the equipment 2 is operated by manual operation and when the equipment 2 is operated using the operation model 401. Therefore, the quality of the operation results by using the operation model 401 can be accurately determined.

In addition, when setting a target range for evaluation, the values of each selected parameter from past operations of equipment 2 are displayed in a coordinate space with each selected parameter selected by the operator as its coordinate axis, making it easy to understand the past values and ranges of the selected parameters and to set a target range for evaluation.

[3.5. Operation of Equipment 2]
8 shows the operation of the facility 2. The apparatus 1 may operate the facility 2 by performing the processes of steps S181 to S191.

In step S181, the state parameter acquisition unit 403 acquires state parameters related to equipment 2. In step S183, the parameter acquisition unit 403 supplies the acquired state parameters to the operation model 401. As a result, recommended values of the control parameters are output from the operation model 401. In step S185, the operation unit 402 acquires the recommended values of the control parameters output from the operation model 401. In step S187, the operation unit 402 operates equipment 2 according to the recommended values of the control parameters from the operation model 401.

In step S189, the operation department 402 determines whether or not an instruction to end operations has been given. If it is determined in step S189 that an instruction to end operations has not been given (step S189; No), processing may proceed to step S181 described above. If it is determined in step S189 that an instruction to end operations has been given (step S189; Yes), processing proceeds to step S191, and the operation department 402 ends operations of equipment 2.

[4. Variation (1) of the goal setting model 414]
In the above embodiment, the target setting model 414 is subjected to a learning process using an operation plan and learning data including identification information and target ranges of parameters in the target setting data used in learning the operation model 401, and the target setting model 414 that has undergone the learning process outputs, in response to an input of an operation plan, the identification information and target ranges of parameters from the target setting data to be used in learning the operation model 401. However, the combination of the contents of the learning data, input data, and output data for the target setting model 414 is not limited to this.

For example, the target setting model 414 performs a learning process using the operation plan and learning data including the identification information of the parameters in the target setting data used to learn the operation model 401, and may output only the identification information of the parameters from the target setting data to be used to learn the operation model 401 in response to the input of the operation plan, without outputting the target range. The learning data may include both the identification information and the target range of the parameters in the target setting data used to learn the operation model. The target setting model 414 may output the identification information of a single parameter, or may output the identification information of multiple parameters. When only the identification information of the parameters is output from the target setting model 414, the second supply unit 417 may display the identification information of each output parameter on the display control unit 432, obtain the target range input by the operator for each identification information of the parameter, generate target setting data indicating these identification information and the target range, and supply it to the second learning processing unit 413.

In addition, the target setting model 414 performs a learning process using the operation plan and learning data including the identification information and target range of the parameter in the target setting data used to learn the operation model 401, and outputs only the target range for the parameter among the target setting data to be used for learning the operation model 401 in response to the input of the operation plan and the identification information of the parameter to be set as the target range, and does not need to output the identification information of the parameter. In this case, the identification information of a single parameter to be set as the target range may be input from the operator to the target setting model 414 via the input unit 411, and the target range of the single parameter may be output from the target setting model 414. Alternatively, the identification information of multiple parameters to be set as the target range may be input from the operator to the target setting model 414 via the input unit 411, and the target range of each parameter may be output from the target setting model 414. The second supply unit 417 may generate goal setting data indicating the identification information of each parameter input to the goal setting model 414 and the target range of each parameter output from the goal setting model 414, and supply the data to the second learning processing unit 413.

[5. Variation (2) of the goal setting model 414]
Furthermore, in the above embodiment, the goal setting model 414 has been described as a single model, but it may have a plurality of models with different functions.

Figure 9 shows the target setting model 414A according to this modified example. The target setting model 414A may have at least one parameter setting model 4141 and at least one target range setting model 4142. In this embodiment, as an example, the target setting model 414A has two parameter setting models 4141a and 4141b and four target range setting models 4142a to 4142d. Each parameter setting model 4141 outputs identification information of a parameter for which a target range is to be set in response to input of an operation plan. Each target range setting model 4142 outputs a target range to be set for that parameter in response to input of an operation plan and identification information of a parameter for which a target range is to be set.

Of these, the parameter setting model 4141a may output identification information of a parameter (also referred to as parameter Pa) for which a target range is to be set in response to input of an operation plan. In the present embodiment, as an example, in response to input of an operation plan from the first supply unit 415, the parameter setting model 4141a supplies the operation plan and identification information of parameter Pa to the target range setting model 4142a.

The target range setting model 4142a may output the upper limit of the target range to be set for the parameter in response to input of the operation plan and the identification information of the parameter. In the present embodiment, as an example, in response to input of the operation plan and the identification information of the parameter Pa from the parameter setting model 4141a to which the operation plan has been input by the first supply unit 415, the target range setting model 4142a supplies the operation plan, the identification information of the parameter Pa, and the upper limit value V _PaMAX of the target range of the parameter Pa to the target range setting model 4142b. Note that the operation plan may be directly input from the first supply unit 415 to the target range setting model 4142a instead of being input from the first supply unit 415 to the target range setting model 4142a via the parameter setting model 4141a. The same applies to the target range setting models 4142b to 4142d and the parameter setting model 4141b described below.

The target range setting model 4142b may output the lower limit of the target range to be set for the parameter in response to the input of the operation plan and the identification information of the parameter. In the present embodiment, as an example, in response to the input of the operation plan, the identification information of the parameter Pa, and the upper limit V _PaMAX of the target range of the parameter Pa from the parameter setting model 4141a to which the operation plan has been input by the first supply unit 415, the target range setting model 4142b supplies the operation plan, the identification information of the parameter Pa, and the upper and lower limits V _PaMAX and V _PaMIN of the target range of the parameter Pa to the parameter setting model 4141b.

In response to input of an operation plan and identification information of parameter Pa for which a target range has already been set, the parameter setting model 4141b may output identification information of another parameter (also referred to as parameter Pb) for which a target range should be set. In the present embodiment, as an example, in response to input of an operation plan, identification information of parameter Pa, and upper and lower limit values V _PaMAX and V _PaMIN of the target range from the target range setting model 4142b, the parameter setting model 4141b supplies the operation plan, identification information of parameter Pa, upper and lower limit values V _PaMAX and V _PaMIN , and identification information of parameter Pb different from parameter Pa to the target range setting model 4142c.

The target range setting model 4142c may output the upper limit of the target range to be set for the parameter in response to input of the operation plan and the identification information of the parameter. In the present embodiment, as an example, in response to the operation plan, the identification information of the parameter Pa, the upper and lower limit values V PaMAX and V _PaMIN of the target range, and the identification information of the parameter Pb being supplied from the parameter setting model 4141b, the target range setting model 4142c supplies the operation plan, the identification information of the parameter Pa, the upper and lower limit values V _PaMAX and V _PaMIN of the target range of the parameter Pa, the identification information of the parameter _Pb , and the upper limit value V _PbMAX of the target range of the parameter Pb to the target range setting model 4142d.

The target range setting model 4142d may output the lower limit of the target range to be set for the parameter in response to input of the operation plan and the identification information of the parameter. In the present embodiment, as an example, in response to the operation plan, the identification information of the parameter Pa, the upper and lower limit values V _PaMAX and V _PaMIN of the target range, the identification information of the parameter Pb, and the upper limit value V _PbMAX of the target range of the parameter Pb being supplied from the target range setting model 4142c, the target range setting model 4142d outputs the operation plan, the identification information of the parameter Pa, the upper and lower limit values V _PaMAX and V _PaMIN of the target range of the parameter Pa, and the identification information of the parameter Pb, and the upper and lower limit values V _PbMAX and V _PbMIN of the target range of the parameter Pb.

According to the above-described target setting model 414A, when an operation plan is input to the parameter setting model 4141, identification information of the parameters for which the target range should be set is output. Therefore, the parameters for which the target range is set can be matched to the parameters of the target setting data used to train the operation model 401.

In addition, in response to inputting an operation plan and identification information of parameter Pa for which a target range has already been set to parameter setting model 4141b, identification information of parameter Pb for which a target range should be set is output. Therefore, parameter Pb for which a target range is set can be matched to a parameter used in the target setting data together with parameter Pa in the learning of operation model 401.

Furthermore, in response to inputting an operation plan, identification information of parameter Pa for which a target range has already been set, and the upper and lower limits _VPaMAX and _VPaMIN of the target range to the parameter setting model 4141b, identification information of parameter Pb for which a target range should be set is output. Therefore, the parameter Pb for which a target range is set can be matched to the parameter used in the target setting data together with the identification information and the upper and lower limits _VPaMAX and _VPaMIN of parameter Pa in learning the operation model 401.

In addition, in response to inputting an operation plan and identification information of parameters Pa and Pb for which the target ranges are to be set to the target range setting model 4142, the target ranges to be set for the parameters Pa and Pb are output. Therefore, the target ranges to be set for parameters Pa and Pb can be matched to the target ranges of the target setting data used to train the operation model 401.

Furthermore, in response to an operation plan being input to the parameter setting model 4141a, identification information of the parameter Pa for which a target range is to be set is output from the parameter setting model 4141a, and in response to an operation plan being input to the target range setting model 4142a and the identification information of the parameter Pa being input from the parameter setting model 4141, the target ranges V _PaMAX and V _PaMIN to be set for the parameter Pa are output from the target range setting model 4142a. Therefore, the parameters of the target setting data and the target ranges can be automatically acquired in sequence.

The parameter setting model 4141a of the above target setting model 414A may be subjected to learning processing by the first learning processing unit 422 using learning data including the parameter identification information acquired by the first acquisition unit 421 and the operation plan. This makes it possible to omit the target range of the parameters from the learning data for learning the parameter setting model 4141a, thereby facilitating the learning processing.

The parameter setting model 4141b and the target range setting models 4142a to 4142d may be subjected to learning processing by the first learning processing unit 422 using learning data including the parameter identification information and target range acquired by the first acquisition unit 421, and the operation plan. This allows the contents of the output data from the model to be approximated to the contents of the target setting data used in learning the operation model 401 to achieve the operation plan.

[6. Other Modifications]
In the above embodiment, the device 4 has been described as having the operation model 401 and the target setting model 414, but it may not have either of them. When the device 4 does not have the operation model 401 and the target setting model 414, the device 4 may perform a learning process on the operation model 401 and the target setting model 414 in an externally connected storage device, may perform an evaluation on the operation model 401 in an externally connected storage device, or may perform an operation using the operation model 401 in an externally connected storage device.

Although the device 4 has been described as having a first acquisition unit 421 and a first learning processing unit 422 to perform learning processing of the goal setting model 414, these may not be included. In this case, the device 4 may perform learning processing of the operation model 401 using the learned goal setting model 414. The learned goal setting model 414 may be shared by multiple devices 4, and each device 4 may perform learning processing of a different operation model 401.

In addition, although the device 4 has been described as having a second learning processing unit 413 and the like to perform learning processing of the operation model 401, this need not be the case. In this case, the device 4 may externally acquire identification information of parameters in the target setting data used in learning the operation model 401, and perform learning processing of the target setting model 414.

In addition, the evaluation value acquisition unit 434 has been described as acquiring a reference evaluation value and a model evaluation value according to the simulation results, but it may also acquire a reference evaluation value and a model evaluation value according to the results of actually operating the equipment 2.

In addition, although the evaluation value acquisition unit 434 has been described as calculating the reference evaluation value, the reference evaluation value may be stored in advance in the device 4 as a fixed value.

Various embodiments of the present invention may also be described with reference to flow charts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of an apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable medium, and/or a processor provided with computer readable instructions stored on a computer readable medium. Dedicated circuitry may include digital and/or analog hardware circuitry and may include integrated circuits (ICs) and/or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

A computer-readable medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, and the like.

The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.

The computer-readable instructions may be provided to a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc., to execute the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

10 shows an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part. A program installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to an embodiment of the present invention, and/or to perform a process or steps of the process according to an embodiment of the present invention. Such a program may be executed by the CPU 2212 to cause the computer 2200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input/output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input/output controller 2220. The computer also includes legacy input/output units such as a ROM 2230 and a keyboard 2242, which are connected to the input/output controller 2220 via an input/output chip 2240.

The CPU 2212 operates according to the programs stored in the ROM 2230 and the RAM 2214, thereby controlling each unit. The graphics controller 2216 retrieves image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into itself, and causes the image data to be displayed on the display device 2218.

The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and/or writes programs and data to an IC card.

ROM 2230 stores therein a boot program, etc., executed by computer 2200 upon activation, and/or a program that depends on the hardware of computer 2200. I/O chip 2240 may also connect various I/O units to I/O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. The information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by realizing the manipulation or processing of information according to the use of the computer 2200.

For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

The CPU 2212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and may undergo information processing. CPU 2212 may perform various types of processing on data read from RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search/replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to RAM 2214. CPU 2212 may also search for information in a file, database, etc. in the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, CPU 2212 may search for an entry that matches a condition in which an attribute value of the first attribute is specified from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

The above-described program or software module may be stored on a computer-readable medium on the computer 2200 or in the vicinity of the computer 2200. Also, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network.

The present invention has been described above using an embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiment. It is clear from the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and it should be noted that the processes may be performed in any order, unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is necessary to perform the processes in this order.

REFERENCE SIGNS LIST 1 System 2 Facility 4 Apparatus 401 Operation model 402 Operation section 403 Parameter acquisition section 411 Input section 412 Storage section 413 Second learning processing section 414 Goal setting model 415 First supply section 416 Second acquisition section 417 Second supply section 421 First acquisition section 422 First learning processing section 431 Simulator 432 Display control section 433 Target range acquisition section 434 Evaluation value acquisition section 435 Evaluation section 2200 Computer 2201 DVD-ROM
2210 host controller 2212 CPU
2214 RAM
2216 Graphic controller 2218 Display device 2220 Input/output controller 2222 Communication interface 2224 Hard disk drive 2226 DVD-ROM drive 2230 ROM
2240 Input/Output Chip 2242 Keyboard 4141 Parameter Setting Model 4142 Target Range Setting Model

Claims (14)

a supply unit that supplies the value of a state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of the state parameter related to the equipment;
a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the value of the state parameter to the operation model;
an acquisition unit that acquires a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition unit;
an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value calculated based on a result of inputting a manual operation into a simulator of the equipment ;
An apparatus comprising : a supply unit that supplies the value of a state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of the state parameter related to the equipment;
a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the value of the state parameter to the operation model;
an acquisition unit that acquires a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition unit;
an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value according to a result of operating the equipment by manual operation;
Equipped with
The model evaluation value is calculated based on whether a parameter related to the equipment operated according to the recommended value falls within a target range;
An apparatus in which the reference evaluation value is calculated based on whether or not a parameter related to the equipment operated by manual operation falls within the target range. The device according to claim 2, further comprising a target range acquisition unit that acquires the target range set by an operator for a selected parameter selected by the operator from among multiple types of parameters related to the equipment. The device according to claim 3, further comprising a display control unit that displays the value of the selected parameter in past operations of the equipment in response to the selection of the selected parameter from the multiple types of parameters. The device according to claim 4, wherein the display control unit displays the values of each selected parameter in past operations of the equipment in a coordinate space with each selected parameter as a coordinate axis. The facility is a facility for manufacturing an object,
The apparatus according to claim 2 , wherein the parameter relating to the equipment is at least one of an index value indicating a quality of the product or a production volume of the product. The device according to any one of claims 1 to 6, wherein the evaluation unit evaluates the operation model as good if the model evaluation value is better than the reference evaluation value. The device according to any one of claims 1 to 7, wherein the model evaluation value is calculated based on the result of inputting the recommended value acquired by the control parameter acquisition unit into a simulator of the equipment. The device according to any one of claims 1 to 8, further comprising a learning processing unit that executes learning processing of the operation model using learning data including values of state parameters and values of control parameters. The device according to claim 9, wherein the learning processing unit executes learning processing of the operation model using the learning data and a reward value determined by a preset reward function. a supply step of supplying the value of the state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to input of the value of the state parameter indicating a state of the equipment;
a control parameter acquisition step of acquiring recommended values of control parameters output from the operation model in response to the supply of the state parameter values to the operation model in the supply step;
an acquisition step of acquiring a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired in the control parameter acquisition step;
an evaluation stage in which the operation model is evaluated based on the model evaluation value and a reference evaluation value calculated based on a result of inputting a manual operation into a simulator of the equipment ;
A method for providing the above. a supply step of supplying the value of the state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to input of the value of the state parameter indicating a state of the equipment;
a control parameter acquisition step of acquiring recommended values of control parameters output from the operation model in response to the supply of the state parameter values to the operation model in the supply step;
an acquisition step of acquiring a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired in the control parameter acquisition step;
an evaluation stage in which the operation model is evaluated based on the model evaluation value and a reference evaluation value according to a result of operating the equipment by manual operation;
Equipped with
The model evaluation value is calculated based on whether a parameter related to the equipment operated according to the recommended value falls within a target range;
The method in which the reference evaluation value is calculated based on whether or not a parameter related to the equipment operated by manual operation falls within the target range. Computer,
a supply unit that supplies the value of a state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of the state parameter related to the equipment;
a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the value of the state parameter to the operation model;
an acquisition unit that acquires a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition unit;
a program that functions as an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value calculated based on a result of inputting manual operations into a simulator of the equipment . Computer,
a supply unit that supplies the value of a state parameter to an operation model that outputs a recommended value of a control parameter of the equipment in response to an input of a value of the state parameter related to the equipment;
a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model in response to the supply unit supplying the value of the state parameter to the operation model;
an acquisition unit that acquires a model evaluation value corresponding to a result of operating the equipment using the recommended value acquired by the control parameter acquisition unit;
an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value according to a result of operating the equipment by manual operation;
The model evaluation value is calculated based on whether a parameter related to the equipment operated according to the recommended value falls within a target range;
A program in which the reference evaluation value is calculated based on whether or not a parameter related to the equipment operated by manual operation falls within the target range.