JP7443609B1 - Learning devices, learning methods and learning programs - Google Patents

Abstract

The present invention aims to improve model accuracy in imitation learning by using learning data suitable for sequential learning using the JIT method. A processing device 10 includes a collection unit 132 that collects a history that is a combination of an explanatory variable that represents a situation in a product production process and a target variable that represents an operation of equipment in the production process; A registration unit 133 adds at least an autopilot flag indicating whether or not the automatic operation is performed and registers it in the history DB 121, and from the history registered in the history DB 121, at least the autopilot flag for a predetermined period earlier than the current time is automatically operated. and an updating unit 135 that uses the history acquired by the acquisition unit 134 to update a model that outputs an objective variable from an explanatory variable. [Selection diagram] Figure 2

Description

The present invention relates to a learning device, a learning method, and a learning program.

2. Description of the Related Art Conventionally, a technique called imitation learning is known in which human behavior is learned by a machine learning model, and the model is used to teach motion to humans, robots, or the like.

Also, just-in-time (just-in-time), which accumulates a large amount of observed data, extracts data near the required point from the accumulated data, and sequentially trains the model using the extracted data. A technique called the JIT method is known (for example, see Non-Patent Document 1).

For example, in a chemical plant, changes in the environment occur over time, such as deterioration of equipment over time, deterioration of catalysts, and changes in production load plans.

On the other hand, it is conceivable to apply the JIT method to imitation learning that learns the operation of equipment by operators in chemical plants to adapt the model to changes in the environment.

JP 2019-185194 Publication

Shigeru Yamamoto, “Just-In-Time Predictive Control: Predictive Control Based on Accumulated Data”, Measurement and Control Vol. 52, No. 10, October 2013 issue (https://www.jstage.jst.go.jp/article /sicejl/52/10/52_878/_pdf/-char/ja)

A plant operation support system using a model that performs imitation learning has been proposed. For example, imitation learning is performed by having a model learn operations actually performed by an operator at a chemical plant. Specifically, the model learns the input amounts of raw materials input by operators in the past in a particular process. The model then outputs the raw material input amount as a recommended operation value. Operators can imitate past operator operations by setting raw material inputs according to the model's output.

Using a model that performs such imitation learning, it becomes possible not only to support plant operation but also to autopilot operations.

Here, operation data is accumulated even during autopilot operation of the plant. However, the data in the autopilot is for operations output by the model for the autopilot, and even if the model is sequentially trained using the data in the autopilot, it will only reinforce the current model. For this reason, if sequential learning is performed using data from the autopilot, the accuracy of the model may deteriorate, making it impossible to respond to changes in actual plant conditions.

The present invention has been made in view of the above, and provides a learning device, a learning method, and a learning program that can improve the accuracy of a model by using learning data suitable for sequential learning using the JIT method in imitation learning. The purpose is to

In order to solve the above-mentioned problems and achieve the purpose, the learning device collects a history that is a combination of explanatory variables representing the situation in the product production process and objective variables representing the operation of equipment in the production process. a collection unit; a registration unit that adds at least first attached information indicating whether or not the operation is automatic by a control device to the history and registers it in a database; an acquisition unit that acquires a first history in which the first given information for a predetermined period before time is not an automatic operation by the control device; The present invention is characterized by comprising an updating unit that updates a model that outputs a target variable.

According to the present invention, in imitation learning, learning data suitable for sequential learning using the JIT method is used to improve the accuracy of a model.

FIG. 1 is a diagram illustrating a plant operation system. FIG. 2 is a diagram illustrating a configuration example of a processing device according to an embodiment. FIG. 3 is a diagram showing an example of a history DB. FIG. 4 is a diagram illustrating the processing of the processing device. FIG. 5 is a flowchart showing the processing procedure in the embodiment. FIG. 6 is a diagram showing an example of a computer that executes a program.

Embodiments of a learning device, a learning method, and a learning program according to the present application will be described in detail below based on the drawings. Note that the present invention is not limited to the embodiments described below.

[Embodiment]
[Configuration of embodiment]
First, the plant operation system will be explained using FIG. The plant operation system 1 is a system for managing and controlling product production processes in a plant. Plants include chemical plants for producing chemical products.

As shown in FIG. 1, it has a processing device 10, a terminal device 20, and a plant system 30.

The processing device 10 performs processing related to a model (machine learning model) for performing imitation learning. Processing device 10 can function as a learning device.

Furthermore, the processing device 10 and the plant system 30 are connected to each other via a network so that they can communicate data with each other. For example, networks are the Internet and intranets. When the processing device 10 satisfies predetermined autopilot (automatic operation) conditions set by an operator or the like and is instructed by the operator to start the autopilot, the processing device 10 starts the automatic operation of the plant system 30 using the model. Perform pilot control.

The plant system 30 may include equipment used in the production process and a distributed control system (DCS). For example, the equipment is a reactor, a cooler, a gas-liquid separator, etc.

The terminal device 20 is an information processing device such as a personal computer, a tablet terminal, and a smartphone, or a dedicated terminal for operating plant equipment.

The operator is a user who operates equipment included in the plant system 30 via the terminal device 20. Furthermore, when the processing device 10 indicates that the autopilot can be started, the operator may instruct the processing device 10 to start the autopilot. When the operator is presented with an instruction to stop the autopilot from the processing device 10, the operator instructs the processing device 10 to stop the autopilot. Note that the models used in the processing device 10 are appropriately managed by a system administrator or the like.

Based on FIG. 1, processing of each device of the plant operation system 1 will be explained.

The terminal device 20 operates the equipment of the plant system 30 according to the operator's operation (step S1). For example, the terminal device 20 is operated to set the temperature inside the device, the pressure inside the device, the target value of the production amount in the production process, the amount of raw materials to be input into the device, and the like.

The plant system 30 operates according to the operation from the terminal device 20 (step S2). Then, the plant system 30 transmits the operation history to the processing device 10 (step S3).

For example, the history includes sensor values of sensors installed at various locations in the plant system 30 and setting values set by operations from the terminal device 20. Further, the history may be time-series data in which each record has a time stamp.

The terminal device 20 transmits the autopilot conditions in response to the operator's operation (step S4). The autopilot conditions are set based on the error between the model's predicted value and the actual measured value. For example, the autopilot condition may be that the error between the model's predicted value and the actual measured value is less than a predetermined threshold for a predetermined number of times, or that the average number of times the error between the model's predicted value and the actual measured value is determined within the nearest range is less than a predetermined threshold. It must be less than The autopilot conditions may be set by a system administrator or the like.

The processing device 10 adds to the history collected from the plant system 30 at least an autopilot flag (first attached information) indicating whether or not the autopilot is performed by the processing device 10 . Then, the processing device 10 adds a manual operation flag (second attached information) indicating that the operation of the device is a manual operation to a history in which the operation of the device is a manual operation by an operator. The processing device 10 then registers each history in a history database (DB).

Next, the processing device 10 acquires learning data used for model learning (for example, machine learning) from the history, assigns weights, performs model learning, and performs inference using the model. (Step S5).

At this time, the processing device 10 refers to at least the autopilot flag and the manual operation flag and acquires the learning data of the model from the history stored in the history DB. Specifically, the processing device 10 selects, as learning data, a first history that is a predetermined period earlier than the current time (inference time) and in which the autopilot flag is "OFF" from the past history group. get. Then, the processing device 10 acquires, as learning data, a second history in which the autopilot flag is "ON" and the manual operation flag is "ON" from the past history group.

The processing device 10 uses the autopilot conditions to determine whether the processing device 10 can operate the autopilot. Details of each process of the processing device 10 will be described later.

Furthermore, the processing device 10 presents the operator's terminal device 20 with a guidance screen 21 showing the inference result and the autopilot implementation determination result (step S6). If the autopilot conditions are satisfied, the processing device 10 displays on the guidance screen 21 that the autopilot can now be started. When the terminal device 20 instructs the processing device 10 to start the autopilot (step S7), the processing device 10 performs autopilot control of the plant system 30 using the model (step S8).

Furthermore, when the autopilot condition is not satisfied during the autopilot operation, the processing device 10 displays an instruction to stop the autopilot on the guidance screen 21. When the terminal device 20 instructs to stop the autopilot (step S7), the processing device 10 stops the autopilot control of the plant system 30. Then, the terminal device 20 operates the equipment of the plant system 30 according to the operator's operation (step S1).

Here, the model learns the contents of the operator's operations by imitation learning. Therefore, other operators can imitate the operation by following the operation details obtained as a result of inference by the model.

The processing device 10 will be explained in detail using FIG. 2. FIG. 2 is a diagram showing a configuration example of the processing device 10 according to the embodiment.

As shown in FIG. 2, the processing device 10 includes a communication section 11, a storage section 12, and a control section 13.

The communication unit 11 performs data communication with other devices via the network. For example, the communication unit 11 is a NIC (Network Interface Card).

The storage unit 12 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disc. Note that the storage unit 12 may be a data-rewritable semiconductor memory such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory).

The storage unit 12 stores an OS (Operating System) and various programs executed by the processing device 10. The storage unit 12 stores a history DB 121 and model information 122.

The history DB 121 is information including history provided from the plant system 30. For example, records are accumulated in the history DB every minute. FIG. 3 is a diagram showing an example of the history DB 121. As shown in FIG. 3, the history DB 121 includes a list of explanatory variables such as time, operator, situation, and operation, and setting values that are objective variables. Further, the history DB 121 may include weights. Furthermore, an autopilot flag and/or a manual operation flag is added to the history in the history DB 121. Furthermore, the history in the history DB 121 is given a manual operation before/after flag (third given information).

The time indicates the time at which the operation is performed. The status item includes items of first temperature, second temperature, and first flow rate. The status items may include the first pressure, the second pressure, the concentration of gas generated in the production process, and the like.

The first temperature, second temperature, first pressure, second pressure, and first flow rate are sensor values of sensors installed at various locations in the plant system 30, respectively.

The first temperature, second temperature, first pressure, second pressure, and first flow rate are explanatory variables of the model, and are examples of explanatory variables that represent the situation in the product production process.

Note that the time is a timestamp indicating the date and time when the first temperature, second temperature, first pressure, second pressure, flow rate, and gas concentration were acquired.

The implementation details (operation) are, for example, setting values set by an operation from the terminal device 20. The set value may be a normalized value of the actually set value. Furthermore, the set value corresponds to the objective variable of the model.

The set value is an objective variable of the model, and is an example of an objective variable that represents the operation of equipment in the production process.

The autopilot flag indicates whether or not the processing device 10 (control device) is an autopilot (autopilot). When the autopilot flag is "ON", this is the history obtained when the autopilot implementation by the processing device 10 is "ON". When the autopilot flag is "OFF", this is the history obtained when the autopilot implementation by the processing device 10 is "OFF".

The manual operation flag indicates that the device is operated manually by an operator. When the manual operation flag is "ON", this is the history obtained when the manual operation was performed by an operator.

The manual operation flag before and after is a flag that is attached to the history within a predetermined period (for example, w minutes) before and after the history to which the manual operation flag is attached. The preceding and succeeding predetermined periods are set based on the operating status of the equipment, changes in each value in the past history, etc., and are updated as appropriate.

For example, in FIG. 3, the first temperature at time "13:21:01" is "102.1°C", the second temperature is "102.8°C", and the first flow rate is "311.5 m ³ /s". Yes, and the operation content is "203.5". "OFF" is assigned as the autopilot flag, and "ON" is assigned as the manual operation before/after flag (cell C1-9). This history is a history for a predetermined period before and after the time when the manual operation was performed by the operator.

For example, in FIG. 3, the first temperature at time "13:23:01" is "101.5°C", the second temperature is "102.3°C", and the first flow rate is "311.4 m ³ /s". Yes, and the operation content is "206.3". "OFF" is assigned as the autopilot flag, and "ON" is assigned as the manual operation flag (cell C3-8). This history is a history of manual operations performed by the operator.

For example, in FIG. 3, the first temperature at the time "15:33:01" is "102.5°C", the second temperature is "103.3°C", and the first flow rate is "311.4 m ³ /s". ”, indicating that the operation is “206.3”. "ON" is assigned as the autopilot flag, and "ON" is assigned as the manual operation flag (cell C9-8). This history is a history when a manual operation is performed during autopilot, that is, when an operator manually overwrites a predicted value calculated by a model while using autopilot.

The model information 122 is information such as parameters for constructing a model. For example, if the model is a neural network, the model information 122 is the weights and biases of each layer. Furthermore, the model information 122 includes parameters such as the order of preprocessing and the window width (window size) in moving average processing.

The control unit 13 controls the entire processing device 10 . The control unit 13 includes, for example, electronic circuits such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array). It is an integrated circuit.

Further, the control unit 13 has an internal memory for storing programs and control data that define various processing procedures, and executes each process using the internal memory. Further, the control unit 13 functions as various processing units by running various programs.

For example, the control unit 13 includes a reception unit 131, a collection unit 132, a registration unit 133, an acquisition unit 134, an update unit 135, an inference unit 136, a determination unit 137, a display control unit 138, and an autopilot control unit 139. Processing of each functional unit of the processing device 10 will be explained using FIG. 4. FIG. 4 is a diagram illustrating the processing of the processing device 10.

The reception unit 131 receives autopilot conditions from the terminal device 20, for example. The reception unit 131 not only accepts new autopilot conditions, but also accepts modifications and deletions (including partial deletion) of autopilot conditions. The receiving unit 131 receives an autopilot start instruction and an autopilot stop instruction from the terminal device 20 .

The collection unit 132 collects the history of operation in the plant system 30 ((1) in FIG. 4), and outputs the collected history to the registration unit 133. History is a combination of explanatory variables and objective variables. The collecting unit 132 also collects information indicating whether each history is caused by manual operation or by autopilot.

The registration unit 133 collects the ON state or OFF state of the autopilot for the plant system 30 ((2) in FIG. 4). The registration unit 133 collects update information on the ON state or OFF state of the autopilot for the plant system 30 from the autopilot control unit 139.

Then, the registration unit 133 sets the autopilot flag of each history to "ON" or "OFF" based on the information indicating whether the operation was performed manually or by the autopilot, and the information indicating the ON or OFF state of the autopilot. Make it. Further, the registration unit 133 sets a manual operation flag to "ON" for a history in which the device was operated manually by an operator (for example, cells C3-8, C9-8, and C10-8 in FIG. 3). In addition, the registration unit 133 sets the manual operation flag before and after the history within a predetermined period before and after the history with the manual operation flag "ON" out of the history (for example, cells C1-9 and C2- in FIG. 3). 9, C4-9, C5-9, C8-9, C11-9).

The registration unit 133 registers the flagged history in the history DB 121 ((3) in FIG. 4).

The acquisition unit 134 refers to each flag of the history from among the histories included in the history DB 121 and acquires similar history to be used as learning data based on the distance between the explanatory variable and the designated explanatory variable (see Fig. 4 (4)). The acquisition unit 134 may acquire similar histories from the histories included in the history DB 121 based on the distance between the explanatory variable and the designated explanatory variable as well as the weight.

When a history search key (explanatory variable and/or objective variable) is specified, the acquisition unit 134 acquires a past history group similar to this history search key from the history DB 121. The acquisition unit 134 acquires from the history DB 121 as a past history group (explanatory variables, objective variables) collected in situations similar to the history search key (for example, an explanatory variable indicating the current situation).

The specified explanatory variable is called a required point. For example, the required point is an explanatory variable (corresponding to each sensor value in the history DB 121) at a predetermined time. Note that the objective variable (set value) at the request point may be unknown.

Here, in the JIT method, a past history group is acquired based on the Euclidean distance between training data (corresponding to the history DB 121 of this embodiment), which is a multidimensional vector, and a request point, which is a multidimensional vector. For example, the acquisition unit 134 uses the JIT method to acquire k Nearest Neighbors (k-NN), which are k (k is an integer) records with small calculated Euclidean distances. Note that the distance between the training data and the required point is not limited to the Euclidean distance, and may be, for example, the Mahalanobis distance or cosine similarity.

Further, the acquisition unit 134 may acquire records by referring not only to the distance between the training data and the request point but also to the weight in the history DB 121. Here, if the larger the weight is, the more desirable the data is as an acquisition target, for example, the acquisition unit 134 preferentially acquires data that is the k-nearest neighbor and has a higher weight from the history DB 121.

Further, the acquisition unit 134 refers to each flag attached to the history, further selects the acquired past history group, and acquires similar history that is learning data.

From the past history group, the acquisition unit 134 acquires a first signal that is a predetermined period earlier than the current time (inferred time) (before the latest H hours (for example, 12 hours)) and whose autopilot flag is "OFF". Get history. In the case of the example in FIG. 3, the acquisition unit 134 acquires the history B11 that is 12 hours earlier than the current time (for example, 16:00:00) and whose autopilot flag is "OFF" in the first Obtain as history.

Thereby, the acquisition unit 134 can exclude the history during autopilot from the learning data based on the autopilot flag and time, and can acquire only the history manually created by the operator. Note that the H time is set based on the operator's operation interval and the like.

Then, the acquisition unit 134 acquires a second history in which the autopilot flag is "ON" and the manual operation flag is "ON" from the past history group. In the example of FIG. 3, the acquisition unit 134 acquires the history group B12 in which the autopilot flag is "ON" and the manual operation flag is "ON" as the second history.

Thereby, the acquisition unit 134 can include in the learning data a history in the case where an operator intervenes and manually overwrites the predicted value calculated by the model while using the autopilot. Since this history is considered to be the history of a situation that has never existed before, it is desirable to include it in model learning. Note that when acquiring the second history, the condition that the time is a predetermined period earlier than the current time at the time of acquiring the first history is set to OFF.

The acquisition unit 134 also acquires a third history in which the autopilot flag is "ON" and the manual operation before/after manual operation flag is "ON" from the past history group. In the example of FIG. 3, the acquisition unit 134 acquires the history groups B15 and B16 in which the autopilot flag is "ON" and the manual operation before and after manual operation flags are "ON" as the third history. The acquisition unit 134 makes it possible to reinforce learning regarding the operator's manual operations by including the history before and after the history of the operator's manual operations in the learning data.

The update unit 135 uses the learning data acquired by the acquisition unit 134 to learn a model that outputs an objective variable from explanatory variables ((5) in FIG. 4), and updates the model ((6) in FIG. )).

The update unit 135 generates an objective function that represents the difference between the objective variable calculated by inputting the explanatory variables into the model constructed from the model information 122 and the objective variable included in the learning data narrowed down by the acquisition unit 134. The parameters of the model, that is, the model information 122, are repeatedly updated until the learning termination condition is satisfied so that the objective function becomes smaller.

Note that if weights are assigned to the learning data, the learning data is learned using the assigned weights.

For example, the update unit 135 uses the second history with higher importance than the first history. Alternatively, the updating unit may use the third history with a higher degree of importance than other histories (for example, the first history). For example, the update unit 135 sets the importance of the second history and the third history to be higher than the first history. Typically, the same importance level is set for the second history and the third history. For example, when using a method of updating parameters to minimize the sum (average) of squared errors, the importance is determined based on the second history and/or the third history, rather than using a simple sum or average. It can be set by using a weighted average. The weight given to the history is set in advance, for example, depending on the type of each history, the time interval between the second history and the third history, and the like.

The inference unit 136 calculates a target variable by inputting explanatory variables for prediction into a model constructed from the updated model information 122. That is, the inference unit 136 performs inference processing ((7) in FIG. 4). The inferred objective variable is, for example, the content of the operation predicted from the situation.

The determining unit 137 uses the autopilot conditions to determine whether or not the autopilot can be implemented ((8) in FIG. 4). The determining unit 137 determines that it is possible to start implementing the autopilot if the calculation result based on the error between the target variable predicted by the model and the actual measured value satisfies the autopilot condition. The determination unit 137 determines that the autopilot cannot be implemented if the calculation result based on the error between the target variable predicted by the model and the actual measured value does not satisfy the autopilot condition.

The display control unit 138 causes the terminal device 20 to display a guidance screen 21 showing the result of the autopilot execution determination together with the inferred objective variable (for example, operation details). Thereby, the display control unit 138 presents the inference result and the autopilot implementation determination result to the operator.

For example, when it is determined that it is possible to start implementing the autopilot, the display control unit 138 displays presentation content indicating that it is possible to start implementing the autopilot, and an instruction button to start implementing the autopilot. A guidance screen including the following is displayed on the terminal device 20.

When the autopilot control unit 139 is instructed to start the autopilot from the terminal device 20, the autopilot control unit 139 performs autopilot control of the plant system 30 using the model ((10) in FIG. 4).

Alternatively, if it is determined that the autopilot cannot be started, the display control unit 138 displays a guidance screen on the terminal device 20 that includes an autopilot stop instruction and an autopilot stop button. let

When the autopilot control unit 139 is instructed to stop the autopilot from the terminal device 20, the autopilot control unit 139 stops the autopilot control of the plant system 30 and switches to manual operation by the operator ((9) in FIG. 4).

[Processing of embodiment]
The processing procedure in the embodiment will be explained using FIG. 5. FIG. 5 is a flowchart showing the processing procedure in the embodiment.

As shown in FIG. 5, the processing device 10 first collects the history of operation in the plant system 30 (step S11). The processing device 10 also collects information indicating whether each history is caused by manual operation or autopilot. The processing device 10 also collects the ON state or OFF state of the autopilot for the plant system 30.

Subsequently, the processing device 10 adds an autopilot flag, a manual operation flag, and A before/after manual operation flag is assigned and registered in the history DB 121 (step S12).

The processing device 10 refers to each flag of the history from among the history included in the history DB 121, and acquires it as a similar history that is learning data based on the distance between the explanatory variable and the designated explanatory variable (step S13 ). From the past history group, the processing device 10 selects a first history that is a predetermined period earlier than the current time and in which the autopilot flag is "OFF", a first history in which the autopilot flag is "ON", and a manual A second history in which the operation flag is "ON" and/or a third history in which the autopilot flag is "ON" and the manual operation before/after manual operation flag is "ON" is acquired.

The processing device 10 uses the learning data to learn a model that outputs an objective variable from explanatory variables (step S14), and updates the model (step S15).

The processing device 10 inputs explanatory variables for prediction (e.g., first temperature, second temperature, first flow rate, etc.) into the model constructed from the updated model information 122, thereby determining target variables (e.g., operation details). ) is inferred (step S16).

The processing device 10 uses the autopilot conditions to determine whether autopilot can be implemented (step S17).

When the processing device 10 determines that it is possible to start implementing the autopilot (step S17: Yes), the processing device 10 displays the presentation content indicating that it is possible to start implementing the autopilot, and an instruction button to start implementing the autopilot. A guidance screen including the following is displayed on the terminal device 20. When the processing device 10 is instructed to start the autopilot from the terminal device 20, the processing device 10 performs autopilot control of the plant system 30 using the model (step S18).

When the processing device 10 determines that it is impossible to start implementing the autopilot (step S17: No), the processing device 10 displays a guidance screen including an autopilot stop instruction and an autopilot stop button on the terminal device 20. let When the processing device 10 is instructed to stop the autopilot from the terminal device 20, the processing device 10 stops the autopilot control of the plant system 30, and switches the manual operation by the operator or continues the manual operation (step S19). .

[Effects of embodiment]
In this way, the processing device 10 according to the embodiment receives, for example, a history from the plant system 30 that is a combination of an explanatory variable representing the situation in the product production process and an objective variable representing the operation of equipment in the production process. collect. Then, the processing device 10 adds at least an autopilot flag and a manual operation flag to the history, and registers the history in the history DB 121.

From the histories registered in the history DB 121, the processing device 10 acquires at least a first history that is a predetermined period earlier than the current time and in which the autopilot flag is "OFF". The processing device 10 acquires a second history in which the autopilot flag is “ON” and the manual operation flag is “ON”.

The processing device 10 uses the acquired first history and second history to update the model that outputs the target variable from the explanatory variables. In this way, the processing device 10 excludes the history during autopilot from the learning data and updates the model using only the manual history of the operator, so that deterioration in model accuracy can be prevented. The processing device 10 then includes in the learning data a history of cases in which an operator interferes and manually overwrites the predicted value calculated by the model while using the autopilot, thereby providing unprecedented operator operation content. can be trained by a model.

Therefore, the processing device 10 can improve the accuracy of the model by using the first history and the second history suitable for sequential learning using the JIT method as learning data in imitation learning.

Furthermore, when updating the model, the processing device 10 uses the second history with a higher degree of importance than the first history, so that the processing device 10 uses the second history with a higher degree of importance than the first history, so that the contents of the operator's operation that did not exist before are included in the model as particularly important history. It is possible to train the model and improve the accuracy of the model.

Furthermore, the processing device 10 further updates the model using a third history in which the autopilot flag is “ON” and the manual operation before/after manual operation flag is “ON”. For this reason, the processing device 10 reinforces the learning regarding the operator's manual operation by including in the learning data the history of the time period during which the operator is monitoring before and after the manual operation during autopilot, and improves the accuracy of the model. It is possible to improve the When updating the model, by using the third history with a higher degree of importance than other histories, the details of operations during the time period when the operator is monitoring before and after manual operation during autopilot can be especially evaluated. This can be learned by the model as an important history, and the accuracy of the model can be improved.

Therefore, even during autopilot, the processing device 10 can update the model using only the history suitable for learning, thereby improving the inference accuracy of the model. In particular, since the processing device 10 updates the model using learning data suitable for sequential learning using the JIT method in imitation learning, even when autopilot is applied, the processing device 10 can properly operate the plant system 30 and update the model. Accuracy can be improved at the same time.

[System configuration, etc.]
Further, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices may be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured. Furthermore, each processing function performed by each device is realized in whole or in part by a CPU (Central Processing Unit) and a program that is analyzed and executed by the CPU, or by hardware using wired logic. It can be realized as Note that the program may be executed not only by the CPU but also by another processor such as a GPU.

Further, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified.

[program]
As one embodiment, the processing device 10 can be implemented by installing a learning program that executes the above-described learning process into a desired computer as packaged software or online software. For example, by causing the information processing device to execute the above learning program, the information processing device can be made to function as the processing device 10. The information processing device referred to here includes a desktop or notebook personal computer. In addition, information processing devices include mobile communication terminals such as tablet terminals, smartphones, mobile phones, and PHS (Personal Handyphone System), as well as slate terminals such as PDAs (Personal Digital Assistants). included in the category.

Furthermore, the processing device 10 can also be implemented as a server that uses a terminal device used by a user as a client and provides the client with services related to the above-mentioned learning process. For example, the server is implemented as a server device that provides a learning service in which the specification of a request point is input and a trained model is output. In this case, the server may be implemented as a Web server, or may be implemented as a cloud that provides services related to the above-mentioned learning processing by outsourcing.

FIG. 6 is a diagram showing an example of a computer that executes a program. Computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These parts are connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1090. Disk drive interface 1040 is connected to disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into disk drive 1100. Serial port interface 1050 is connected to, for example, mouse 1110 and keyboard 1120. Video adapter 1060 is connected to display 1130, for example.

The hard disk drive 1090 stores, for example, an OS 1091, application programs 1092, program modules 1093, and program data 1094. That is, a program that defines each process of the processing device 10 is implemented as a program module 1093 in which computer-executable code is written. Program module 1093 is stored in hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration of the processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

Furthermore, the setting data used in the processing of the embodiment described above is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary, and executes the processing of the embodiment described above.

Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in a removable storage medium, for example, and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from another computer via the network interface 1070.

1 Plant operation system 10 Processing device 20 Terminal device 30 Plant system 11 Communication section 12 Storage section 13 Control section 121 History DB
122 Model information 131 Reception unit 132 Collection unit 133 Registration unit 134 Acquisition unit 135 Update unit 136 Inference unit 137 Judgment unit 138 Display control unit 139 Autopilot control unit

Claims (5)

a collection unit that collects a history that is a combination of an explanatory variable that represents a situation in a product production process and a target variable that represents an operation of equipment in the production process;
a registration unit that adds at least first addition information indicating whether or not the operation is automatic by a control device to the history and registers it in a database;
an acquisition unit that acquires at least a first history in which the first given information for a predetermined period before the current time is not an automatic operation by the control device, from the history registered in the database;
an updating unit that uses the history acquired by the acquisition unit to update a model that outputs the objective variable from the explanatory variable;
A learning device characterized by having. The registration unit adds third attached information to a history within a predetermined period before and after a history in which the operation of the device is a manual operation by an operator, among the history,
The acquisition unit acquires a history to which the first added information indicating that the operation is automatic by the control device is added, from the history registered in the database, and to which the third added information is added. The learning device according to claim 1, wherein a third history is acquired. The learning device according to claim 2, wherein the updating unit uses the third history with higher importance than other histories. A learning method executed by a learning device, comprising:
a step of collecting a history that is a combination of explanatory variables representing the situation in the product production process and objective variables representing the operation of equipment in the production process;
Adding at least first added information indicating whether or not the operation is automatic by a control device to the history and registering it in a database;
from the history registered in the database, acquiring at least a first history in which the first given information is not automatically operated by the control device for a predetermined period before the current time;
updating a model that outputs the objective variable from the explanatory variable using the history acquired in the acquiring step;
A learning method characterized by including. collecting a history that is a combination of an explanatory variable representing the situation in a product production step and a target variable representing the operation of equipment in the production step;
Adding at least first added information indicating whether or not the operation is automatic by a control device to the history and registering it in a database;
from the history registered in the database, acquiring at least a first history in which the first given information for a predetermined period before the current time is not an automatic operation by the control device;
updating a model that outputs the objective variable from the explanatory variable using the history acquired in the acquiring step;
A learning program for making a computer execute

Images