JP6837615B1 - Monitoring device - Google Patents

Abstract

The monitoring device (5) includes a worker information acquisition unit (21) for acquiring worker information including text data obtained by converting the position information and voice data of the worker terminal (3) into text, and a production device (4). The device information acquisition unit (23) that acquires device information including historical data of operations performed on) and sensor information indicating the sensing result by the sensor, event information indicating an event that occurred at the production site, and event occurrence. An event that occurs at the production site and an event occur based on the worker information and device information acquired at the timing and the worker information and device information acquired during a certain period before the event occurrence timing. A trained model storage unit (123) that stores the trained model generated by learning the state of the worker at the time and the state of the production device, the trained model, the newly acquired worker information, and the trained model. It includes a reasoning unit (252) that predicts an event that occurs at the production site based on the device information.

Description

The present disclosure relates to a monitoring device for monitoring the situation at a production site.

At the production site, the information obtained at the production site is monitored, and the monitoring results are used for various purposes such as improving production efficiency, supporting workers, and maintaining the production system.

For example, in Patent Document 1, when a trouble occurs, information such as the content of work performed by the worker and the work method for dealing with the trouble is stored in association with the information of the place where the trouble occurred. An invention that allows an operator to check information such as work contents and work methods for troubleshooting that was performed in the past at the place where a new trouble occurred when a new trouble occurs is described. Has been done.

Japanese Unexamined Patent Publication No. 2017-068397

According to the invention described in Patent Document 1, when an event such as a trouble occurs, the time required for the necessary work can be shortened and the work efficiency can be improved. On the other hand, if an event that may occur in the future can be predicted before the event occurs, the necessary work can be prepared in advance after the event occurs, and in this case as well, the necessary work after the event occurs can be performed efficiently. Is thought to be possible. For example, if a failure of the device can be predicted, it is possible to make advance preparations such as securing parts necessary for repairing the device and securing maintenance workers. Further, if the stop of the device can be predicted, it is possible to take measures such as eliminating the cause of the stop of the device in advance.

The present disclosure has been made in view of the above, and an object of the present invention is to obtain a monitoring device capable of notifying an operator of an event occurring at a production site in advance.

In order to solve the above-mentioned problems and achieve the purpose, the present disclosure is a monitoring device for monitoring the situation at the production site, and the position information of the worker terminal held by the worker and the conversation of the worker. The worker information acquisition unit that acquires the worker information including the text data obtained by converting the voice data including the contents into text, the history data of the operations performed on the production equipment installed at the production site, and the history data. A device information acquisition unit that acquires device information including sensor information indicating a sensing result by a sensor provided in the production device. In addition, the monitoring device includes event information indicating an event that occurred at the production site, worker information and device information acquired at the event occurrence timing, which is the timing at which the event occurred, and a certain period before the event occurrence timing. Based on the acquired worker information and device information, learn the event that occurs at the production site, the state of the worker when the event occurs, and the state of the production device when the event occurs. It is provided with a trained model storage unit that stores the generated trained model. In addition, when the worker information and the device information are newly acquired, the monitoring device predicts an event that occurs at the production site based on the learned model and the newly acquired worker information and the device information. It has a reasoning unit.

The monitoring device according to the present disclosure has an effect that it is possible to notify the operator in advance of an event occurring at the production site.

The figure which shows the configuration example of the monitoring system which is configured including the monitoring device which concerns on embodiment. Diagram showing a configuration example of a learning device The figure which shows the configuration example of the data acquisition part provided in the learning device Flowchart showing learning operation by learning device The figure which shows the configuration example of the inference device Flowchart showing inference operation by inference device The figure which shows an example of the method in which a worker terminal notifies a prediction result by a monitoring device.

Hereinafter, the monitoring device according to the embodiment of the present disclosure will be described in detail with reference to the drawings. The present disclosure is not limited to this embodiment.

Embodiment.
FIG. 1 is a diagram showing a configuration example of a monitoring system including the monitoring device according to the embodiment. The monitoring system 100 includes a management server 1, a field device 2, a worker terminal 3, and a production device 4.

The management server 1 has a storage device 12 that receives and stores information acquired from the worker terminal 3 and the production device 4 by the field device 2, an event that occurs at the production site, and the state and event of the worker when the event occurs. It includes a learning device 11 that learns the state of the production device 4 when it occurs by using the information stored in the storage device 12. Examples of events that occur at the production site include a stoppage of the production device 4, a failure of the production device 4, a repair of the production device 4, an injury of a worker, and the like. The details of the learning device 11 will be described later.

The storage device 12 includes a worker information storage unit 121, a device information storage unit 122, and a learned model storage unit 123. The worker information storage unit 121 stores the worker information configured by the field equipment 2 including various information regarding the worker's state acquired from the worker terminal 3. The device information storage unit 122 stores device information configured by the field device 2 including various information regarding the state of the production device 4 acquired from the production device 4. The trained model storage unit 123 stores the trained model generated by the learning device 11. The management server 1 may be connected to the field device 2 via the Internet or the like, that is, the monitoring system 100 may be realized by using cloud computing.

The field device 2 is, for example, an industrial personal computer installed at a production site, and has a function of communicating with a worker terminal 3 and a production device 4 via a wired line or a wireless line. The field device 2 may be a programmable logic controller, a programmable display, or the like. In the configuration example shown in FIG. 1, one field device 2 is used, but a configuration including two or more field devices 2 may be included.

The field equipment 2 acquires various information output from the worker terminal 3 and transmits the worker information to the management server 1 and the worker information acquisition unit 21 that collectively generates the worker information by collecting the acquired various information. The worker information transmission unit 22, the device information acquisition unit 23 that acquires various information output from the production device 4 and generates the device information by collecting the acquired various information, and the device that transmits the device information to the management server 1. It includes an information transmission unit 24. Here, the worker information is assumed to include time information indicating the time when various information is acquired and identification information of the worker terminal 3 from which the information is acquired. Further, the device information includes time information indicating the time when various information is acquired and identification information of the production device 4 from which the information is acquired. The worker information acquisition unit 21 repeatedly acquires the worker information at regular intervals. Similarly, the device information acquisition unit 23 repeatedly acquires device information at regular intervals.

Further, the field device 2 includes an inference device 25. The inference device 25 is in a state in which learning by the learning device 11 of the management server 1 has been completed, that is, in a state in which the learned model storage unit 123 of the management server 1 stores the learned model, and the worker information acquisition unit 21 And when the device information acquisition unit 23 newly acquires the worker information and the device information, it is based on the newly acquired worker information and the device information and the learned model stored in the learned model storage unit 123. And perform inference processing. In the inference process performed by the inference device 25, an event occurring at the production site is inferred. The details of the inference device 25 will be described later.

Each component of the field device 2 constitutes the monitoring device 5 together with the learning device 11 and the storage device 12 included in the management server 1. Note that FIG. 1 shows an example in which the field equipment 2 acquires the above information (worker information, device information) from the worker terminal 3 and the production device 4 and transmits the above information (worker information, device information) to the management server 1. It is also possible to configure the management server 1 to have the functions of each component shown in FIG. 1 of the field equipment 2. That is, the management server 1 may be configured to directly acquire the above information from the worker terminal 3 and the production device 4.

The worker information transmission unit 22 transmits the worker information acquired by the worker information acquisition unit 21 to the management server 1 to some extent. The worker information transmission unit 22 transmits, for example, worker information to the management server 1 at regular intervals. Similarly, the device information transmission unit 24 also transmits the device information acquired by the device information acquisition unit 23 to the management server 1 to some extent. The timing at which the worker information transmitting unit 22 transmits the worker information and the timing at which the device information transmitting unit 24 transmits the device information may be the same or different.

The worker terminal 3 is a terminal device held by a worker at a production site, and corresponds to a tablet terminal, a smartphone, or the like. One or more worker terminals 3 are included in the monitoring system 100.

The worker terminal 3 has a display unit 31, an operation unit 32, a position information acquisition unit 33 that acquires the position information of the worker terminal 3, and a voice that collects sound and generates sound data including the conversation content of the worker. The acquisition unit 34, the video acquisition unit 35 that shoots and generates video data, and the voice conversion unit 36 that converts the content of the worker's conversation included in the sound data generated by the voice acquisition unit 34 into text data. Be prepared. The position information, the video data, and the text data constitute the above-mentioned worker information. That is, the worker information includes the position information of the worker terminal 3, the text data in which the voice of the worker is converted into text, the video data taken by the worker terminal 3, the time information, and the worker terminal 3. It is configured to include identification information. The worker terminal 3 may not be provided with the voice conversion unit 36. In this case, the field equipment 2 or the management server 1 converts the contents of the worker's conversation into text data as needed.

The production device 4 is various devices installed on the production line, and corresponds to, for example, an industrial robot. One or more production devices 4 are included in the monitoring system 100.

The production device 4 includes an information collecting unit 41 that collects information on the operating state of the production device 4, and a video acquisition unit 42 that shoots and generates video data. The information collected by the information collecting unit 41 includes an operation log 411 which is history data of operations performed by an operator on the production device 4, and sensor information 412 which shows sensing results by various sensors (not shown). The video data, the operation log 411, and the sensor information 412 constitute the device information described above. That is, the device information includes video data, operation log 411, sensor information 412, time information, and identification information of the production device 4.

Next, the details of the learning device 11 will be described with reference to FIGS. 2 to 4.

FIG. 2 is a diagram showing a configuration example of the learning device 11. For convenience of explanation, FIG. 2 also shows a storage device 12 connected to the learning device 11. As shown in FIG. 2, the learning device 11 includes a data acquisition unit 111 and a model generation unit 112. The learning device 11 is realized by the arithmetic unit 10 that constitutes the management server 1.

The data acquisition unit 111 extracts and learns information related to an event that has occurred at the production site from the worker information stored in the worker information storage unit 121 and the device information stored in the device information storage unit 122. Generate data for. FIG. 3 is a diagram showing a configuration example of the data acquisition unit 111 included in the learning device 11. The data acquisition unit 111 includes an event detection unit 111A and a learning data generation unit 111B. The event detection unit 111A analyzes the worker information stored in the worker information storage unit 121 and the device information stored in the device information storage unit 122 to detect an event generated at the production site. The learning data generation unit 111B is based on the worker information and device information acquired at the event occurrence timing and the worker information and device information generated in a certain period before the event occurrence timing. To generate. That is, the learning data generation unit 111B includes the worker information and device information acquired at the event occurrence timing, the worker information and device information generated in a certain period before the event occurrence timing, and the generated event. Data for learning is generated by associating with the event information indicating.

In the model generation unit 112, the data acquisition unit 111 generates a correspondence relationship between the event that occurs at the production site, the state of the worker when the event occurs, and the state of the production device 4 when the event occurs. A trained model is generated by training based on the training data. Here, as a machine learning algorithm in the model generation unit 112, a Bayesian network, a Monte Carlo method, or the like can be considered.

A flowchart of the learning operation by the learning device 11 is shown in FIG. FIG. 4 is a flowchart showing a learning operation by the learning device 11. The operation shown in FIG. 4 is started when, for example, the management server 1 receives an operation instructing the start of the learning operation from the user.

When the learning device 11 starts operation, first, the data stored in the storage device 12, specifically, the worker information stored in the worker information storage unit 121 and the device information storage unit 122 are stored. Start the analysis of the device information and check the event that occurred at the production site. This process is performed by the event detection unit 111A of the data acquisition unit 111. Specifically, the event detection unit 111A includes the first acquired worker information among the worker information stored in the worker information storage unit 121 and the device information stored in the device information storage unit 122. The device information, that is, the worker information and the device information acquired at the earliest time is analyzed, and the presence or absence of an event occurrence is confirmed (step S11). The event detection unit 111A determines that an event has occurred, for example, when the sensor information 412 included in the device information includes the sensor information 412 indicating an abnormal value. Further, the event detection unit 111A determines that an event has occurred when the operation log 411 included in the device information includes information indicating that the operation for urgently stopping the production device 4 has been executed. Further, the event detection unit 111A determines that the event has occurred when the text data included in the worker information includes words reminiscent of the occurrence of the event. Words that evoke the occurrence of an event are, for example, "ah", "eh", "bad", "that?", "Funny", and so on. For example, these words are registered in the management server 1 in advance as keywords reminiscent of the occurrence of an event, and the event detection unit 111A determines that an event has occurred when the registered keywords are detected. Further, the event detection unit 111A may determine whether or not an event has occurred from the operation of the operator obtained by analyzing the video data.

When the event detection unit 111A detects the occurrence of an event (step S11: Yes), the learning data generation unit 111B generates learning data (step S12). Specifically, the learning data generation unit 111B includes the occurrence timing of the detected event, that is, the occurrence time, and stores the worker information and the device information acquired in a certain period before the occurrence timing in the work of the storage device 12. Learning data is generated based on the extracted worker information and device information extracted from the person information storage unit 121 and the device information storage unit 122, and event information indicating the detected event. The learning data is composed of position information and text data in the information included in the worker information, operation logs 411 and sensor information 412 in the information included in the device information, and event information.

The event information is, for example, the name of the event. Here, the event information may be generated by the learning data generation unit 111B from the contents of the worker information and the contents of the device information, but the user of the management server 1 inputs the event information using an input device such as a keyboard or a mouse. You may do so. When the user inputs event information, for example, in the learning device 11, when the event detection unit 111A detects an event, the worker information and the device information at the time of event detection are not described in FIG. Is displayed, and the user is asked to confirm this display and input event information. Further, when the learning data generation unit 111B generates the event information, the generated event information may be displayed on the display device so that the user can confirm the contents and the user can modify the information as necessary.

When the creation of the learning data by the learning data generation unit 111B is completed, the model generation unit 112 performs learning processing using the learning data to generate a trained model (step S13). As described above, the learning process here uses existing algorithms such as the Bayesian network and the Monte Carlo method. The model generation unit 112 performs learning by associating the position information, the text data, the operation log 411 and the sensor information 412 included in the learning data with the event information.

When the learning process by the model generation unit 112 in step S13 is completed, the process returns to step S11, and the event detection unit 111A returns to the worker information and device information acquired at the earliest time among the unconfirmed worker information and device information. To check if an event has occurred.

When the event detection unit 111A does not detect the occurrence of an event at the production site (step S11: No), the event detection unit 111A confirms whether the data analysis is completed, that is, whether there is unconfirmed worker information and device information (step S14). ). When the data analysis is not completed (step S14: No), the process returns to step S11, and the event detection unit 111A returns to the worker information and the device acquired at the earliest time among the unconfirmed worker information and the device information. Analyze the information and check if an event has occurred.

On the other hand, when the data analysis is completed (step S14: Yes), the model generation unit 112 outputs the trained model to the trained model storage unit 123, and the trained model storage unit 123 stores this (step S15). , The learning operation by the learning device 11 is completed.

The learning device 11 generates a learned model in the past in a state where the learned model storage unit 123 stores the learned model, that is, the learning device 11 performs a learning operation according to the flowchart shown in FIG. When the operation for instructing the start of the learning operation is received in the completed case, re-learning is performed according to the flowchart shown in FIG. 4, and the learned model stored in the learned model storage unit 123 is updated.

Next, the details of the inference device 25 will be described with reference to FIGS. 5 and 6.

FIG. 5 is a diagram showing a configuration example of the inference device 25. For convenience of explanation, FIG. 5 also describes other devices and components connected to the inference device 25. As shown in FIG. 5, the inference device 25 includes a data acquisition unit 251 and an inference unit 252. The inference device 25 is realized by the arithmetic unit 20 that constitutes the field device 2.

The data acquisition unit 251 acquires data to be used in the inference process from the worker information output by the worker information acquisition unit 21 and the device information output by the device information acquisition unit 23. Specifically, the data acquisition unit 251 acquires the position information and the text data from the worker information, and also acquires the operation log 411 and the sensor information 412 from the device information. In the following description, for convenience, the position information, text data, operation log 411, and sensor information 412 acquired by the data acquisition unit 251 may be collectively referred to as inference data.

The inference unit 252 estimates an event that occurs at the production site based on the inference data acquired by the data acquisition unit 251 and the learned model stored in the learned model storage unit 123, and outputs the inference result. To do. The inference result output by the inference unit 252 is transmitted to, for example, the worker terminal 3. In this case, the worker terminal 3 displays the inference result 311 on the display unit 31. The inference result output by the inference unit 252 may be transmitted to other devices such as a management server 1, a production device 4, and a terminal device held by a manager at a production site (not shown).

FIG. 6 shows the inference operation by the inference device 25 in a flowchart. FIG. 6 is a flowchart showing an inference operation by the inference device 25. The operation shown in FIG. 6 is executed, for example, when the field equipment 2 acquires the worker information and the device information.

When the inference device 25 starts operation, the data acquisition unit 251 first performs data acquisition processing, and the inference device 25 is used for inference from the worker information output by the worker information acquisition unit 21 and the device information output by the device information acquisition unit 23. Acquire data (step S21). Next, the inference unit 252 performs inference processing using the inference data acquired by the data acquisition unit 251 and the learned model stored in the trained model storage unit 123, and generates an event that occurs at the production site. Predict (step S22). At this time, the inference unit 252 makes an inference using the inference data acquired in the past by the data acquisition unit 251 and predicts where in the production site, which event will occur in the future, and with what probability. .. The inference unit 252 outputs an inference result when the inference process is completed (step S23).

The inference result output by the inference unit 252 is transmitted to the worker terminal 3, the terminal device held by the manager at the production site, and the like, and is notified to the worker, the manager, and the like via these devices.

As an example, a notification method in which the worker terminal 3 notifies the worker of the inference result will be described. FIG. 7 is a diagram showing an example of a method in which the worker terminal 3 notifies the prediction result by the monitoring device 5. In the example shown in FIG. 7, the worker terminal 3 superimposes the prediction result on the video acquired by the video acquisition unit 35 and notifies the worker. FIG. 7 is an example of a notification method when the inference unit 252 predicts the occurrence of an event related to the production device 4, such as when the production device 4 is stopped, and the location (device) where the event is predicted to occur is indicated. Hatchings 321 and 322 are provided to indicate the effect. If different events are predicted to occur, different hatching will be used for each event. Places with a high probability of event occurrence may be emphasized as hatching that is different from the surroundings. Note that hatching is an example, and the type of event and the probability of occurrence may be determined using text display, color, symbol, and the like. Details of the event can be confirmed by moving the mouse over the place where the event is expected to occur, or by displaying a pop-up when the place where the event is expected to occur is touched. By using the notification method as shown in FIG. 7, the worker can know at a glance where the event occurs at the production site.

In the present embodiment, the inference result by the inference device 25 is transmitted to another external device such as the worker terminal 3, and the other device notifies the inference result. However, the field device 2 provided with the inference device 25 May be notified. Further, all or a part of a plurality of devices existing at the production site or in the vicinity of the production site, such as the field device 2 and the worker terminal 3, may be configured to notify the inference result.

As described above, the monitoring device 5 according to the present embodiment is obtained from the worker information acquired from the worker terminal 3 held by the worker who works at the production site and the production device 4 installed at the production site. A learning device 11 is provided that learns an event that occurs at a production site, a state of a worker when the event occurs, and a state of the production device 4 based on the acquired device information, and generates a trained model. .. Further, when the monitoring device 5 acquires the worker information and the device information in the state where the learned model has been generated by the learning device 11, the monitoring device 5 is based on the acquired worker information and the device information and the learned model at the production site. Predict the events that will occur. This makes it possible to notify workers in advance of events that occur at the production site, which can contribute to the realization of risk prediction and preventive maintenance. Further, by feeding back the prediction result to the worker, it can be used to prevent troubles related to the production device 4 and the worker, and the safety and reliability of the production site can be improved.

The configuration shown in the above-described embodiment shows an example of the contents of the present disclosure, can be combined with another known technique, and is one of the configurations as long as it does not deviate from the gist of the present disclosure. It is also possible to omit or change the part.

1 management server, 2 field equipment, 3 worker terminals, 4 production equipment, 5 monitoring equipment, 10, 20 arithmetic equipment, 11 learning equipment, 12 storage equipment, 21 worker information acquisition unit, 22 worker information transmission unit, 23 Device information acquisition unit, 24 device information transmission unit, 25 inference device, 31 display unit, 32 operation unit, 33 position information acquisition unit, 34 voice acquisition unit, 35, 42 video acquisition unit, 41 information collection unit, 100 monitoring system, 111,251 Data acquisition unit, 111A event detection unit, 111B learning data generation unit, 112 model generation unit, 121 worker information storage unit, 122 device information storage unit, 123 learned model storage unit, 252 inference unit, 311 inference Results, 411 operation logs, 412 sensor information.

Claims (6)

It is a monitoring device that monitors the situation at the production site.
A worker information acquisition unit that acquires worker information including the position information of the worker terminal held by the worker and text data obtained by converting voice data including the contents of the worker's conversation into text.
A device information acquisition unit that acquires device information including historical data of operations performed on a production device installed at the production site and sensor information indicating a sensing result by a sensor provided in the production device. ,
The event information indicating the event that occurred at the production site, the worker information and the device information acquired at the event occurrence timing that is the timing at which the event occurred, are acquired in a certain period before the event occurrence timing. Based on the worker information and the device information, the event that occurs at the production site, the state of the worker when the event occurs, and the state of the production device when the event occurs are learned. A trained model storage unit that stores the trained model generated in
When the worker information and the device information are newly acquired, an event occurring at the production site is predicted based on the learned model and the newly acquired worker information and the device information. Inference part and
A monitoring device characterized by being provided with. The event information, the position information, the text data, the history data, and the sensor information acquired at the event occurrence timing, and the position information, the text data, the history data, and the sensor information acquired in the fixed period. Based on the sensor information, the learning has been completed by learning the correspondence between the event that occurs at the production site, the state of the worker when the event occurs, and the state of the production device when the event occurs. Model generator that generates a model,
The monitoring device according to claim 1, further comprising. A display unit that displays the prediction result by the inference unit,
The monitoring device according to claim 1 or 2, wherein the monitoring device is provided. The inference unit predicts the event that occurs at the production site and the place where the event occurs, and
Based on the prediction result by the inference unit, the display unit displays that the event is predicted to occur at a place in the image of the production site where the event is predicted to occur.
The monitoring device according to claim 3. The inference unit transmits a prediction result of an event occurring at the production site to the worker terminal, and causes the worker terminal to notify the worker of the prediction result.
The monitoring device according to claim 1 or 2. The inference unit predicts an event that occurs at the production site and a place where the event occurs, and displays a display on the worker terminal indicating that the event is predicted to occur at the event in the image of the production site. Let it be done in the place where the occurrence of
The monitoring device according to claim 5.

Images