JP2019197373A - Notification device, information processing device, information processing system, information processing method, and information processing program - Google Patents

Abstract

To provide a technology capable of preventing accidents by automatically predicting accident occurrence in construction sites and notifying the prediction results to the workers at the construction sites.SOLUTION: The present invention is a notification device, comprising: prediction means for predicting accident occurrence from an image photographed by a camera, based on a camera worn by workers at a construction site and a prescribed model; notification means for, when the accident occurrence is predicted, notifying the workers with the prediction; transmission means for generating a model from the image photographed by the camera and the correctness information of the prediction inputted by inputting means which the workers input the correctness of the prediction when the prediction is notified to the workers, by a machine learning having the learning data of an image where the construction site is captured, and sending thereof to a server updating the model by the machine learning having the learning data of the image photographed by the camera and the correctness information of the prediction; and acquisition means for acquiring the updated model from the server.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a notification device, an information processing device, an information processing system, an information processing method, and an information processing program.

  Various types of sensors are used to detect human physiological information and prevent accidents (for example, Patent Documents 1-2).

Japanese Patent No. 4038571 Japanese Patent No. 5476137

  At the construction site, accidents such as cuts due to the use of tools, falls from stepladders, falls due to rolling, and entanglement in equipment may occur. As for accidents occurring at the construction site, the cause of the accident, the form of injury of the worker due to the accident, the frequency of occurrence, etc. are usually recorded as data, and measures to prevent recurrence are usually taken based on the data related to the accident.

  However, even if a recurrence prevention measure is taken, there is a possibility that the accident will recur due to human error such as the operator forgetting the recurrence prevention measure.

  Therefore, it is conceivable to prevent accidents by automatically predicting the occurrence of an accident and notifying the operator of the prediction result when the worker works at the construction site, but the technology still exists. do not do.

  Therefore, the present application aims to provide a technology capable of preventing an accident in advance by automatically predicting the occurrence of an accident at the construction site and notifying the operator of the prediction result at the construction site. To do.

  In order to solve the above-described problems, the present invention is based on a model that is generated by machine learning using an image showing a construction site as learning data and is referenced when predicting the occurrence of an accident. I decided to predict.

  More specifically, the present invention is photographed by a camera based on a camera attached to an operator working at a construction site and a model referred to when an occurrence of an accident at the construction site is predicted from an image reflected on the construction site. Predicting means for predicting the occurrence of an accident from an object shown in the captured image, notifying means for notifying the operator of the occurrence of an accident when the predicting means is predicted, an image taken by the camera, and notification When the prediction is notified to the worker by the means, the model is obtained by machine learning using the image showing the construction site as learning data, and the prediction correctness information input by the input means by which the worker inputs the correctness of the prediction. A transmission means for generating and transmitting to the server that updates the model by machine learning using the image captured by the camera and the prediction correctness information as learning data; Comprising acquisition means for acquiring the updated model in over server from the server, and a notification device.

  If it is such a notification device, based on the model that is referenced when predicting the occurrence of an accident at the construction site from the image reflected in the construction site, from the object reflected in the image taken by the camera attached to the worker Predict the occurrence of an accident. Then, the operator is notified of the predicted occurrence of the accident. That is, by automatically predicting the occurrence of an accident and notifying the operator of the prediction, an accident by the worker is prevented in advance. In addition, the correctness information of the prediction of the occurrence of the accident input by the worker is transmitted to the server, and the notification device is updated by machine learning using the correctness information of the prediction of the occurrence of the accident as learning data from the server. Can be obtained. That is, the accuracy of predicting the occurrence of an accident in the notification device is improved.

  Further, the present invention provides a model generation means for generating a model to be referred to when predicting the occurrence of an accident at a construction site by machine learning using an image showing the construction site as learning data, and a model generated by the model generation means. Based on the above, the occurrence of an accident is predicted from the object shown in the image taken by the camera attached to the worker working at the construction site, the worker is notified of the occurrence, and the correctness information of the prediction notified by the worker is input The information is taken by the transmission means for transmitting the model to the information device, the image taken by the camera, and the prediction information entered by the operator from the information device and the camera received by the reception means. And a model updating unit that updates the model by machine learning using the prediction correctness information as learning data. It may be.

  Such an information processing apparatus generates a model that is referred to when an occurrence of an accident at a construction site is predicted by machine learning using an image showing the construction site as learning data. Therefore, it is possible to automatically predict the occurrence of an accident from the image showing the construction site. And based on the generated model, the occurrence of an accident is predicted from the object shown in the image taken by the camera attached to the worker who works at the construction site, and the worker is notified, and the worker notifies The generated model is transmitted to an information device that inputs prediction correctness information. That is, in the information device, it is possible to predict the occurrence of an accident from an object shown in an image photographed by a camera based on the model. Further, when the information device notifies the operator of the prediction of the occurrence of the accident, an accident by the operator can be prevented in advance.

  In addition, from the information device that has transmitted the model, the image taken by the camera attached to the worker and the correctness information of the prediction of the occurrence of the accident based on the model are received, and the correctness information of the prediction of the image and the accident is received The model is updated by machine learning using as learning data. Therefore, the accuracy of predicting the occurrence of an accident based on the model can be improved.

Further, the present invention is formed by a notification device for notifying an operator working at a construction site of an accident prediction and an information processing device for generating a model that is referred to when an accident occurrence is predicted from an image showing the construction site. The information processing system is an information processing system in which an accident occurs from a camera worn by an operator, an acquisition unit that acquires a model from the information processing device, and an object that is captured by an image captured by the camera based on the model. When the prediction means predicts the occurrence of an accident, the prediction means notifies the worker of the prediction, and the notification means notifies the worker of the prediction, the worker Input means for inputting the correctness of the prediction, an image captured by the camera, and a transmission means for transmitting the correctness information of the prediction input by the input means to the information processing apparatus; Provided, an information processing device includes a model generation unit that generates a model by machine learning using an image showing a construction site as learning data, a transmission unit that transmits the model generated by the model generation unit to the notification device, and a notification device An image photographed by the camera and a prediction correctness information input by the input means of the notification device are received from the notification device, an image photographed by the camera received by the reception means, and the prediction correctness information A model updating unit that updates the model by machine learning as learning data, and the acquisition unit of the notification device may be an information processing system that acquires the updated model from the information processing device.

  With such an information processing system, the information processing apparatus generates a model that is referred to when an occurrence of an accident at the construction site is predicted by machine learning using an image showing the construction site as learning data. And the produced | generated model is transmitted to the notification apparatus which notifies the prediction of generation | occurrence | production of an accident to the worker who works on a construction site.

  Then, the notification device predicts the occurrence of an accident from an object shown in an image taken by a camera attached to the worker based on the received model. Further, when the occurrence of an accident is predicted, the operator is notified of the prediction. Therefore, the occurrence of an accident is automatically predicted, and the operator is notified of the prediction, thereby preventing an accident by the worker.

  In addition, the worker inputs the correctness of the predicted occurrence of the notified accident. Then, the notification device transmits an image captured by the camera and correct / incorrect information on the prediction of the occurrence of the accident to the information processing device.

  In the information processing apparatus, the occurrence of an accident at the construction site generated by the generation unit is predicted by machine learning using the image taken by the camera received from the notification apparatus and the correctness information of the prediction of the occurrence of the accident as learning data. Update the model referenced in the case. Then, the notification device acquires the updated model, and predicts the occurrence of an accident from an object shown in an image taken by a camera attached to the worker. That is, with the information processing system, the accuracy of predicting the occurrence of an accident can be improved.

  The model may include an identification model for identifying an identification object from an image showing a construction site, a correspondence relationship between the identification object and an accident type caused by the identification object.

  With such an information processing system, if there is information about the type of accident that occurred in the past at the construction site and the correspondence relationship of the object that caused the accident, the identification target object is identified from the correspondence relationship. Correspondence with the type of accident caused by the object can be created. That is, it is possible to predict the occurrence of an accident by utilizing information on accidents that have occurred in the past at the construction site.

  The correspondence relationship may be a correspondence relationship between a plurality of identification objects and the types of accidents caused by the plurality of identification objects.

  In such an information processing system, when an accident is assumed to occur when a plurality of predetermined objects are gathered, a plurality of predetermined identification objects and an accident caused by the plurality of identification objects Correspondence with types can be generated and used to predict the occurrence of accidents.

The present invention can also be understood from a method aspect. That is, for example, it is formed by a notification device that notifies a worker working at a construction site of the prediction of an accident and an information processing device that generates a model that is referred to when predicting the occurrence of an accident from an image showing the construction site. An information processing method executed by an information processing system, in which a notification device acquires a model from an information processing device, and an object reflected in an image photographed by a camera attached to an operator based on the model A prediction step for predicting the occurrence of an accident, a notification step for notifying the worker of the prediction when the occurrence of the accident is predicted by the prediction step, and a worker when the prediction is notified to the worker by the notification step. Inputs the correctness of the prediction, the image taken by the camera, and the correctness of the prediction input by the input step The broadcast, perform a transmission step of transmitting to the information processing apparatus, and a model generating step of the information processing apparatus, to generate a model by machine learning images appearing are construction sites was learning data,
A transmission step of transmitting the model generated by the model generation step to the notification device, an image captured by the camera, and a reception step of receiving the prediction correctness information input by the input step of the notification device from the notification device; An image captured by the camera received in the receiving step, and a model updating step for updating the model by machine learning using the prediction correctness information as learning data, and the acquisition step of the notification device includes the updated model An information processing method acquired from the information processing apparatus may be used.

  Further, the present invention is formed by a notification device for notifying an operator working at a construction site of an accident prediction and an information processing device for generating a model that is referred to when an accident occurrence is predicted from an image showing the construction site. An information processing program to be executed by an information processing system to be displayed on an image captured by a camera attached to an operator based on the acquisition processing for acquiring a model from the information processing device on the notification device and the model When the prediction process that predicts the occurrence of an accident from the object, the notification process that notifies the worker of the prediction when the occurrence of the accident is predicted by the prediction process, and the prediction that is notified to the worker by the notification process, An input process in which an operator inputs prediction correctness, an image captured by the camera, and prediction correctness information input by the input process are sent to the information processing apparatus. The information generation device generates a model by machine learning using an image showing the construction site as learning data, and transmits the model generated by the model generation processing to the notification device. A receiving process for receiving from the notification device, an image taken by the camera, and an image taken by the camera received by the receiving process; The model update process for updating the model by machine learning using the prediction correctness information as learning data, and the acquisition process to be executed by the notification device is an information processing program for acquiring the updated model from the information processing device. There may be.

  In addition, the present invention is a model generated by machine learning using an image showing the construction site as learning data in the information processing apparatus, and is based on a model referred to when predicting the occurrence of an accident at the construction site When the occurrence of an accident is predicted from an object shown in an image taken by a camera attached to a worker working at a construction site, a receiving process for receiving prediction information and a machine for receiving prediction information It may be an information processing program that executes an input process for inputting a prediction correctness used as learning data for machine learning when a model is updated by learning.

  The above notification device, information processing device, information processing system, information processing method, and information processing program automatically predict the occurrence of an accident at a construction site and notify the worker who works at the construction site of the prediction result. Thus, it is possible to provide a technique capable of preventing an accident in advance.

FIG. 1 shows an example of the configuration of an information processing system according to this embodiment. FIG. 2 shows an example of an outline of the hardware configuration of the notification device. FIG. 3 shows an example of an outline of the hardware configuration of the terminal. FIG. 4 shows an example of an outline of the hardware configuration of the server. FIG. 5 shows an example of an outline of a functional configuration of the camera module of the notification device. FIG. 6 shows an example of an outline of the functional configuration of the terminal of the notification device. FIG. 7 shows an example of an outline of the functional configuration of the server. FIG. 8 shows an example of a flowchart illustrating the operation of the camera module that collects learning data used when generating an identification model. FIG. 9 shows an example of a flowchart for explaining the operation of the camera module for notifying the worker working at the construction site of the danger based on the identification model. FIG. 10 illustrates an example of the correspondence between the name of the object and the type of accident caused by the object. FIG. 11 shows an example of a flowchart for explaining the operation of the terminal when learning data used when the identification model is generated by the camera module is collected. FIG. 12 shows an example of a flowchart for explaining the operation of the terminal when danger notification is made from the camera module. FIG. 13 shows an example of a flowchart for explaining the operation of the server when an identification model is generated. FIG. 14 shows an example of a flowchart for explaining the operation of the server when the identification model is updated. FIG. 15 is a sequence diagram illustrating an example of processing of the information processing system when a learning phase for generating an identification model is executed. FIG. 16 is a sequence diagram illustrating an example of processing of the information processing system when an identification phase for identifying an object based on an identification model is executed. FIG. 17 shows an example of an image of the construction site taken by the camera module. FIG. 18 shows an example of an image captured by the camera module when the worker is working.

  Hereinafter, embodiments of the present invention will be described. Embodiment shown below is an example of embodiment of this invention, and does not limit the technical scope of this invention to the following aspects.

<System configuration example>
FIG. 1 shows an example of the configuration of an information processing system 100 according to the present embodiment. As shown in FIG. 1, the information processing system 100 includes a notification device 1 that notifies an operator working on a construction site of an accident prediction.

  In addition, the information processing system 100 includes a server 50 that generates a prediction model of an accident at a construction site. The server 50 is a computer that can communicate with the notification device 1 via the network N. The server 50 may be a cloud server. The network N may be any network that can exchange data by interconnecting computers. For example, the Internet, an intranet, and other various networks are applicable. In FIG. 1, the number of notification devices 1 is three, but the number of notification devices 1 may be any number.

<Hardware configuration example>
Next, an example of a hardware configuration of each device forming the information processing system 100 will be described. FIG. 2 shows an example of an outline of the hardware configuration of the notification device 1. As shown in FIG. 2, the notification device 1 includes a helmet 2 worn by an operator who works at a construction site. Two camera modules 3 are fixed to the front part and the rear part of the helmet 2, respectively. Here, the camera module 3 is an example of the “camera included in the notification device” of the present invention. The camera module 3 is fixed to the helmet 2 with, for example, an adhesive tape. In addition, the camera module 3 includes an IC (Integrated Circuit) chip 4 and an antenna 5 that performs communication with an external electronic device. The IC chip 4 includes a CPU (Central Processing Unit) 6 and a RAM (Random Access Memory) 7. A light 8 that illuminates the surroundings is fixed to the helmet 2. Further, the notification device 1 includes a speaker 9 and a terminal 20 that communicate with the camera module 3 wirelessly.

  FIG. 3 shows an example of an outline of the hardware configuration of the terminal 20. As illustrated in FIG. 3, the terminal 20 includes, for example, a CPU 21, a RAM 22, a ROM (Read Only Memory) 23, a wireless communication I / F (Interface) 24, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like. This can be realized by a computer having the large-capacity storage device 25, the display device 27, and the input device 28. Specific examples of the terminal 20 include devices such as smart phones and tablet terminals. The terminal 20 includes a touch panel display 29 as an example of the display device 27 and the input device 28. Here, the touch panel display 29 is an example of the “input unit” in the present invention.

  FIG. 4 shows an example of an outline of the hardware configuration of the server 50. As shown in FIG. 4, the server 50 is a computer including a CPU 51, a RAM 52, a ROM 53, a mass storage device 54 such as an HDD or an SSD, and a NIC (Network Interface Controller) 55 connected to the Internet via a gateway or the like. is there. In the mass storage device 54, information required to be persistent such as various programs loaded in the RAM 52 is written and read. The server 50 may include a dedicated processor, a dedicated circuit, and the like in addition to a general-purpose processor such as a CPU.

<Functional configuration example>
Next, an example of a functional configuration of each device forming the information processing system 100 will be described. FIG. 5 shows an example of an outline of a functional configuration of the camera module 3 of the notification device 1. As shown in FIG. 5, the camera module 3 includes a communication unit 10 that controls execution of transmission and reception of wireless signals. Here, the communication unit 10 is an example of “a transmission unit included in the notification device” and “a reception unit included in the notification device” of the present invention. In addition, the camera module 3 includes a photographing processing unit 11 that controls execution of photographing processing. In addition, the camera module 3 includes an identification unit 12 that manages execution of processing for identifying an object from image information captured by the imaging processing unit 11. In addition, the camera module 3 includes an accident prediction unit 13 that manages execution of a process for predicting an accident from the object identified by the identification unit 12. The accident prediction unit 13 outputs object information and information on the type of accident. Here, the identification unit 12 and the accident prediction unit 13 are examples of the “prediction unit” of the present invention. Further, the camera module 3 includes a notification unit 14 that controls execution of a process of transmitting a control signal for controlling the speaker 9 to the speaker 9 when an accident is predicted. The notification unit 14 also performs execution of processing for transmitting image information and information on the type of accident to the terminal 20 when an accident is predicted. Here, the notification unit 14 is an example of the “notification unit” in the present invention. In addition, the camera module 3 includes a storage unit 15 that manages execution of storage processing of information such as programs and data for executing each processing. The camera module 3 is operated by a CPU 6 that interprets and executes a computer program developed in the RAM 7 to operate the communication unit 10, the imaging processing unit 11, the identification unit 12, the accident prediction unit 13, the notification unit 14, and the storage unit 15. To realize.

  FIG. 6 shows an example of an outline of a functional configuration of the terminal 20 of the notification device 1. As illustrated in FIG. 6, the terminal 20 includes a communication unit 30 that controls transmission and reception of radio signals with an external electronic device. In addition, the terminal 20 includes a display unit 31 that controls execution of information to be displayed on the touch panel display 29. In addition, the terminal 20 includes an input unit 32 that controls execution of an input signal from the touch panel display 29. The terminal 20 implements a communication unit 30, a display unit 31, and an input unit 32 by operating a CPU 21 that interprets and executes a computer program loaded in the RAM 22.

FIG. 7 shows an example of an outline of the functional configuration of the server 50. As illustrated in FIG. 7, the server 50 includes a communication unit 56 that manages execution of Internet communication processing with an external electronic device. Here, the communication unit 56 is an example of “a transmission unit included in the information processing apparatus” and “a reception unit included in the information processing apparatus” of the present invention. The server 50 also includes a storage unit 57 that manages execution of processing for storing information such as image data and programs including algorithms used for machine learning. The algorithm used for machine learning is, for example, an algorithm such as an error back-propagation method or regression analysis used when optimizing the weight factors of each layer of the neural network model. Further, the server 50 performs execution of processing for generating an identification model for identifying an object shown in the image and outputting the identified object information by using the image data as input information by machine learning using the image data as learning data. An identification model generation unit 58 is provided. Here, the identification model generation unit 58 is an example of the “model generation unit” in the present invention.

  Here, the procedure for generating the identification model is as follows. First, feature amounts are extracted from image data of input information. Examples of the extracted feature quantity include HOG (Histgram Of Oriented Gradients).

  Next, an identification model is generated that receives the feature amount extracted from the image and outputs object information shown in the image. The output object information is, for example, the name of the object. Examples of the identification model include a neural network model and an identification function based on regression analysis. The identification model is optimized by learning learning data related to the feature amount extracted from the image and the name of the object shown in the image. Further, the identification model generation unit 58 generates an identification model for each object. For example, an identification model X for identifying that the object shown in the image is X and an identification model Y for identifying that the object shown in the image is Y are generated.

  In addition, the server 50 includes a correspondence generation unit 59 that manages execution of processing for generating a correspondence between the name of an object and the type of accident caused by the object. Here, the correspondence between the identification model and the name of the object and the type of accident caused by the object is an example of the “model referred to when predicting the occurrence of an accident at the construction site” of the present invention. is there. In addition, the server 50 updates the identification model for executing the process of updating the identification model by machine learning using the image data and the information about the correctness of the accident predicted from the image data using the identification model as learning data. The unit 60 is provided. Here, a machine learning algorithm stored in the storage unit 57 is used for machine learning in the identification model generation unit 58 and the identification model update unit 60. Still, the identification model update unit 60 is an example of the “model update unit” of the present invention.

  Here, the identification model update unit 60 extracts feature amounts from the image data of the input information, as in the case of generation of the identification model. The update of the identification model is realized by learning and optimizing learning data relating to the correctness of the accident predicted using the identification model from the feature amount extracted from the image and the image data.

  The server 50 also has a communication unit 56, a storage unit 57, an identification model generation unit 58, and the like by the CPU 51 that interprets and executes the computer program expanded in the RAM 52 cooperating with the mass storage device 54 and the NIC 55. A relationship generation unit 59 and an identification model update unit 60 are realized.

<Device operation example>
Next, an example of the operation of each device forming the information processing system 100 will be described. The camera module 3 collects learning data used when generating an identification model. FIG. 8 shows an example of a flowchart for explaining the operation of the camera module 3 that collects learning data used when generating an identification model.

(Step S11)
In step S <b> 11, the construction site is photographed by the camera module 3 fixed to the helmet 2. Since two camera modules 3 are fixed on the front and rear of the helmet, the surrounding environment in almost all directions is photographed. When photographing a dark environment such as at night or indoors, the light 8 is turned on. Information regarding the captured image is processed by the imaging processing unit 11. For example, when the image photographed by the camera module 3 is a moving image, the image is converted into a still image by the photographing processing unit 11.

(Step S12)
In step S <b> 12, information related to the captured image is transmitted to the terminal 20 by the communication unit 10.

(Step S13)
In step S <b> 13, the communication unit 10 receives information about the image and the name of the object shown in the image from the terminal 20. The received information is stored in the RAM 7 in the storage unit 15.

(Step S14)
In step S14, it is determined whether or not a predetermined amount of information related to the image and the name of the object shown in the image is stored in the RAM 7. If the RAM 7 does not store a predetermined amount of information about the image and the name of the object shown in the image, the processing from step S11 to step S13 is executed.

(Step S15)
In step S14, when it is determined that a predetermined amount of information about the image and the name of the object shown in the image is stored in the RAM 7, the communication unit 10 transmits information about the image and the name of the object shown in the image to the server 50. The

  Moreover, the camera module 3 notifies the danger to the worker who works on the construction site based on the identification model. FIG. 9 shows an example of a flowchart for explaining the operation of the camera module 3 for notifying a worker working on the construction site of danger based on the identification model.

(Step S101)
In step S <b> 101, the communication unit 10 receives information regarding the correspondence between the identification model and the name of the object and the type of accident caused by the object from the server 50. Information regarding the received identification model and the correspondence relationship is stored in the storage unit 15. Here, FIG. 10 illustrates an example of the correspondence between the name of the object and the type of accident caused by the object. The correspondence relationship shown in FIG. 10 is created based on the totaled data obtained by totaling accidents that occurred in the past at the construction site in the server 50.

(Step S102)
In step S <b> 102, the environment around the worker is photographed by the camera module 3 when the worker is working on the construction site.

(Step S103)
In step S103, the identification unit 12 identifies the object shown in the image from the image information captured in step S102, and outputs the identified object information. The identification of the object is realized by inputting image information to the identification model received from the server 50 and stored in the storage unit 15. When there are a plurality of identification models, the image information is input to each identification model, and an output of object information is obtained.

(Step S104)
In step S104, the accident prediction unit 13 outputs the type of accident predicted based on the object information shown in the image obtained by the identification unit 12. Here, the object information used in the case of an accident prediction is, for example, the name of an object. The predicted type of accident is realized by referring to the correspondence between the name of the object shown in FIG. 10 and the type of accident caused by the object. When the name of the object shown in the image obtained by the identification unit 12 exists in the correspondence relationship, the accident type corresponding to the name of the object described in the correspondence relationship is an output of the predicted accident type. Here, the output of the accident type may be performed when a plurality of objects corresponding to the predetermined accident type described in the correspondence relationship are all identified from the image. On the other hand, when the name of the object shown in the image obtained in the identification unit 12 does not exist in the correspondence relationship, the type of accident is not output.

(Step S105)
In step S105, the accident prediction unit 13 determines whether or not the predicted accident type has been output.

(Step S106)
In step S <b> 106, when the predicted type of accident is output, the notification unit 14 generates a control signal for the speaker 9 and transmits the control signal to the speaker 9 via the communication unit 10. Then, when the speaker 9 receives the control signal and emits a sound, the operator is notified of the danger.

  Further, in step S106, when the predicted accident type is output, the notification unit 14 also transmits information about the predicted accident to the terminal 20 via the communication unit 10, which may be dangerous for the worker. Is notified. The information on the predicted accident is, for example, image information when the predicted accident type is output, information on the name of the object shown in the image, information on the predicted accident type, and the like.

  However, in the storage unit 15, for example, the notification time and information on the notified accident type are stored in the RAM 7. When information on the predicted accident type is newly output, the information on the accident type stored in the RAM 7 is referred to, and the information matches the newly output information on the predicted accident type. In this case, the newly output information on the predicted accident type is not notified. That is, the notification of the danger already performed is not performed again.

  In the storage unit 15, when the notification time stored in the RAM 7 is a time at which a predetermined time has elapsed, the information regarding the type of accident notified at the notification time may be deleted from the RAM 7. With such a notification device 1, when an already notified accident is predicted again after a predetermined time has passed, the operator is again notified of the danger.

  When the predicted accident type is not output in step S105, the danger is not notified and the process returns to the execution of the construction site shooting (S102) by the camera module 3.

(Step S107)
In step S <b> 107, the communication unit 10 receives information about the correctness of the predicted accident notified from the terminal 20. The information on the correctness of the predicted accident is, for example, the image notified to the terminal 20, correctness information on the name of the object shown in the image, or information on the correct name of the object shown in the image.

(Step S108)
In step S <b> 108, information related to the correctness of the predicted accident received from the terminal 20 is stored in the storage unit 15.

(Step S109)
In step S <b> 109, it is determined whether or not the storage unit 15 stores a predetermined amount of information related to the correctness of the predicted accident received from the terminal 20.

(Step S110)
In step S <b> 110, when it is determined in the storage unit 15 that a predetermined amount of information regarding the correctness of the predicted accident received from the terminal 20 has been stored, the information regarding the correctness of the predicted accident is stored in the server 50 via the communication unit 10. Sent to.

  Next, an example of the operation of the terminal 20 will be described. FIG. 11 shows an example of a flowchart for explaining the operation of the terminal 20 when learning data used when the identification model is generated by the camera module 3 is collected.

(Step S21)
In step S <b> 21, the communication unit 30 receives information related to the image of the construction site taken by the camera module 3 from the communication unit 10 of the notification device 1.

(Step S22)
In step S <b> 22, the received image information is displayed on the touch panel display 29 in the display unit 31.

(Step S23)
In step S <b> 23, information related to the name of the object shown in the image displayed on the touch panel display 29 is input by tapping the touch panel display 29. Then, the input signal from the touch panel display 29 is processed in the input unit 32.

(Step S24)
In step S <b> 24, the image displayed on the touch panel display 29 and the information related to the name of the object input in the input unit 32 are transmitted to the communication unit 10 of the camera module 3 in the communication unit 30.

  FIG. 12 shows an example of a flowchart for explaining the operation of the terminal 20 when danger notification is made from the camera module 3.

(Step S201)
In step S201, the communication unit 30 receives information on the predicted accident from the communication unit 10 of the notification device 1, and notifies the danger. The information on the predicted accident is, for example, image information when the predicted accident type is output, information on the name of the object shown in the image, information on the predicted accident type, and the like.

(Step S202)
In step S <b> 202, the received image information, information about the name of the object shown in the image, and information about the predicted accident type are displayed on the touch panel display 29 on the display unit 31.

(Step S203)
In step S203, correct / incorrect information on the name of the object shown in the image displayed on the touch panel display 29 and information on the correct name of the object shown in the image are input by tapping the touch panel display 29. Then, the input signal from the touch panel display 29 is processed in the input unit 32.

(Step S204)
In step S <b> 204, information regarding the correctness of the predicted accident input through the input unit 32 is transmitted to the communication unit 10 of the camera module 3 through the communication unit 30.

  Next, an example of the operation of the server 50 will be described. FIG. 13 shows an example of a flowchart for explaining the operation of the server 50 when an identification model is generated.

(Step S31)
In step S <b> 31, the communication unit 56 receives information on the construction site image and the name of the object shown in the image from the communication unit 10 of the camera module 3. The received information is stored in the mass storage device 54 in the storage unit 57.

(Step S32)
In step S <b> 32, machine learning is performed in the identification model generation unit 58 using learning site information stored in the large-capacity storage device 54 and information on the names of objects shown in the image as learning data. Then, using the image data of the construction site as input information, an identification model for identifying an object reflected in the image and outputting the identified object information is generated.

(Step S33)
In step S33, the correspondence generation unit 59 generates a correspondence between the name of the object as shown in FIG. 10 and the type of accident caused by the object.

(Step S34)
In step S <b> 34, the communication unit 56 compresses the generated information related to the identification model and the correspondence relationship, and transmits the compressed information to the communication unit 10 of the camera module 3.

  FIG. 14 shows an example of a flowchart explaining the operation of the server 50 when the identification model is updated.

(Step S301)
In step S <b> 301, the communication unit 56 receives information regarding the correctness of the predicted accident from the communication unit 10 of the camera module 3. The information on the correctness of the predicted accident is, for example, the correctness information of the information (for example, the name of the object) related to the object identified by the identification model from the image captured by the camera module 3, and the correct name of the object. Information. The received information regarding the correctness of the predicted accident is stored in the mass storage device 54.

(Step S302)
In step S <b> 302, the identification model update unit 60 updates the identification model by executing machine learning using information related to whether the predicted accident is correct or not as learning data.

(Step S303)
In step S303, the communication unit 56 transmits the updated identification model to the communication unit 10 of the notification device 1.

<System operation example>
Next, an example of processing of the information processing system 100 will be described. FIG. 15 is a sequence diagram illustrating an example of processing of the information processing system 100 when a learning phase for generating an identification model is executed.

(Pattern 1P01)
The process of step S11 is executed, and the construction site is photographed by the camera module 3. FIG. 17 shows an example of an image of a construction site taken by one camera module 3. The container 16 and the wire 17 are reflected in the image.

(Pattern 1P02)
The process of step S <b> 12 is executed, and an image of the construction site taken by the camera module 3 is transmitted to the terminal 20.

(Pattern 1P03)
In the terminal 20, the processing from step S21 to step S23 is executed. By executing the processing from step S <b> 21 to step S <b> 23, the name of the object (container, wire) reflected in the image of the construction site photographed by the camera module 3 is input via the touch panel display 29 of the terminal 20.

(Pattern 1P04)
The process of step S24 is executed, and information on the construction site image and the name of the object shown in the construction site image is transmitted from the terminal 20 to the camera module 3.

(Pattern 1P05)
The process of step S <b> 13 is executed, and the camera module 3 receives information about the construction site image and the name of the object shown in the construction site image from the terminal 20. Further, the process of step S14 is executed, and it is determined whether or not a predetermined amount of information on the construction site image and the name of the object shown in the construction site image is stored in the RAM 7. When it is determined that a predetermined amount of information regarding the construction site image and the name of the object shown in the construction site image is not stored in the RAM 7, the processes from the pattern 1P01 to the pattern 1P04 are repeated.

  On the other hand, when it is determined that a predetermined amount of information on the construction site image and the name of the object shown in the construction site image is stored in the RAM 7, the process of step S15 is executed, and the construction site image Information regarding the name of the object shown in the image of the construction site is transmitted to the server 50. Here, the patterns 1P01 to 1P05 are executed in a plurality of notification devices 1 connected to the server 50 via the network N. That is, the server 50 collects information on various construction site images and names of objects shown in the construction site images.

(Pattern 1P06)
Processing from step S31 to step S33 is executed. For example, in the server 50, machine learning is executed using learning site information (for example, FIG. 17) received from the camera module 3 and information on the names of objects (containers, wires) reflected in the construction site image. An identification model is generated. As the identification model, an identification model for each object is generated, such as an identification model for identifying a container from an image at a construction site and an identification model for identifying a wire from an image at a construction site. In addition, the identification model is not limited to a container and a wire, and an identification model for each object that can exist at a construction site such as a knife, a stepladder, a pipe, and a floor step is generated. Also, a correspondence relationship between a plurality of objects as shown in FIG. 10 and the types of accidents caused by the plurality of objects is generated.

(Pattern 1P07)
The process of step S34 is executed, and the identification model generated in the server 50 and the information on the correspondence between the plurality of objects and the type of accident caused by the plurality of objects are compressed and transmitted to the camera module 3. The

  Next, an example of processing of the information processing system 100 when an identification phase for identifying an object based on an identification model is executed will be described. FIG. 16 is a sequence diagram illustrating an example of processing of the information processing system 100 when an identification phase for identifying an object based on an identification model is executed. Further, the information processing system 100 predicts an accident based on the identified object information and notifies the danger, and updates the identification model based on the prediction correctness information.

(Pattern 2P01)
The processing from step S101 to step S104 is executed. The camera module 3 images the construction site, predicts the accident based on the captured image, the identification model received from the server 50, and the correspondence between the name of the object and the type of accident caused by the object. Do.

  FIG. 18 shows an example of an image taken by the camera module 3 when the worker is working. The container 18 and the wire 19 are reflected in the image shown in FIG. Here, in the present embodiment, it is assumed that the container 18 is identified from the image and the wire 19 is not identified based on the identification model. Therefore, based on the correspondence shown in FIG. 10, it is assumed that only “sandwiched” as an accident corresponding to the container 18 is predicted, and “falling” as an accident corresponding to the wire 19 is not predicted.

(Pattern 2P02)
The process of step S105 is executed, and it is determined whether or not an accident type has been output from the identification model. If it is determined that an accident type output has been performed, the process of step S106 is executed, information on the predicted accident is transmitted to the terminal 20, and a danger is notified. The information related to the predicted accident includes, for example, the image shown in FIG. 18, name information (container) of the object shown in the image, and information related to the predicted type of accident (sandwiched). Further, a sound is emitted from the speaker 9 to notify the operator of the danger.

(Pattern 2P03)
The processing from step S201 to step S203 is executed. Information on the predicted accident is displayed on the touch panel display 29 of the terminal 20.

  In addition, information regarding the correctness of the predicted accident displayed on the touch panel display 29 is input by tapping the touch panel display 29. The information on the correctness of the predicted accident is, for example, the image notified to the terminal 20, correctness information on the name of the object shown in the image, or information on the correct name of the object shown in the image. Here, the operator recognizes that the wire 19 has not been identified. Then, the operator inputs “prediction reject” and “wire” from the touch panel display 29.

(Pattern 2P04)
The process of step S <b> 204 is executed, and information regarding the correctness of the predicted accident input from the touch panel display 29 is transmitted to the camera module 3.

(Pattern 2P05)
The processing from step S107 to step S108 is executed, information related to the correctness of the predicted accident is received from the terminal 20, and the received information related to the correctness of the predicted accident is stored in the RAM 7.

(Pattern 2P06)
The process of step S109 is executed, and it is determined whether or not a predetermined amount of information related to the correctness of the predicted accident is stored in the RAM 7. If it is determined that a predetermined amount of information regarding the correctness of the predicted accident is not stored in the RAM 7, the process of pattern 2P06 is repeatedly executed from the process of S102 of pattern 2P01.

(Pattern 2P07)
When it is determined that a predetermined amount of information regarding the correctness of the predicted accident is stored in the RAM 7, the process of step S <b> 110 is executed, and information regarding the correctness of the predicted accident is transmitted to the server 50.

(Pattern 2P08)
The processing from step S301 to step S302 is executed, and in the server 50, machine learning is performed using the received information on whether the predicted accident is correct or not as learning data, and the identification model is updated. In this embodiment, since the wire is not identified from the image shown in FIG. 18, for example, the server 50 executes machine learning using the image shown in FIG. 18 and the data of the object name “wire” as learning data, The identification model is updated.

(Pattern 2P09)
The process of step S <b> 303 is executed, and the updated identification model is transmitted from the server 50 to the camera module 3. Then, the process from the pattern 2P01 is repeatedly executed.

<Action and effect>
In the case of the information processing system as described above, the server 50 generates an identification model for identifying an object from an image showing the construction site by machine learning using an image showing the construction site as learning data. Also, as shown in FIG. 10, a correspondence between the name of the object and the type of accident caused by the object is generated. Therefore, an accident can be predicted from an image photographed by a camera attached to the worker. In addition, when the accident is predicted, the operator can be prevented from accidents by notifying the operator of the danger via the speaker 9 or the terminal 20.

  In addition, the worker can input information on whether the prediction is correct by looking at information on the predicted accident displayed on the touch panel display 29 of the terminal 20. Then, information regarding whether the inputted predicted accident is correct or not is transmitted to the server 50. In the server 50, the identification model is updated by machine learning using the information on whether the predicted accident is correct or not as learning data. In the camera module 3, the accident is predicted again based on the updated identification model. In the case of the above embodiment, the wire 19 cannot be identified from the image shown in FIG. 18 with the initially generated identification model. However, the identification model is updated in the server 50 based on the information that the prediction input by the worker is negative and the information that the object is a wire. That is, the accuracy of accident prediction is improved.

  Further, the correspondence relationship between the name of the object shown in FIG. 10 and the type of accident caused by the object is created based on data obtained by totaling accidents that occurred in the past at the construction site. That is, the information processing system 100 can use data on accidents that have occurred in the past as know-how.

The camera module 3 includes a flash memory, a USB memory, and an SD (Secure
An auxiliary storage device 33 such as a digital memory card may be provided. And the memory | storage of the information regarding the name of the object reflected in the image of the construction site in the pattern 1P05 and the said construction site may be memorize | stored in the auxiliary storage device 33 instead of RAM7. In addition, storage of information regarding the correctness of the predicted accident in the pattern 2P05 may be stored in the auxiliary storage device 33 instead of the RAM 7. Then, the transmission of information from the camera module 3 to the server 50 realized in the pattern 1P05 or the pattern 2P07 may be realized by removing the auxiliary storage device 33 from the camera module 3 and inserting it into the server 50.

  In the present embodiment, the learning data for generating the identification model was captured by the camera module 3, but not only the camera module 3, but any electronic device as long as it is an image showing a construction site It may be an image taken in The learning data may be a construction site diagram generated by, for example, CAD (Computer Aided Design).

  Further, in the identification model generation unit 58, for example, as shown in the correspondence relationship in FIG. 10, an identification model for each object is generated such as an identification model A for identifying a pipe and an identification model B for identifying a thread turning lathe. However, the sharp shape is identified from the image, the identification model C is identified as an object such as a sander, a cutter knife, or the like, or the ground is identified from the image, and the object reflected in the vicinity of the ground is a carriage, piping material, duct material, An identification model D that identifies an object such as an aerial work vehicle or a container may be generated.

  Further, when the identification model generation unit 58 or the identification model update unit 60 of the server 50 generates or updates an identification model for each object, and the types of accidents corresponding to the identified objects are similar. Common learning data may be used. For example, an identification model for identifying a floor opening that is an object corresponding to a fall that is one of the types of accidents is generated and updated by learning an image in which the floor opening is reflected. Here, since a fall that is one of the types of accidents is similar to a fall, an identification model for identifying a floor gap that is an object corresponding to a fall is a case of generating or updating a floor opening identification model It may be generated / updated by learning the image data used for.

  Further, the notification device 1 may include a glasses-type display that can perform wireless communication with an external electronic device, for example. With such a notification device 1, for example, even when a worker uses a thunder, it is difficult to hear the sound emitted from the speaker 9, and the worker may not be able to recognize the danger notification through hearing. By displaying information on the predicted type of accident on the glasses-type display or by causing a predetermined part of the glasses-type display to emit light, it is possible to make the operator recognize the danger of the accident through vision. Further, the notification device 1 may change the danger notification method for each identification model. For example, when an accident is predicted based on an identification model that identifies a sander from an image showing a construction site, the notification device 1 displays information “cut and rub” on the glasses-type display, while the construction site If an accident is predicted based on an identification model that identifies a floor step from an image in which a noise is reflected, the danger may be notified by emitting a sound from the speaker 9.

Further, notification priority information may be included in the identification model and the type of accident in the correspondence relationship. Then, for example, in the accident prediction unit 13 in step S105, the determination as to whether or not the predicted accident type has been output may be determined in order from the output based on the identification model having a high notification priority. Also, for example, when a plurality of predicted accident types are output based on a plurality of identification models and correspondences from an image showing the construction site, the operator is notified in order from the type of accident with the highest notification priority. Also good. Further, the loudness emitted from the speaker 9, the display content of the terminal 20, and the like may be changed according to the priority. Also, the priority is
For example, it may be set based on data such as the total number of days of injury caused by accidents that occurred in the past.

  As mentioned above, although an example of preferable embodiment of this invention was demonstrated, this invention is not limited to the form of illustration. It is obvious for those skilled in the art that various modifications or modifications can be conceived within the scope of the idea described in the claims, and these naturally belong to the technical scope of the present invention. It is understood.

1. Notification device; 2. Helmet; 3. Camera module; 4. IC chip; 5. Antenna; 6. CPU: 7. RAM: 8. Light: 9. Speaker: 10.・ Communication section; 11. ・ Photographing processing section; 12 ・ ・ Identification section; 13 ・ ・ Accident prediction section; 14 ・ ・ Notification section; 15 ・ ・ Storage section; 16 ・ ・ Container; 17 ・ ・ Wire; 18 ・ ・ Container 19 .Wire; 20 ..Terminal; 21 ..CPU; 22 ..RAM; 23 ..ROM, 24 ..Wireless communication I / F; 25 .. Mass storage device; ..Input device; 29..Touch panel display; 30..Communication part; 31..Display part; 32..Input part; 33..Auxiliary storage device; 50..Server; 51..CPU; 52. 53. ROM; 54 Mass storage device; 5 .. NIC; 56. Communication unit; 57. Storage unit; 58 ... Discrimination model generation unit; 59 ... Correspondence relationship generation unit; 60 ... Discrimination model update unit; 100 ... Information processing system; ·network

Claims (8)

A camera attached to a worker who works at the construction site;
Based on a model that is referenced when predicting the occurrence of an accident at a construction site from an image showing the construction site, a predicting means for predicting the occurrence of an accident from an object reflected in the image taken by the camera;
A notification means for notifying the operator of the prediction when an occurrence of an accident is predicted by the prediction means;
An image photographed by the camera and the prediction correctness information input by the input means for the operator to input the correctness of the prediction when the prediction is notified to the worker by the notification means. The model is generated by machine learning using an image showing the construction site as learning data, and transmitted to the server that updates the model by machine learning using the image captured by the camera and the prediction correctness information as learning data. Sending means to
Obtaining means for obtaining the model updated in the server from the server,
Notification device. Model generation means for generating a model to be referred to when predicting the occurrence of an accident at the construction site by machine learning using an image showing the construction site as learning data,
Based on the model generated by the model generation means, from an object reflected in an image taken by a camera attached to a worker working at a construction site, predicting the occurrence of an accident and notifying the worker, Transmitting means for transmitting the model to an information device that inputs the correctness information of the prediction notified by the worker;
Receiving means for receiving, from the information device, the image taken by the camera and the prediction correctness information input by the worker;
Model update means for updating the model by machine learning using the image received by the camera received by the receiving means and the prediction correctness information as learning data,
Information processing device. An information processing system formed by a notification device that notifies workers working on the construction site of the prediction of the accident and an information processing device that generates a model that is referenced when predicting the occurrence of the accident from the image of the construction site There,
The notification device includes:
A camera mounted on the operator;
Obtaining means for obtaining the model from the information processing apparatus;
Prediction means for predicting the occurrence of an accident from an object shown in an image photographed by the camera based on the model;
A notification means for notifying the operator of the prediction when an occurrence of an accident is predicted by the prediction means;
When the notification is notified to the worker by the notification means, the input means by which the worker inputs the correctness of the prediction;
Transmission means for transmitting the image captured by the camera and the prediction correctness information input by the input means to the information processing apparatus,
The information processing apparatus includes:
A model generating means for generating the model by machine learning using an image showing a construction site as learning data;
Transmitting means for transmitting the model generated by the model generating means to the notification device;
Receiving means for receiving, from the notification device, an image photographed by the camera of the notification device and the prediction correctness information input by the input means of the notification device;
An image taken by the camera received by the receiving means, and a model updating means for updating the model by machine learning using the prediction correctness information as learning data,
The acquisition unit of the notification device acquires the updated model from the information processing device;
Information processing system. The model is
An identification model for identifying an identification object from an image showing the construction site;
The correspondence between the identification object and the type of accident caused by the identification object,
The information processing system according to claim 3. The correspondence relationship is a correspondence relationship between a plurality of identification objects and types of accidents caused by the plurality of identification objects.
The information processing system according to claim 4. An information processing system formed by a notification device that notifies a worker working at a construction site of an accident prediction and an information processing device that generates a model that is referenced when predicting the occurrence of an accident from an image of the construction site. An information processing method to be executed,
The notification device is
An acquisition step of acquiring the model from the information processing apparatus;
Based on the model, a prediction step of predicting the occurrence of an accident from an object shown in an image taken by a camera attached to the worker;
A notification step of notifying the operator of the prediction when the occurrence of an accident is predicted by the prediction step;
An input step in which the operator inputs whether the prediction is correct or not when the prediction is notified to the worker in the notification step;
Transmitting the image photographed by the camera and the prediction correctness information input in the input step to the information processing apparatus, and
The information processing apparatus is
A model generation step for generating the model by machine learning using an image showing a construction site as learning data;
A transmission step of transmitting the model generated by the model generation step to the notification device;
A receiving step of receiving, from the notification device, the image captured by the camera and the prediction correctness information input by the input step of the notification device;
A model update step of updating the model by machine learning using the image received by the camera received by the reception step and the prediction correctness information as learning data; and
The obtaining step of the notification device obtains the updated model from the information processing device;
Information processing method. An information processing system formed by an information processing system that generates a model that is referenced when predicting the occurrence of an accident from an image that shows the construction site, and a notification device that notifies the worker working at the construction site of the prediction of the accident An information processing program to be executed,
In the notification device,
An acquisition process for acquiring the model from the information processing apparatus;
Based on the model, a prediction process for predicting the occurrence of an accident from an object shown in an image taken by a camera attached to the worker;
A notification process for notifying the operator of the prediction when an accident is predicted by the prediction process;
When the prediction is notified to the worker by the notification processing, the worker inputs the prediction whether the prediction is correct;
A transmission process for transmitting the image captured by the camera and the prediction correctness information input by the input process to the information processing apparatus;
In the information processing apparatus,
Model generation processing for generating the model by machine learning using an image showing a construction site as learning data;
A transmission process for transmitting the model generated by the model generation process to the notification device;
A reception process for receiving, from the notification apparatus, the image captured by the camera and the prediction correctness information input by the input process of the notification apparatus;
An image captured by the camera received by the reception process, and a model update process for updating the model by machine learning using the prediction accuracy information as learning data,
The acquisition process to be executed by the notification device acquires the updated model from the information processing device.
Information processing program. In the information processing device,
It is a model generated by machine learning using learning images as images of the construction site, and is attached to workers working at the construction site based on a model that is referenced when predicting the occurrence of an accident at the construction site. A reception process for receiving the prediction information when an accident is predicted from an object reflected in an image captured by a camera.
When the prediction information is received, an input process for inputting whether the prediction is correct or not, which is used as learning data for the machine learning when the model is updated by machine learning, is executed.
Information processing program.

Images