JP6977607B2 - Safety judgment device, safety judgment system, safety judgment method - Google Patents

Description

The present invention relates to a safety determination device, a safety determination system, and a safety determination method.

For example, there is known a work safety plan management system that confirms whether or not a worker is working according to a safety procedure. (For example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2008-146502

In Patent Document 1, an ID-TAG that identifies each of the workplace, the machine in the workplace, the safety equipment, or the worker is attached, and the worker's work is performed according to the safety procedure based on the recognition code of the ID-TAG. A system that automatically confirms whether or not it has been damaged is disclosed. However, in this system, since it is necessary to attach an ID-TAG to a worker or a work object, it takes time and cost to attach the ID-TAG, and further, the work object to which the ID-TAG should be attached is required. If the ID-TAG is forgotten to be attached to the system, there is a risk that the safety of the work cannot be confirmed.

The main invention for solving the above-mentioned problems is an image input unit for inputting an image taken by a camera taken toward the line of sight of a worker, and an object recognition for recognizing a work object in the image area of the image. The unit, the hand recognition unit that recognizes the worker's hand in the image area of the image, and the action determination of determining whether or not the work object and the hand overlap in the image area of the image. When the unit and the action determination unit determine that the work object and the worker's hand overlap, the work safety is not ensured by the worker in relation to the work object. It is provided with a condition determination unit for determining whether or not a predetermined condition is satisfied.
Other features of the invention will become apparent with reference to the accompanying drawings and the description herein.

According to the present invention, it is possible to easily and quickly grasp whether or not the safety of the worker is ensured by using the image processing technique, so that the work safety of the worker can be improved.

It is a figure which shows an example of the structure of a safety judgment system. It is a figure which shows an example of the structure of a safety determination device. It is a figure which shows an example of the hardware composition of a safety determination device. It is a figure which shows an example of the software structure of a safety determination device. It is a figure which shows an example of the hardware configuration of a monitoring device. It is a figure which shows an example of the software composition of a monitoring apparatus. It is a figure which shows an example of the processing flow of a safety determination apparatus. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows a specific example of the processing of a safety determination device. It is a figure which shows an example of the processing flow of a safety judgment system. It is a figure which shows an example of the structure of a safety judgment system. It is a figure which shows a specific example of the processing of a safety judgment system. It is a figure which shows a specific example of the processing of a safety judgment system.

The description of this specification and the accompanying drawings will clarify at least the following matters. In the following description, the parts with the same reference numerals represent the same elements, and their basic configurations and operations are the same.

=== Configuration of safety judgment system 1 ===
The safety determination system 1 will be described with reference to FIGS. 1 to 17. FIG. 1 is a diagram showing an example of the configuration of the safety determination system 1. The safety determination system 1 is a system that determines whether or not the work safety of workers is ensured by using image processing technology. As shown in FIG. 1, the safety determination system 1 includes, for example, a camera 10 worn by an operator, a sensor 20, a safety determination device 30, a surveillance camera 40 worn by an observer, and a surveillance device 50. It is configured to include. Note that FIG. 1 shows that the safety determination device 30 is configured separately from the camera 10, but the safety determination device 30 is not limited to this, and the safety determination device 30 is built in the camera 10 and is integrally configured. You may. Further, although it is shown that the surveillance device 50 is configured separately from the surveillance camera 40, the present invention is not limited to this, and the surveillance device 50 may be built in the surveillance camera 40 and integrally configured. Further, in FIG. 1, the flow of data is indicated by an arrow.

<< Camera 10 >>
The camera 10 is, for example, a device for photographing a work object in the line-of-sight direction of a worker. The camera 10 is composed of, for example, an image pickup element such as a single-plate or multi-plate CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Sensor), and a plurality of components including a lens having a wide angle of view (wide-angle lens). It is desirable to have an infrared mode. However, the specifications are not limited. Further, the camera 10 has, for example, a function of wireless communication or wire communication with the first communication unit of the safety determination device 30, and can transmit image information to the safety determination device 30 by wireless communication or wire communication.

<< Sensor 20 >>
The sensor 20 is, for example, a device worn by a worker to measure the physical condition of the worker. The sensor 20 measures the state of the worker's body, such as a thermometer, a sphygmomanometer, and a heart rate monitor. The sensor 20 can output a measurement signal indicating the measurement result to the safety determination device 30.

<< Safety Judgment Device 30 >>
The safety determination device 30 is a device that determines whether or not the safety of the work is ensured when the worker works on the work object by using the image processing technique. As shown in FIG. 2, the safety determination device 30 carried by the worker receives the image information taken by the camera 10 from the camera 10 and determines whether or not the safety of the worker is ensured based on the image information. judge. In FIG. 2, the broken line arrow indicates the shooting direction, and the solid line arrow indicates the flow of information. Such a safety determination device 30 includes the hardware configuration and the software configuration shown below.

FIG. 3 is a diagram showing an example of the hardware configuration of the safety determination device 30. As shown in FIG. 3, the hardware of the safety determination device 30 includes a processor 31, a memory 32, a storage device 33, an input device 34, an output device 35, and a first communication device 36. Has been done. The processor 31 is, for example, an MPU, a CPU, or the like. The memory 32 is, for example, a RAM, a ROM, an NVRAM, or the like. The storage device 33 is, for example, a RAM, a ROM, an NVRAM, or the like. The input device 34 is a user interface that receives operation input from the user, and is, for example, an operation input device (keyboard, mouse, touch panel, etc.), a voice input device (microphone, etc.), and the like. The output device 35 is a user interface that provides various information to the user, and is, for example, a display device (liquid crystal monitor or the like), an audio output device (speaker or the like), or the like. The output device 35 displays, for example, that it is in a dangerous state on a display screen (not shown) based on a danger signal, or outputs a voice indicating that it is in a dangerous state by a speaker. The first communication device 36 is an interface for connecting to a network, and is, for example, a wireless LAN adapter, a NIC (Network Interface Card), or the like.

FIG. 4 is a diagram showing an example of the software configuration of the safety determination device 30. As shown in FIG. 4, the software of the safety determination device 30 includes an image input unit 30a, an object recognition unit 30b, a hand recognition unit 30c, an action determination unit 30d, a condition determination unit 30e, and a signal reception unit 30f. Has the function of. These functions are realized, for example, by the processor 31 of the safety determination device 30 reading and executing the program stored in the memory 32. It should be noted that these functions may be realized by hardware such as ASIC. Further, the processor 31 may be realized by reading and executing a program stored in an external storage medium. Note that FIG. 3 shows that the safety determination device 30 realizes each function with one information processing device (computer), but the present invention is not limited to this. For example, each of the above-mentioned functions may be configured to be distributed and realized by two or more information processing devices. Each function of the software configuration of the safety determination device 30 will be described as follows.

The image input unit 30a is a function of inputting a captured image captured by the camera 10. The image input unit 30a transmits the input captured image to the object recognition unit 30b and the hand recognition unit 30c. Note that FIG. 1 shows that the captured image is directly input from the camera 10 to the image input unit 30a, but the present invention is not limited to this. For example, the safety determination device 30 may be configured to transmit from the camera 10 to a server device (not shown) installed outside and input the captured image from the server device to the image input unit 30a.

The object recognition unit 30b is a function of recognizing a work object reflected in a captured image. The object recognition unit 30b performs preprocessing such as image correction, image quality improvement, and image coordination to remove noise contained in the input captured image and extract features of the work object to work. Recognize things. The object recognition unit 30b has a function of transmitting work object information indicating a work object recognized on the image to the action determination unit 30d.

As an example of the function of the object recognition unit 30b for recognizing the work object described above with reference to FIG. 8, when the work object is the handle 110 of the cubicle 100, for example, the central portion surrounded by the broken line in the image. When the handle 110 appears on the screen, the object recognition unit 30b automatically extracts, for example, the contour of the handle 110 (for example, the thick line in FIG. 8), and collates the extracted contour with the object contour information stored in advance. Recognize the handle 110 of the work object. The above is an example, and as a method for recognizing an image of a work object, for example, a method of learning an image of the work object using a neural network and recognizing the work object may be used. The method is not limited. Other examples will be described in the operation procedure of the safety determination device 30 described later.

The hand recognition unit 30c is a function of recognizing a worker's hand reflected in a captured image. The hand recognition unit 30c, for example, extracts a region showing a predetermined color as preprocessing, performs noise removal, and extracts only the region of the worker's hand. At this time, the hand recognition unit 30c also recognizes the color of the hand. The hand recognition unit 30c tracks the contour of the region of the hand extracted in this way to identify the contour of the hand (for example, the thick line in FIG. 8). The hand recognition unit 30c, for example, calculates the circularity of the contour of the hand and recognizes that it is a worker's hand. The hand recognition unit 30c has a function of transmitting hand information indicating a worker's hand recognized on the image to the action determination unit 30d.

As an example of the function of the hand recognition unit 30c for recognizing the worker's hand described above, the hand recognition unit 30c will be described when the hand is reflected in the central portion surrounded by the broken line of the image. For example, by extracting the skin-colored area of the hand and extracting only the hand area, the contour of the worker's hand is specified. When the worker wears insulating gloves, for example, an orange region is extracted to specify the contour of the worker's hand. By collating the specified hand contour with the hand contour information stored in advance, the worker's hand is recognized. The above is only an example, and as a method of recognizing a hand image, for example, a method of learning a hand image using a neural network and recognizing a hand may be used, and the method is limited. is not it. Other examples will be described in the operation procedure of the safety determination device 30 described later.

The action determination unit 30d is a function of determining that the work object shown in the captured image and the worker's hand overlap. The action determination unit 30d recognizes that the worker's hand overlaps the image-recognized work object on the image based on the work object information and the hand information. Specifically, the action determination unit 30d determines, for example, the brightness value of the handle 110 and a predetermined threshold value that change when the silver region of the handle 110 and the skin color region of the hand or the color region of the insulating glove overlap each other. By comparing, the overlap between the handle 110 and the hand is detected. The above is just an example, and as for the method of image recognition of the overlap between the work object and the worker's hand, for example, a neural network is used to learn an image in which the work object and the worker's hand overlap. A method of recognizing overlap may be used, and the method is not limited. The action determination unit 30d has a function of transmitting a first determination signal indicating the result of detecting the overlap between the work object and the hand to the condition determination unit 30e. This function creates an opportunity to determine whether or not work is safe by determining the overlap between the work object and the worker's hands that occurs when the worker is actually trying to work. can.

The condition determination unit 30e is a function of determining whether or not the worker satisfies a predetermined condition in relation to the work object when the first determination signal or the second determination signal is received. When the condition determination unit 30e determines that the predetermined conditions are satisfied when receiving both or one of the first determination signal and the second determination signal, the condition determination unit 30e determines that the safety of the worker is not ensured. Then, a danger signal indicating that the worker is in a dangerous state is transmitted. Further, when the condition determination unit 30e determines that the predetermined conditions are not satisfied when receiving both or one of the first determination signal and the second determination signal, the safety of the worker is ensured. It may be determined that a safety signal indicating the safety of the worker is transmitted. The predetermined conditions will be exemplified in the operation procedure of the safety determination device 30 and the safety determination system 1 described later.

The signal receiving unit 30f is a function of receiving the measurement information measured by the sensor 20. The signal receiving unit 30f outputs the measurement information to the condition determination unit 30e. As a result, the condition determination unit 30e can use the measurement information to determine whether or not a predetermined condition is satisfied.

<< Surveillance camera 40 >>
The surveillance camera 40 is, for example, a device for photographing a worker in the line-of-sight direction of the observer who monitors the worker. For example, it is composed of an image sensor such as a single-plate or multi-plate CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Sensor), and a plurality of components such as a lens with a wide angle of view (wide-angle lens), and has an infrared mode. It is desirable to have. However, the specifications are not limited. Further, the surveillance camera 40 has a function of wirelessly or wired communication with, for example, the second communication unit of the surveillance device 50, and transmits the surveillance image information to the surveillance device 50 by wireless communication or wired communication.

<< Monitoring device 50 >>
The monitoring device 50 is a device that is confirmed by a watchman who monitors a worker, and uses image processing technology to check whether or not the work safety is ensured when the worker works on a work object. It is a device to judge by using. By providing the monitoring device 50, it is possible to grasp whether or not the worker is accompanied by a predetermined operation, so that the safety can be improved as compared with the safety judgment by the safety judgment device 30 alone.

FIG. 5 is a diagram showing an example of the hardware configuration of the monitoring device 50. As shown in FIG. 5, the hardware of the monitoring device 50 includes a processor 51, a memory 52, a storage device 53, an input device 54, an output device 55, and a second communication device 56. ing. The processor 51 is, for example, an MPU, a CPU, or the like. The memory 52 is, for example, a RAM, a ROM, an NVRAM, or the like. The storage device 53 is, for example, a RAM, a ROM, an NVRAM, or the like. The input device 54 is a user interface that receives operation input from the user, and is, for example, an operation input device (keyboard, mouse, touch panel, etc.), a voice input device (microphone, etc.), and the like. The output device 55 is a user interface that provides various information to the user, and is, for example, a display device (liquid crystal monitor or the like), an audio output device (speaker or the like), or the like. The output device 55 displays, for example, a determination result on a display screen (not shown) according to the determination signal, or outputs the determination result by voice through a speaker. The second communication device 56 is an interface for connecting to a network, and is, for example, a wireless LAN adapter, a NIC (Network Interface Card), or the like.

FIG. 6 is a diagram showing an example of the software configuration of the monitoring device 50. As shown in FIG. 6, the software of the monitoring device 50 has the functions of the monitoring image input unit 50a, the target recognition unit 50b, the person recognition unit 50c, and the monitoring determination unit 50d. These functions are realized, for example, by the processor 51 of the monitoring device 50 reading and executing the program stored in the memory 52. It should be noted that these functions may be realized by hardware such as ASIC. Further, the processor 51 may be realized by reading and executing a program stored in an external storage medium. Note that FIG. 6 shows that the monitoring device 50 realizes each function with one information processing device (computer), but the present invention is not limited to this. For example, each of the above-mentioned functions may be configured to be distributed and realized by two or more information processing devices. Each function of the software configuration of the monitoring device 50 will be described below.

The surveillance image input unit 50a is a function of inputting a surveillance image taken by the surveillance camera 40. The surveillance image input unit 50a transmits the input surveillance image to the target recognition unit 50b and the person recognition unit 50c. In addition, although it is shown in FIG. 1 that the surveillance image is directly input from the surveillance camera 40 to the surveillance image input unit 50a, it is transmitted from the surveillance camera 40 to a server device (not shown) installed outside. However, the monitoring image may be configured to be input to the monitoring image input unit 50a from the server device.

The object recognition unit 50b is a function of recognizing a work object reflected in a surveillance image. The object recognition unit 50b executes so-called preprocessing such as image correction, image quality improvement, and image coordination to remove noise contained in the input surveillance image and extract the characteristics of the object. Recognize. The object recognition unit 50b has a function of transmitting object information indicating an object recognized on the image to the monitoring determination unit 50d. Since the means for recognizing an object is the same as that of the object recognition unit 30b of the safety determination device 30, the description thereof will be omitted.

The human recognition unit 50c is a function of recognizing a worker (person) shown in a surveillance image. The human recognition unit 50c recognizes a worker by using, for example, a HOG (Histograms of Oriented Gradients) feature quantity. The method of human recognition using this HOG feature is to collect the gradient direction and intensity from the brightness on both the top, bottom, left, and right sides of a predetermined pixel coordinate, generate a histogram for cells in units of several pixels, and then generate a histogram. It is a method of recognizing a person by calculating the feature amount by normalizing with a block of each cell. Further, it is desirable that the human recognition unit 50c has the same function as the hand recognition unit 30c of the safety determination device 30. The above is an example, and as for the method of recognizing a worker as an image, for example, a method of learning a worker's image by using a neural network to recognize the worker or a method of recognizing a worker from a difference from a background image is used. A recognition method or the like may be used, and the method is not limited. The human recognition unit 50c has a function of transmitting worker information indicating a worker recognized on the image to the monitoring determination unit 50d.

The monitoring determination unit 50d is a function of determining that the object and the worker appearing in the monitoring image overlap. The monitoring determination unit 50d recognizes that the worker overlaps the image-recognized object on the image based on the object information and the worker information. Specifically, the monitoring determination unit 50d compares, for example, the luminance value of the object that changes when the color area of the work object and the color area of the worker overlap with a predetermined threshold value. Detects the overlap between the object and the worker. The above is only an example, and as a method of recognizing an overlap, for example, a method of learning an image in which an object and a worker overlap and recognizing the overlap may be used by using a neural network. , The method is not limited. The monitoring determination unit 50d has a function of transmitting a second determination signal indicating the result of detecting the overlap between the object and the worker to the safety determination device 30. With this function, it is possible to grasp whether or not the worker is working according to the safety procedure.

Although it has been described above that the worker carries the safety determination device 30, the present invention is not limited to this. The safety determination device 30 may be integrally configured with the monitoring device 50, and in this case, the observer can grasp the safety state of the worker.

=== Operation procedure of the safety determination device 30 ===
With reference to FIGS. 7 to 13, the procedure for safety determination by the safety determination device 30 carried by the worker will be described in detail with reference to specific examples. In the following description, when each function is the subject, it means that the processor 31 reads a program and realizes a predetermined function.

First, the worker confirms the work object. The work object is photographed by the camera 10 that photographs the tip of the worker's line of sight. The image input unit 30a acquires the captured image from the camera 10 (S10). The object recognition unit 30b recognizes a work object in a predetermined area in the acquired captured image (S11). The hand recognition unit 30c recognizes the worker's hand in a predetermined area in the captured image (S12). The action determination unit 30d determines that the work object and the worker's hand overlap (S13), and outputs a first determination signal indicating the determination result to the condition determination unit 30e (S14). The condition determination unit 30e receives the first determination signal, and then determines whether or not a predetermined condition is satisfied (S14). When it is determined by the condition determination unit 30e that the worker is working in a dangerous state, the condition determination unit 30e outputs a danger signal, and the output device 15 to which the danger signal is input determines the danger state. Display (S16). The above-mentioned processing procedure will be described below with some specific examples.

First, as an example, when a worker performs wiring work in the cubicle 100, the object recognition unit 30b recognizes the handle 110 (work object) of the cubicle 100 as shown in FIG. 8A, and further, FIG. 9 As shown in (A), the wiring portion 120 (working object) in the cubicle 100 is recognized. The hand recognition unit 30c recognizes the worker's hand on the captured image. As shown in FIG. 8B, the action determination unit 30d determines that the handle 110 and the worker's hand overlap. Further, the action determination unit 30d subsequently determines that the wiring unit 120 and the worker's hand overlap each other as shown in FIGS. 9A and 9B, and outputs a first determination signal indicating the determination result. Output to the condition determination unit 30e. The condition determination unit 30e receives a first determination signal indicating that the wiring unit 120 and the worker's hand overlap after the handle 110 and the worker's hand overlap, and the worker's hand is flesh-colored (the worker's hand). It is determined whether or not it is a predetermined condition). When the condition determination unit 30e determines that the worker's hand is flesh-colored (FIG. 9A), it determines that the predetermined condition is satisfied. In this case, the condition determination unit 30e determines that the safety of the worker is not ensured in the work, and outputs a danger signal to the output device 15. The worker can recognize that the work is in a dangerous state by the danger display displayed on the output device 15 and the voice output notifying the danger. Here, when the worker is wearing insulating gloves, the worker's hand does not show the skin color (FIG. 9B), so that the action determination unit 30d uses the second determination signal as the condition determination unit 30e. Do not output to. In this case, the condition determination unit 30e may output a safety signal to the output device 15.

Further, as another example, when the worker operates the crane, the object recognition unit 30b recognizes the operation button 200 (work object) of the crane. The hand recognition unit 30c recognizes the worker's hand on the captured image. As shown in FIG. 10A, the action determination unit 30d determines that the operation button 200 and the worker's hand overlap, and outputs the first determination signal to the condition determination unit 30e. Whether the condition determination unit 30e has received the first determination signal and, in the past, has received hand information indicating a predetermined hand shape as shown in FIG. 10B from the hand recognition unit 30c (predetermined condition). ) Judge whether or not. This determination procedure is a procedure for determining whether or not the hook 210 of the crane or the like is pointed and confirmed before the operation button 200 is pressed. That is, when the condition determination unit 30e determines that the hand information indicating the shape of the predetermined hand has not been received in the past, it is determined that the predetermined condition is satisfied. In this case, the condition determination unit 30e determines that the safety of the worker in the work is not ensured, and outputs a danger signal to the output device 15. The worker can recognize that the work is in a dangerous state by the danger display displayed on the output device 15 and the voice output notifying the danger.

Further, as another example, when a worker climbs a steel tower or a utility pole, the object recognition unit 30b recognizes a ladder 300 (work object). The hand recognition unit 30c recognizes the worker's hand on the captured image. As shown in FIG. 11A, the action determination unit 30d determines that the ladder 300 and the worker's hand overlap, and outputs the first determination signal to the condition determination unit 30e. The condition determination unit 30e receives the first determination signal, and then determines whether or not the object recognition unit 30b recognizes the landing place 310 of the steel tower as shown in FIG. 11B, for example (predetermined condition). For recognizing the landing 310 of the iron tower of the object recognition unit 30b, for example, an analysis by structural texture, an analysis by density distribution, an analysis by template matching, an analysis by a decision tree method, an analysis by clustering, or the like is used. When the object recognition unit 30b recognizes the landing 310, the condition determination unit 30e can determine that the worker has climbed to a height that is off limits, and therefore determines that the predetermined condition is satisfied. In this case, the condition determination unit 30e determines that the safety of the worker is not ensured in the work, and outputs a danger signal to the output device 15. The worker can recognize that the work is in a dangerous state by the danger display displayed on the output device 15 and the voice output notifying the danger.

Further, as another example, when the worker performs weeding work with the mower 400, the object recognition unit 30b recognizes the handle 410 (work object) of the mower 400. The hand recognition unit 30c recognizes the worker's hand on the captured image. As shown in FIG. 12, the action determination unit 30d determines that the handle 410 of the mower 400 and the worker's hand overlap, and outputs the first determination signal to the condition determination unit 30e. The condition determination unit 30e receives the first determination signal, and then acquires the measurement signal from the sensor 20 via the signal reception unit 30f. The condition determination unit 30e determines whether or not the measurement signal exceeds a predetermined threshold value (predetermined condition), and if it exceeds the predetermined threshold value, it can be determined that the worker is in poor physical condition, so that the predetermined condition is satisfied. It is determined that it has been done. In this case, the condition determination unit 30e determines that the safety of the worker is not ensured in the work, and outputs a danger signal to the output device 15. The worker can recognize that the work is in a dangerous state by the danger display displayed on the output device 15 and the voice output notifying the danger.

Further, as another example, when the worker performs the charging operation, the safety determination device 30 recognizes the contact button 510 (work object) of the contact terminal 500. The hand recognition unit 30c recognizes the worker's hand on the captured image. As shown in FIG. 13A, the action determination unit 30d determines that the contact button 510 and the worker's hand overlap, and outputs the first determination signal to the condition determination unit 30e. The condition determination unit 30e has received the first determination signal and has recognized in the past that the object recognition unit 30b has removed the Class B earth 520 (determination object) as shown in FIG. 13B from the ground electrode 530. It is determined whether or not (predetermined condition). This determination procedure is a procedure for determining whether or not the Class B earth 520 is removed from the ground electrode 530 before pressing the contact button 510. The condition determination unit 30e recognizes that the object recognition unit 30b removes the class B earth 520 from the grounding electrode 530, that is, when the state in which the class B earth 520 is attached to the grounding electrode 530 does not continue, a predetermined state is determined. It is determined that the conditions are satisfied. In this case, the condition determination unit 30e determines that the safety of the worker in the work is not ensured, and outputs a danger signal to the output device 15. The worker can recognize that the work is in a dangerous state by the danger display displayed on the output device 15 and the voice output notifying the danger.

=== Operation procedure of safety judgment system 1 ===
With reference to FIGS. 14 to 17, the procedure for safety determination of the safety determination system 1 by the safety determination device 30 carried by the worker and the monitoring device 50 confirmed by the observer will be described as follows.

First, the worker confirms the work object. The work object is photographed by the camera 10 that photographs the tip of the worker's line of sight. The image input unit 30a of the safety determination device 30 acquires the captured image from the camera 10 (S20). The object recognition unit 30b of the safety determination device 30 recognizes a work object in a predetermined area in the acquired captured image (S21). The hand recognition unit 30c of the safety determination device 30 recognizes the worker's hand in a predetermined area in the captured image (S22). The action determination unit 30d of the safety determination device 30 determines that the work object and the worker's hand overlap (S23), and outputs a first determination signal indicating the determination result to the condition determination unit 30e (S24).

The observer then confirms the worker. For example, as shown in FIG. 15, in a situation where a worker inspects the cubicle 100, the worker is photographed by a surveillance camera 40 that captures the line of sight of the observer. In FIG. 15, the broken line indicates the shooting direction, and the solid line indicates the flow of information. The surveillance image input unit 50a of the surveillance device 50 acquires the captured surveillance image from the surveillance camera 40 (S25). The object recognition unit 50b of the monitoring device 50 recognizes an object in a predetermined area in the acquired monitoring image (S26). The human recognition unit 50c of the monitoring device 50 recognizes a worker in a predetermined area in the monitoring image (S27). The monitoring determination unit 50d of the monitoring device 50 determines that the object and the worker overlap each other (S28), and outputs a second determination signal indicating the determination result to the condition determination unit 30e of the safety determination device 30 (S29). ..

Next, the condition determination unit 30e of the safety determination device 30 receives the first determination signal and the second determination signal, and determines whether or not the predetermined conditions are satisfied (S30). When it is determined by the condition determination unit 30e that the worker is working in a dangerous state, the condition determination unit 30e outputs a danger signal, and the output device 15 to which the danger signal is input determines the danger state. Display (S31). The above-mentioned processing procedure will be described below with some specific examples.

First, as an example, when the worker prohibits the removal of the Class B ground 610 during a power failure, the safety determination device 30 is a handle portion 620 (work object) of the Class B ground 610 as shown in FIG. 16 (A). In addition to recognizing the worker's hand, the overlap between the handle portion 620 of the Class B earth 610 and the worker's hand is recognized. The first determination signal is input to the condition determination unit 30e. Next, the observer confirms the worker who performs the charging operation. As shown in FIG. 16B, the monitoring device 50 recognizes the handle portion 620 (object) of the class B earth 610 and the worker, and recognizes the overlap between the handle portion 620 of the class B earth 610 and the worker. .. The monitoring device 50 outputs the second determination signal to the condition determination unit 30e of the safety determination device 30. The condition determination unit 30e of the safety determination device 30 determines the overlap between the handle unit 620 of the class B earth 610 and the worker's hand based on the first determination signal and the second determination signal, and then determines the overlap of the hands of the worker, and then the handle unit 620 of the class B earth 610. Since it is determined that the workers overlap with each other, it is determined that the Class B earth 610 is removed from the grounding electrode 600 during the power failure, and it is determined that the predetermined conditions are satisfied. In this case, the condition determination unit 30e determines that the safety of the worker in the work is not ensured, and outputs a danger signal to the output devices 15 and 25. Workers and observers can recognize that the work is in a dangerous state by the danger display displayed on the output devices 15 and 25 and the voice output notifying the danger.

Further, as another example, when a worker climbs a steel tower or a utility pole, the safety determination device 30 recognizes the ladder 700 (work object) as shown in FIG. 17 (A), and the ladder 700 and the worker. Recognize the overlap with the hand. The first determination signal is input to the condition determination unit 30e. Next, the observer confirms the worker who climbs the ladder 700. The monitoring device 50 recognizes the ladder 700 and the worker. Then, the monitoring device 50 determines whether or not the ladder 700 and the worker overlap each other. As shown in FIG. 17B, the monitoring device 50 outputs a second determination signal to the condition determination unit 30e when it is determined that the ladder 700 and the worker overlap each other. Since the condition determination unit 30e determines the overlap between the ladder 700 and the worker's hand based on the first determination signal and the second determination signal, and then determines the overlap between the ladder 700 and the worker, the observer determines. Since it can be determined that the worker is climbing the ladder 700 while the worker is being monitored, it is determined that the safety of the worker is ensured, and it is determined that the predetermined condition is not satisfied. In this case, the condition determination unit 30e outputs a safety signal indicating the result of determining that the worker is working in a safe state to the output devices 15 and 25. As a result, the worker and the observer can recognize that the work is in a safe state by the safety display displayed on the output devices 15 and 25 and the voice output informing the safety. If the condition determination unit 30e does not receive the second determination signal from the monitoring device 50 after receiving the first determination signal, the condition determination unit 30e determines that the safety of the worker is not ensured in the work, and the output device 15 A danger signal may be output to, 25. As a result, the worker and the observer can recognize that the work is in a dangerous state by the danger display displayed on the output devices 15 and 25 and the voice output notifying the danger.

=== Summary ===
As described above, the safety determination device 30 according to the present embodiment has an image input unit 30a for inputting a captured image captured by the camera 10 captured toward the line of sight of the worker, and within the image area of the captured image. The object recognition unit 30b that recognizes the work object as an image, the hand recognition unit 30c that recognizes the worker's hand in the image area of the captured image, and the work object and the hand overlap in the image area of the captured image. When the action determination unit 30d for determining whether or not the image is present and the action determination unit 30d determine that the work object and the worker's hand are overlapped with each other, the worker determines that the work is safe in relation to the work object. A condition determination unit 30e for determining whether or not a predetermined condition indicating an unsecured situation is satisfied is provided. According to the present embodiment, it is possible to easily grasp whether or not the safety state of the work is ensured from the viewpoint of the worker himself, so that the safety of the work can be improved.

Further, the object recognition unit 30b of the safety determination device 30 according to the present embodiment recognizes that the work object exists in a predetermined image area in the captured image. According to this embodiment, it is possible to facilitate image recognition of a work object.

Further, the condition determination unit 30e of the safety determination device 30 according to the present embodiment satisfies the predetermined condition when the hand recognition unit 30c recognizes that the color of the worker's hand is a predetermined color based on the captured image. It is determined that it is. According to this embodiment, it is possible to prevent the work in a situation where the worker is not wearing protective equipment such as insulating gloves.

Further, when the condition determination unit 30e of the safety determination device 30 according to the present embodiment does not recognize that the shape of the worker's hand is a predetermined shape based on the captured image, the hand recognition unit 30e is predetermined. It is determined that the conditions are satisfied. According to the present embodiment, since it is determined that a predetermined operation cannot be confirmed during the work procedure of the worker, the worker can be made to comply with the work procedure.

Further, the condition determination unit 30e of the safety determination device 30 according to the present embodiment determines that the predetermined condition is satisfied when the object recognition unit 30b recognizes the predetermined equipment based on the captured image. According to this embodiment, it is possible to prevent workers from entering a place where they should not enter.

Further, the safety determination device 30 according to the present embodiment further includes a signal receiving unit 30f for receiving a signal output from a predetermined sensor 20 worn by the worker, and the condition determination unit 30e of the safety determination device 30 is a signal receiving unit. Based on the signal received from the sensor 20 by the 30f, it is determined whether or not the predetermined condition is satisfied. According to this embodiment, it is possible to determine whether the work is in a safe state or a dangerous state according to the physical condition of the worker.

Further, the object recognition unit 30b of the safety determination device 30 according to the present embodiment image-recognizes a determination object different from the work object in the image area of the captured image, and the action determination unit 30d is an image area of the captured image. The condition determination unit 30e determines whether or not the determination object and the hand overlap with each other, and the condition determination unit 30e determines that the work object and the worker's hand overlap with each other, and then the determination object 30e. If the overlap between the worker's hand and the worker's hand cannot be recognized, it is determined that the predetermined condition is satisfied. If the predetermined operation for the work object and the predetermined operation for the determination object are not satisfied, it can be determined that the worker is working in a dangerous state.

It should be noted that the above embodiment is for facilitating the understanding of the present invention, and is not for limiting the interpretation of the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes an equivalent thereof.

1 Safety judgment system 10 Camera 20 Sensor 30 Safety judgment device 30a Image input unit 30b Object recognition unit 30c Hand recognition unit 30d Action judgment unit 30e Condition judgment unit 30f Signal reception unit 40 Surveillance camera 50 Surveillance device 50a Surveillance image input unit 50b Target recognition Unit 50c Person recognition unit 50d Monitoring and judgment unit

Claims (10)

An image input unit that inputs images taken by a camera that shoots toward the worker's line of sight,
An object recognition unit that recognizes a work object in the image area of the image,
A hand recognition unit that recognizes the worker's hand in the image area of the image,
An action determination unit that determines whether or not the work object and the hand overlap in the image area of the image.
When the action determination unit determines that the work object and the worker's hand overlap, a predetermined indicating a situation in which the worker does not ensure work safety in relation to the work object. A condition judgment unit that determines whether or not the conditions of
A safety judgment device characterized by being provided with. The safety determination device according to claim 1, wherein the object recognition unit recognizes that the work object exists in a predetermined image area in the image. The condition determination unit is characterized in that when the hand recognition unit recognizes that the color of the worker's hand is a predetermined color based on the image, it determines that the predetermined condition is satisfied. The safety determination device according to claim 1 or 2. When the hand recognition unit does not recognize that the shape of the worker's hand is a predetermined shape based on the image, the condition determination unit determines that the predetermined condition is satisfied. The safety determination device according to claim 1 or 2. The first or second aspect of the present invention, wherein the condition determination unit determines that the predetermined condition is satisfied when the object recognition unit recognizes the predetermined equipment based on the image. Safety judgment device. Further equipped with a signal receiving unit for receiving a signal output from a predetermined sensor worn by the worker.
The first or second aspect of the present invention, wherein the condition determination unit determines whether or not the predetermined condition is satisfied based on the signal received from the sensor by the signal reception unit. Safety judgment device. The object recognition unit image-recognizes a determination object different from the work object in the image area of the image.
The action determination unit determines whether or not the determination object and the hand overlap in the image area of the image.
If the condition determination unit cannot recognize the overlap between the determination object and the worker's hand after the action determination unit determines that the work object and the worker's hand overlap, the condition determination unit may not recognize the overlap. The safety determination device according to claim 1 or 2, wherein it is determined that a predetermined condition is satisfied. The safety determination device according to any one of claims 1 to 7.
The camera, which is worn by the worker and takes a picture toward the worker's line of sight.
Safety judgment system equipped with. A surveillance image input unit for inputting a surveillance image taken by a surveillance camera that shoots toward the line of sight of the observer who monitors the worker, and a monitoring image input unit.
An object recognition unit that recognizes a predetermined object in the image area of the surveillance image,
A person recognition unit that recognizes the worker in the image area of the surveillance image,
A monitoring determination unit that determines whether or not the object and the worker overlap in the image area of the image.
Further equipped with a monitoring device
The condition determination unit of the safety determination device is
When the action determination unit determines that the work object and the worker's hand overlap, and the monitoring determination unit determines that the object and the worker overlap, the predetermined The safety determination system according to claim 8, wherein it is determined whether or not the condition of the above is satisfied. An image input step to input an image taken by a camera taken by a computer toward the worker's line of sight,
An object recognition step of recognizing a work object in the image area of the image,
A hand recognition step of recognizing the worker's hand in the image area of the image,
An action determination step for determining whether or not the work object and the hand overlap in the image area of the image, and
When it is determined in the action determination step that the work object and the worker's hand overlap, a predetermined indicating a situation in which the worker does not ensure work safety in relation to the work object. Condition judgment step to judge whether or not the condition of
Safety judgment method to execute.

Images