KR102551644B1 - Safety management system through image-based core framing detection - Google Patents

Abstract

The present invention is based on an input image taken by a 3D camera or a general CCTV camera without a sensor, and includes only the minimum core skeleton necessary for recognizing the actions of workers and pedestrians in the image in the object detection process to obtain a high-end processing system and By improving the accuracy of human behavior by easily grasping the phase relationship between detected objects without using additional equipment, workers and pedestrians in the input image enter dangerous areas, check whether two people are working in pairs, fall down, wear hard hats, rescue It relates to a safety management system achieved through image-based core skeleton detection, which enables rapid judgment of risk situation analysis such as signal recognition and response and emergency response.

Description

Safety management system through image-based core framing detection}

The present invention is based on an input image taken by a 3D camera or a general CCTV camera without a sensor, and includes the minimum core skeleton necessary for recognizing the actions of workers and pedestrians in the image in the object detection process to obtain a high-end processing system and By improving the accuracy of human behavior by easily grasping the phase relationship between detected objects without using additional equipment, workers and pedestrians entering the danger zone in the input image, checking whether two people are working in pairs, whether they are wearing hard hats, and rescue signals It relates to a safety management system achieved through image-based core skeleton detection that quickly determines risk situation analysis such as recognition and fall, and responds to and responds to emergency situations.

In relation to the present invention, in the 'feature extraction method and apparatus for machine learning for motion recognition from 3-dimensional skeleton information' of Korean Patent Publication No. 10-2018-0028886 (published on March 19, 2018), 'real-time of human motion As a method of extracting features as learning data from 3D skeletal information for recognition, it is frame-by-frame for real-time motion recognition using position information and rotation information of each joint included in skeletal information (skeleton information) representing human motion. A feature extraction method from 3D skeletal information for motion recognition, including extracting biomechanical features with .' has been disclosed.

The published patent is a method for extracting features for a 3D skeleton, and has a disadvantage in that separate equipment such as a 3D camera and a sensor is required to extract the 3D skeleton.

On the other hand, in the present invention, without a 3D camera and sensor, a general CCTV camera extracts the core skeleton of the human body to recognize the action and proposes a method for prevention through risk situation recognition using the same.

In Korean Patent Publication No. 10-2018-0096038 (published on August 29, 2018), 'multiple motion-based omni-view technique for behavior prediction', 'a behavior pattern-based crime prediction device, at least one of a fixed camera and a rotating camera A communication unit for receiving an image transmitted from a camera of the communication unit, an analysis device for predicting a crime by analyzing the behavior of all persons in the image received through the communication unit, and An output unit for notifying an object, and the analysis device is a distance recognition module for recognizing a distance between all persons in the image, and if the distance corresponds to a social distance range or an intimacy range, the distance range The distance recognition module configured to specify a person within, and an omni-view pattern recognition module for analyzing a body pattern and an arm pattern of the specified person, obtaining coordinates of at least a part of a skeleton of the person, Stored as dimensional coordinates, analyzing the pattern of the body and arm of the person, and as a result of the analysis, if the pattern of the body and arm of the person is analyzed to deviate from a predetermined standard pattern as a normal pattern, An omni-view pattern recognition module configured to transmit a corresponding result to a determination module, and a hand pattern recognition module configured to analyze a hand of the person based on the three-dimensional coordinates and transmit a coordinate change that occurs when an abnormal behavior occurs to the determination module. As a hand pattern recognition module configured to predict risk by finding the number of fingers of a person using the convexhull algorithm, and a decision module for predicting abnormal behavior by combining the analysis results of each module, starting from recognizing the distance between people The body pattern, arm pattern, and hand pattern are identified sequentially, and when abnormal behavior is recognized at each stage, it moves to the next stage and when it is identified that the abnormal behavior deviates from the predetermined standard pattern at each stage, the result is output through the output unit. A technique for a 'behavioral pattern-based crime prediction device' characterized in that it includes a decision module configured to notify has been disclosed.

The Patent Publication No. 10-2018-0096038 analyzes the behavior of all characters in the video and determines whether or not to analyze the behavior by limiting the distance between the characters Analyzing the behavior of the person within a certain distance and analyzing the behavior of the interaction with each other Omniview (Using images from multiple angles) Since crime is predicted by analyzing behavioral patterns using the directionality of behavior, it has the disadvantage of requiring cameras from various angles to analyze behavior. In addition, there is a disadvantage in that a high-end processing system and additional equipment are required because the behavior pattern is recognized by stepwise analysis from the entire area such as the body > arm > hand to detailed areas.

On the other hand, the present invention is the same as analyzing the behavior of all characters in the image, but there is a difference in that the action recognition is performed through the detection of the core skeleton of the person in the image, which is different from processing the entire image and processing some images If present, this has the advantage of reducing the amount of computation during image analysis. In addition, the present invention proceeds with the action recognition of the entire area and recognizes the action by dividing it into primary and secondary.

That is, in the case of a rescue signal such as waving the arm above the head, the first action recognition is the recognition of the hand position above the shoulder, and the second action recognition is the action recognition through the amount of change over time. In the case of secondary action recognition, there is a differentiated point of proceeding with action recognition through the amount of change in the skeleton that varies over time.

As such, the conventionally disclosed dangerous situation recognition method recognizes intrusion into a dangerous area, departure from a safe area, etc. based on human movement through object detection and tracking, and uses accurate human behavior such as skeleton extraction and recognition. The risk situation recognition method had the disadvantage of requiring the use of a high-end processing system and additional equipment.

Republic of Korea Patent Publication 10-2018-0028886 (published on March 19, 2018) Republic of Korea Patent Publication 10-2018-0096038 (published on August 29, 2018)

The present invention is based on an input image taken by a 3D camera or a general CCTV camera without a sensor, and includes only the minimum core skeleton necessary for recognizing the actions of workers and pedestrians in the image in the object detection process to obtain a high-end processing system and By improving the accuracy of human behavior by easily grasping the phase relationship between detected objects without using additional equipment, workers and pedestrians in the input image enter dangerous areas, check whether two people are working in pairs, fall down, wear hard hats, rescue It is an object of the invention to provide a safety management system made through image-based core skeleton detection that enables quick determination of dangerous situation analysis such as signal recognition and response and emergency response.

In order to achieve the above purpose,

The present invention is a camera; A communication unit for receiving the image acquired through the camera, and a risk situation recognition unit for detecting a core skeleton based on the image provided through the communication unit and deriving an action recognition result through primary and secondary action recognition processes; It consists of a; safety management processing unit including a sending unit for transmitting the action recognition result derived through the risk situation recognition unit to a manager,

Obtaining an image of a specific area through the camera not including a sensor (S10);

An object detection and tracking step (S20) of detecting a core skeleton including a head, hand, arm or leg as an object based on an image obtained through the camera, recognizing the location of these objects and displaying a movement path; ,

A first action recognition step (S30) of identifying the phase relationship between each object identified through the object detection and tracking step, and proceeding with human action recognition based on this;

After passing through the first action recognition step, a second action recognition step (S40) of recognizing whether or not a distress signal is present through the amount of angle change between the objects;

It provides a safety management system through image-based core skeleton detection, characterized in that it includes a step (S50) of transmitting the action recognition result derived through the second action recognition step to the manager's terminal.

The safety management system through image-based core skeleton detection according to the present invention has the following effects.

first. It has the advantage of being able to simplify the action recognition by performing it through specific trigger action recognition.

second. Since the core skeleton of the human body is included in the object detection fixation, risk situation analysis and action recognition using the skeleton are possible based on the continuous change of the human body movement, it has the advantage of being lightweight and inexpensive, and accurate event recognition is possible.

third. Even if a person is not equipped with a sensor, it has the advantage that action recognition for rescue signals generated from roads, power plants, etc., and sudden collapse can be quickly and accurately performed.

1 is an overall configuration diagram of a safety management system made through image-based core skeleton detection according to the present invention.
Figure 2 is a view showing a behavior recognition process made through the safety management system shown in Figure 1;
Figure 3 is a photograph showing an example of a risk situation analysis method through a conventional human movement.
4 is a photograph showing an example of a method for recognizing a behavior through detailed movement of a conventional skeletal unit.
5 is a photograph showing an emergency situation recognition process through core skeleton detection according to the present invention by way of example.
6 is a photograph showing a state before detection of a fall as an example of an emergency situation recognition process through core skeleton detection according to the present invention.
7 is an example of an emergency situation recognition process through core skeleton detection according to the present invention, and a photograph showing a state after a fall is detected.

Hereinafter, detailed technical contents of the safety management system achieved through image-based core skeleton detection according to the present invention will be reviewed together with drawings.

As shown in FIGS. 1 and 2, the safety management system achieved through image-based core skeleton detection according to the present invention,

camera 10; The communication unit 21 for receiving the image obtained through the camera 10 and the core skeleton detected based on the image provided through the communication unit 21, and the behavior recognition result through the first and second action recognition processes. It consists of a risk situation recognition unit 22 that derives and a safety management processing unit 20 including a transmission unit 23 that transmits the action recognition result derived through the risk situation recognition unit 22 to a manager. ,

Obtaining an image of a specific region through the camera 10 that does not include a sensor (S10);

Object detection and tracking step of detecting the core skeleton including the head, hand, arm or leg as an object based on the image acquired through the camera 10, recognizing the location of these objects and displaying the movement path ( S20) and,

A first action recognition step (S30) of identifying the phase relationship between each object identified through the object detection and tracking step, and proceeding with human action recognition based on this;

After passing through the first action recognition step, a second action recognition step (S40) of recognizing whether or not a distress signal is present through the amount of angle change between the objects;

and transmitting the action recognition result derived through the second action recognition step to a manager's terminal (S50).

As described above, the safety management system according to the present invention includes a camera 10,

It consists of a communication unit 21, a risk situation recognition unit 22, and a safety management processing unit 20 including a sending unit 23.

At this time, the dangerous situation recognition unit 22 detects and tracks the object; primary action recognition; and secondary action recognition; Through the process, the dangerous situation including whether to intrude into the dangerous area, whether to work in a group of two people, whether to fall down, whether to wear a hard hat, or rescue signal recognition is analyzed, and the analyzed result is transmitted to the manager's terminal through the transmission unit 23 serves to provide notifications.

The safety management system according to the present invention includes the head, hand, arm, or leg, which are the core skeletons of the human body, in the object detection process to enable situation analysis through movement and action recognition using the skeleton, thereby detecting existing objects and By resolving the difficulty in analyzing human behavior through simple movements (FIG. 3) and the fact that a large amount of computation is required to analyze detailed behavior by detecting it in units of human skeletons (FIG. 4), Its technical feature is that it can quickly and clearly identify dangerous situations even with a low amount of computation.

The object detection is based on deep learning. Deep learning is machine learning in which computers mimic the human brain to form large artificial neural networks. Deep learning is an advanced form of an artificial neural network model, and is a structure consisting of several hidden layers in an artificial neural network, which is a structure of an input layer, a hidden layer, and an output layer.

The object detection is based on image analysis through a convolutional neural network (CNN) deep learning algorithm. The convolutional neural network (CNN) is a deep learning area of an artificial neural network (ANN) among machine learning models. Build a neural network to improve performance.

The convolution step is a process of extracting features using a filter, and the number of pixels to be skipped when applying the filter is called a stride. The larger the stride, the faster the calculation speed and the lower the hit rate.

The pooling step uses the max pooling technique to compress the size of the feature map obtained through the convolution step.

When the process of the Fully Connected step is performed after repeating the convolution step and the pooling step, it is possible to extract the input image features connected to the output unit of the deep learning.

As shown in FIG. 5, the safety management system achieved through image-based core skeleton detection according to the present invention

Based on the input image, a key object selected from the head, hand, arm, or leg of a person is detected, and a dangerous situation is analyzed through the key object to determine whether or not an emergency is necessary and a notification is provided.

In order to determine accurate behavior through human movement, a high-end processing system and additional equipment are required because a dangerous situation must be recognized through skeleton extraction and recognition. It has the advantage that action recognition can be performed lightly and at low cost by detecting and performing action recognition based on this.

In this way, the safety management system through image-based core skeleton detection detects core objects corresponding to the core skeleton of the head, hand, arm, or leg through the input image, and analyzes the dangerous situation through these core objects. Then, a notification of the analysis result is provided to the manager's terminal.

Examples of the dangerous situation analysis include intrusion into a dangerous area, confirmation of a two-person team operation, wearing of a hard hat, rescue signal recognition, and falling.

In the case of intrusion into the dangerous area, a risk area is designated for each stage and notification is provided only when a person in the area is detected.

In the case of checking whether a group of two people is working, a notification is provided if there are less than two people in the detection area.

Whether or not the safety helmet is worn is determined by checking whether or not the safety helmet is detected in the head region to detect whether the helmet is worn or not worn, and then a notification is provided when the helmet is not worn.

In relation to the distress signal recognition, a picture is exemplarily presented in FIG. 5 .

Looking more closely at the distress signal recognition process,

First, a core object, which is the core skeleton of the human body, is detected.

Next, the topological relationship between objects is identified. For example, based on the lower center point of the face area, it is checked whether the positions of both hands are above.

Next, the rescue signal is recognized through the amount of angle change between objects. For example, it is determined based on the number of times that the change in angle between the hands, the center point under the face area, and the hand exceeds the angle threshold value during a designated time threshold value.

In relation to the fall, pictures are exemplarily presented in FIGS. 6 and 7 .

6 and 7 are examples of behavior recognition through the safety management system according to the present invention, and show the state recognition process before and after the fall detection through the safety management system of the present invention.

In case of 'falling', one of the risk situations, action recognition is made by comprehensively judging the case where the human detection area is shortened along the Y-axis and lengthened along the X-axis, and the center coordinates of the Murray detection area rapidly change.

6 is before fall detection, the red circle on the person detection area means before fall detection, and FIG. 7 is after fall detection, and the green circle on the person detection area means fall detection.

The safety management system using image-based action recognition according to the present invention has the advantage of being able to simplify action recognition at low cost and lightly by performing specific trigger action recognition, and based on continuous changes for each body movement, Since action recognition is performed, it has the advantage of being able to recognize more accurate events, and even if a person is not equipped with a sensor, it has the advantage that action recognition can be performed lightly and accurately for rescue signals generated from roads, power plants, etc., and suddenly falling down. Therefore, it has high industrial applicability.

Claims (3)

camera 10; The communication unit 21 for receiving the image obtained through the camera 10 and the core skeleton detected based on the image provided through the communication unit 21, and the behavior recognition result through the first and second action recognition processes. It consists of a risk situation recognition unit 22 that derives and a safety management processing unit 20 including a transmission unit 23 that transmits the action recognition result derived through the risk situation recognition unit 22 to a manager. ,
Obtaining an image of a specific region through the camera 10 that does not include a sensor (S10);
An object detection and tracking step (S20) of detecting the core skeleton as an object based on the image acquired through the camera 10, recognizing the location of the object, and displaying a movement path;
A first action recognition step (S30) of identifying the phase relationship between each object identified through the object detection and tracking step, and proceeding with human action recognition based on this;
After passing through the first action recognition step, a second action recognition step (S40) of recognizing whether or not a distress signal is present through the amount of angle change between the objects;
It includes a step (S50) of transmitting the action recognition result derived through the second action recognition step to a manager's terminal,
Recognizing whether the distress signal is
A process of detecting a core object, which is a core skeleton of the human body;
A process of figuring out a phase relationship between objects to check whether the positions of both hands are above based on the center point below the face area;
It consists of a process of recognizing a rescue signal through the angle change between objects, which is judged based on the number of times that the change in angle between the hands and the center point under the face area and the hand occurs at or above the angle threshold during a specified time threshold. ;,
The dangerous situation recognition unit 22
object detection and tracking; primary action recognition; and secondary action recognition; Through the process, the risk situation including whether to intrude into the dangerous area, whether to work in a group of two people, whether to fall down, whether to wear a helmet, or to recognize a rescue signal is analyzed, and the analyzed result is notified to the manager's terminal through the transmission unit 23 In the safety management system through image-based core skeleton detection that provides,

Whether or not the fall is determined by comprehensively determining the case where the human detection area is shortened in the Y axis and lengthened in the X axis and the center coordinates of the head detection area rapidly change, and the behavior recognition is performed, and before the fall detection, the red color above the person detection area is determined. A circle is displayed, and after detection of a fall, a green circle is displayed above the human detection area so that analysis of a dangerous situation such as a fall at a power plant can be quickly determined to respond to the judgment and respond to an emergency. Image characterized in that Safety management system through core skeleton detection.






delete delete

Images