KR20230139019A - System for identitifying the wearing of worker's personal protective equipment and worker's face based on deep learning - Google Patents

Abstract

The present invention applies computer vision image processing and deep learning algorithms based on image data to unmannedly check whether workers at industrial sites are wearing personal protective equipment, and at the same time, identifies workers based on deep learning through facial recognition. It relates to a system for wearing protective gear and facial identification, which includes an imaging unit that acquires video data taken of workers, and a model that learns each model to recognize the object of the personal protective equipment worn by the worker and the worker's face from the video data. By applying the optimal model algorithm learned through the learning unit and model learning unit, the worker's personal protective equipment and face are recognized from actual image data and compared with the pre-registered data set to identify whether personal protective equipment is worn and the worker's identity, respectively. It may include a personal protective equipment wearing identification unit and a worker identification unit.

Description

Deep learning-based worker personal protective equipment wearing and facial identification system {SYSTEM FOR IDENTITIFYING THE WEARING OF WORKER'S PERSONAL PROTECTIVE EQUIPMENT AND WORKER'S FACE BASED ON DEEP LEARNING}

The present invention relates to a safety management system at industrial sites. It applies computer vision image processing and deep learning algorithms based on image data to check whether workers at industrial sites are wearing personal protective equipment unmanned, and at the same time, uses facial recognition to determine whether workers are wearing personal protective equipment. It is about a deep learning-based worker wearing personal protective equipment and facial identification system that can verify identity.

In today's industrial sites with complex and unsafe processes, such as large-scale construction and manufacturing, workers' work activities are always exposed to many risks anytime and anywhere. According to the results of analysis of disaster types at domestic industrial sites, 37.3% of all deaths at construction sites occurred without wearing a safety helmet, which is one of the personal protective equipment.

As a system to prevent accidents in these industrial sites, a video monitoring control system using CCTV (Closed Circuit TeleVision) is being applied to determine the work situation of workers in real time. However, these CCTV surveillance systems and safety management systems have a problem in that they are not suitable for prevention due to limited human control capabilities.

Therefore, there is a demand for smart safety management using cutting-edge technology that can identify workers' situations in industrial sites in real time and immediately and safely manage them. Instead of manual safety management by humans, methods using artificial intelligence computer vision and deep learning-based image recognition technology, which have recently been in the spotlight, are being attempted.

The present applicant proposed the present invention to solve the above problems.

Korean Patent No. 10-2147052 (registration date 2020.08.17.) Korean Patent Publication No. 10-2017-0050465 (publication date 2017.05.11.) Korean Patent No. 10-2294574 (registration date 2021.08.23.) Korean Patent No. 10-2263154 (registration date 2021.06.03.) Korean Patent No. 10-1510798 (registration date 2015.04.03.)

The present invention was proposed to solve the above problems, and applies computer vision image processing and deep learning algorithms based on image data to recognize the personal protective equipment worn by the worker and the worker's face from the worker's image. The purpose of the present invention is to provide a deep learning-based worker personal protective equipment wearing and facial identification system that can verify workers' personal protective equipment wearing compliance and identity unmanned rather than through manual safety management by humans.

In order to achieve the above-mentioned tasks, a deep learning-based worker personal protective equipment wearing and facial identification system according to an embodiment of the present invention includes a photographing unit for acquiring image data of workers; A model learning unit that learns each model for recognizing the object of the personal protective equipment worn by the worker and the worker's face from the image data and then infers an optimal model algorithm; A personal protective equipment wearing identification unit that detects the worker's personal protective equipment from actual image data by applying the model algorithm inferred through the model learning unit and identifies whether the personal protective equipment is worn by comparing it with a data set of pre-registered personal protective equipment; And it may include a worker identity identification unit that recognizes the worker's face from actual image data by applying the model algorithm inferred through the model learning unit and identifies the worker's identity by comparing it with a pre-registered worker's identity data set.

Specifically, the model learning unit includes a personal protective equipment detection unit that detects objects of personal protective equipment from the image data using a YOLO (You Only Look Once) model utilizing deep learning; A face recognition unit that recognizes facial features from the image data using a MTCNN (Multi Tasking Convolutional Neural Networks) model using deep learning and a FaceNet model; It may include a model verification unit that adjusts parameters to minimize learning loss through performance analysis of the YOLO model, MTCNN model, and FaceNet model.

In addition, the personal protective equipment detection unit includes a backbone that extracts a feature map from the image data and classifies the class according to whether the personal protective equipment is worn or not worn, and an object of the personal protective equipment based on the extracted feature map. A convolution anchor box that creates an anchor box in an area expected to be located and sets a final bounding box within the anchor box based on the class information, and It may be implemented by including a detection unit that detects the object of the personal protective equipment in a set bounding box.

The face recognition unit includes a configuration for extracting and cropping a face area from the image data using the MTCNN model, and a configuration for extracting and embedding facial features from the cropped image using the FaceNet model; , may be implemented including a configuration for measuring the distance between the embedded vectors to identify the identity of the face.

The model verification unit analyzes the performance of the values output from the YOLO model for detecting personal protective equipment and the FaceNet model for face recognition, and can apply the parameters by adjusting the epoch and batch size of each model.

According to the present invention, it is possible to check whether a worker is wearing personal protective equipment and his/her identity from video data captured at an industrial site, eliminating the need for manual human safety management, and preventing worker accidents in an unmanned manner on a regular, real-time, and preventive basis. and has a significant effect in reducing disasters.

In this way, the present invention can be applied not only to the industrial safety field, but also to the access control and security management fields, thereby expanding its utility.

Figure 1 is a configuration diagram showing a deep learning-based worker personal protective equipment wearing and facial identification system according to an embodiment of the present invention.
Figure 2 is a detailed configuration diagram of a model learning unit in a system according to an embodiment of the present invention.
Figure 3 is a diagram for explaining the process of learning a model in a system according to an embodiment of the present invention.
Figure 4 is a diagram showing an example of the results of labeling a data set.
Figure 5 is a graph showing optimal performance output under optimal parameter conditions for the YOLOv5s model applied to an embodiment of the present invention.
Figure 6 is a diagram showing the detection results of workers' heads and personal protective equipment at industrial sites under various conditions.

The advantages and/or features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various different forms, but the present embodiments only serve to ensure that the disclosure of the present invention is complete and are within the scope of common knowledge in the technical field to which the present invention pertains. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

In addition, the preferred embodiments of the present invention to be implemented below are provided in each system function configuration in order to efficiently explain the technical components constituting the present invention, or system functions commonly provided in the technical field to which the present invention pertains. The configuration will be omitted whenever possible, and the description will focus on the functional configuration that must be additionally provided for the present invention. If a person has ordinary knowledge in the technical field to which the present invention pertains, he or she will be able to easily understand the functions of conventionally used components among the functional configurations not shown and omitted below, as well as the omitted configurations as described above. The relationships between elements and components added for the present invention will also be clearly understood.

In addition, in the following description, "transmission", "communication", "transmission", "reception" and other similar terms of signals or information refer to the direct transmission of signals or information from one component to another component. In addition, it also includes those transmitted through other components. In particular, “transmitting” or “transmitting” a signal or information as a component indicates the final destination of the signal or information and does not mean the direct destination. This is the same for “receiving” signals or information.

Figure 1 is a configuration diagram showing a deep learning-based worker personal protective equipment wearing and facial identification system according to an embodiment of the present invention, Figure 2 is a detailed configuration diagram of the model learning unit in the system according to an embodiment of the present invention, and Figure 3 is the present invention. A diagram for explaining the process of learning a model in a system according to an embodiment of the invention. Figure 4 is a diagram showing an example of the result of labeling a data set. Figure 5 is an optimal diagram for the YOLOv5s model applied to an embodiment of the invention. 6 is a graph showing the optimal performance output under parameter conditions, and FIG. 6 is a diagram showing the detection results of workers' heads and personal protective equipment at industrial sites under various conditions.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a configuration diagram showing a deep learning-based worker personal protective equipment wearing and facial identification system 100 according to an embodiment of the present invention.

Referring to FIG. 1, the deep learning-based worker personal protective equipment wearing and facial identification system 100 (hereinafter referred to as “system”) according to an embodiment of the present invention largely includes an imaging unit 101, a model learning unit 140, It may include a personal protective equipment wearing identification unit 150 and a worker identification unit 160.

The photographing unit 101 acquires images or videos (hereinafter referred to as video data) of people working at an industrial site (hereinafter referred to as workers). Image data can be acquired through an image sensor, which may be a charge-coupled device (CCD) camera, an infrared (IR) camera, a gray scale image converter, or a radiometric temperature ( radiometric temperature) may include any one of the image converters.

The model learning unit 140 may perform a process of learning each model for recognizing two objects from image data acquired by the photographing unit 101. The first learns a model to detect objects in the personal protective equipment worn by workers for safety purposes, and the second learns a face recognition model to confirm the worker's identity.

Here, personal protective equipment (PPE) may include a safety helmet, safety ring, safety shoes, etc. For example, in an embodiment of the present invention, a helmet was selected as personal protective equipment. In order to properly detect personal protective equipment objects, accuracy as well as detection speed is required for various industrial environmental situations such as small, distant, dark places, many objects, and conditions on a person's head.

In an embodiment of the present invention, in order to detect personal protective equipment objects, after experimenting with learning and verification of various conditions such as batch size and epoch for the YOLOv5s model, the optimal model for detecting personal protective equipment objects is inferred and predicted. Do an experiment. The FaceNet model, which has a good recognition rate, can be applied as a model for worker identity verification.

The personal protective equipment wearing identification unit 150 applies the optimal model algorithm deduced through the model learning unit 140 to detect the worker's personal protective equipment from real image data (real input data) and detects the worker's personal protective equipment from the pre-registered personal protective equipment data set. By comparing with (data set), it is possible to identify whether personal protective equipment is worn. At this time, the data set for personal protective equipment can be data on industrial safety helmets shared on Kaggle or any server.

The worker identity identification unit 160 applies the optimal model algorithm deduced through the model learning unit 140 to recognize the worker's face from real input data, and uses the pre-registered worker's identity data set and By comparison, the identity of the worker can be identified.

By configuring this system, by using the system 100 according to an embodiment of the present invention, the identity of the worker and whether or not he or she is wearing personal protective equipment can be detected in real time unmanned at an industrial site, thereby reducing worker accidents and disasters. It can provide smart safety management methods.

In order to efficiently provide this, the system 100 of the present invention requires a model algorithm that can more accurately and precisely detect personal protective equipment and faces from images, and learning this is necessary.

Figure 2 is a detailed configuration diagram of the model learning unit 140 in the system 100 according to an embodiment of the present invention.

Looking at FIG. 2 with reference to the configuration of FIG. 1, the model learning unit 140 of FIG. 1 basically uses a YOLO (You Only Look Once) model using deep learning to learn personal protective equipment from image data. A personal protective equipment detection unit (110 in FIG. 1) that detects objects, and a face recognition unit that recognizes facial features from image data using a MTCNN (Multi Tasking Convolutional Neural Networks) model using deep learning and a FaceNet model. It may include (120 in FIG. 1). Among them, the personal protective equipment detection unit 110 includes a backbone object detection unit 111, a convolution anchor box 112, and a personal protective equipment (PPE) prediction unit 113. It can be implemented including:

The backbone object detection unit 111 can extract a feature map from image data and classify the class into cases of wearing and not wearing personal protective equipment.

The convolutional anchor box 112 creates an anchor box in the area where the object of personal protective equipment is expected to be located (predicted) based on the extracted feature map and creates a final anchor box based on class information within this anchor box. You can set an appropriate bounding box.

The convolution anchor box 112 can use or implement PANet: BottleneckCSP.

The personal protective equipment prediction unit 113 can detect objects of personal protective equipment (Personal Protective Equipment) in the set boundary box.

The personal protective equipment detection unit 110 can perform object detection model training using the YOLOv5s model.

In other words, the personal protective equipment detection unit 110 can use CSP-Darknet as the backbone 111 to extract feature maps from images using the YOLOv5s model structure. The head finds the location of the object based on the extracted feature map. You can initially set an anchor box and use it to create the final bounding box. At this time, transfer learning techniques can be applied to improve learning speed and accuracy.

An experiment was conducted with different epoch and batch size settings for the YOLOv5s model. [Table 1] below shows the optimal performance of mAP at epoch 200 due to differences in epoch settings. [Table 2] shows the difference depending on the batch size, showing the optimal performance of mAP when the batch size is 32. There was no significant change in FPS.

The face recognition unit 120 can perform face recognition model training using the FaceNet model.

Referring to FIG. 2, the face recognition unit 120 includes a face detection unit (121), a face detection unit (121), an MTCNN face cropping unit (122), a feature extraction unit (123), and a face recognition SVM (124). , Face Recognition SVM) may be included.

The face recognition unit 120 uses the FaceNet model's face recognition and SVM (Support Vector Machine) classifier to identify people in the input image as follows. In the configuration of the face detection unit 121 and the MTCNN face cropping unit 122, the face area is extracted from the image as input data through MTCNN (Multi Tasking Convolutional Neural Networks) and cropped to a certain size (e.g., 160 × 160). You can go through the normalization process.

In the configuration of the feature extraction unit 123, facial features can be extracted from the cropped image using the FaceNet model and expressed as a 128 element vector called face embedding. In addition, in the configuration of the face recognition SVM 124, it is possible to determine whether a person is the person or not by comparing distances between embedding vectors using the SVM model.

Meanwhile, the model verification unit (130 in FIG. 1) of the model learning unit 140 of the system 100 according to an embodiment of the present invention can adjust parameters to minimize learning loss through performance analysis of each model. . For example, to detect personal protective equipment, the performance of the values output from the YOLO model applied in the personal protective equipment detection unit 110 and the FaceNet model applied in the face recognition unit 120 are analyzed, and the parameters of each model, such as epoch and placement, are analyzed. Size can be changed and compensated. This will be explained in detail in the model experiment and result analysis below.

Figure 3 is a diagram for explaining the process of learning a model in the system 100 according to an embodiment of the present invention.

The system 100 according to an embodiment of the present invention can input a data set image containing workers at an industrial site to detect and classify the worker's personal protective equipment and derive a facial recognition identity verification model.

Figure 3 is a diagram illustrating an experiment process using the system 100 according to an embodiment of the present invention, including a system environment setup step (S1) and a transfer learning step for face recognition and personal protective equipment (S2, YOLOv5s). and FaceNet modification for transfer learning), face and personal protective equipment data set preparation stage (S3), model selection stage (S4), learning/verification/testing stage (S5), identity verification stage (S6, Combine ResultSet), and performance evaluation analysis. It shows the detailed process for the step (S7, Evaluation).

The face and personal protective equipment data set preparation step (S3) may include a step of preparing a face data set (DataSet (Face) Prepare) and a step of preparing a personal protective equipment data set (DataSet (PPE) Prepare). Here, the steps for preparing the face data set include collecting data from Kaggle (Collection from kaggle), detecting faces using MTCNN (Face Detection (MTCNN)), and preparing face cropping (Face Cropping Preparation) may be included.

In addition, the steps to prepare the personal protective equipment data set include collecting personal protective equipment data from Kaggle (Collection from kaggle (PPE)), image annotation using a manual tool (Image Annotation (Manual Tool)), and learning validation test set. It may include a preparation stage (Train-Validation-Test Set Preparation).

In addition, the model selection step (S4) includes the FaceNet model selection step (Model Selection (FaceNet)) performed after the step of preparing the face data set, and the Yolo model selection step performed after the step of preparing the personal protective equipment data set. May include (Model Selection(Yolo)).

The learning/verification/testing step (S5) is a learning modeling step (Training Model), a test set validation step (Validation on test set), and a video feed performed after the FaceNet model selection step (Model Selection (FaceNet)). It may include a validation step (Validation on Video Feed) and a face identification step (Face ID Identification).

In addition, the learning/verification/testing step (S5) is a learning modeling step (Training Model), validation on test set, and video feed performed after the Yolo model selection step (Model Selection (Yolo)). It may include a validation step (Validation on Video Feed) and an object detection step (Object Detection).

As mentioned above, from a deep learning perspective, the image classification problem can be approached through transfer learning. To do this, first delete the classifier in the original model, add a new classifier suitable for the purpose of the present invention, and then fine-tune the developed model. In other words, you can fine-tune the YOLOv5s model, FaceNet model, and SVM model.

The learning data set for personal protective equipment detection and face recognition can be mixed and used by constructing the industrial site hard hat data set released by Kaggle and images captured by cameras at industrial sites.

In addition, the YOLOv5s model can be used to learn and verify model learning and performance for worker personal protective equipment detection and identity verification by face recognition, and the MTCNN model can be used to detect faces in input images. As preprocessing, only the face part is extracted from the input image, the face recognition process is performed using the FaceNet model, and the identity of the pre-registered worker can be confirmed using the SVM classifier.

* Experiment and result analysis

The environment and data set for the experiment, the indicators used to evaluate the performance of the model algorithm for object detection and face recognition, the results of the personal protective equipment detection and classification experiment, and the results of the face recognition classification experiment are described.

The experimental environment is as follows.

OS (Ubuntu 18.04.5 LTS), GPU (Tesla V100-SXM2), CPU (16-Core Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz), RAM (177GB), CUDA (10.1), cuDNN( 7.6.0), software (python/pytorch), and Deep Learning model (YOLOv5s) were used.

The data set is as follows.

The data set for learning hard hat detection was a mixture of industrial field hard hat data released by Kaggle and images captured by cameras at industrial sites. Kaggle's data set consists of 15,887 learning data, 4,641 validation data, and 2,261 test data. Classes were labeled as head and helmet. For example, class 0 was labeled as head and class 1 as helmet. For each class, there are approximately 84,000 heads and approximately 41,000 helmets. Workers not wearing safety helmets are indicated as heads, and workers wearing safety helmets are indicated as helmets.

It is advisable to set the ground truth area as a bounding box for the head area including the human head or hat to avoid detection when only the hard hat itself is present.

As shown in Figure 4, the class identification value of the detected object, the color of each class for the bounding box, the position coordinate values of the object (X, Y), and the area size of the bounding box (W, H) are expressed. do. In the information file (Helm_0003334.txt), the top line is information for Class 1 (helmet), and the bottom line is information for Class 0 (human head).

In addition, video data on personal protective equipment at domestic industrial sites required for the experiment of the present invention were captured with a smartphone camera to construct a data set. The data set for face recognition learning was a mixture of data sets captured by cameras of workers wearing and not wearing safety helmets at industrial sites. [Table 3] shows the status of data sets for face recognition.

The evaluation indicators are as follows.

Quantitative indicators used to evaluate the performance of model algorithms for object detection and face recognition are evaluated through confusion matrix, PR (Precision-Recall) graph, average precision, harmonic mean (F1 score), etc. do.

Next, the results of the personal protective equipment detection experiment are described.

As an experiment for personal protective equipment detection, experiments were compared for the YOLOv5s model by varying parameters with a batch size of 64 and epochs of 200, 500, 1000, and 2000. As a result of detecting the worker's head and personal protective equipment in the 640x640 input image, the head detection rate is 0.937 to 0.943, the hard hat detection rate is 0.968 to 0.974, and the mAP is 0.953 to 0.964.

Figure 5 shows a PR graph for epoch 200 at a batch size of 64. In the verification result graph of batch size 64, there was no overfitting at epoch 200, but overfitting was seen at epochs 500, 1000, and 2000.

Figure 6 shows the prediction experiment results of batch 64 and epoch 200 of the YOLOv5s model. This is an example showing the results of detection of workers' heads and personal protective equipment in industrial sites under various situations and conditions. Object detection is easy even if there are many or small objects, and through learning, desired objects can be detected even if the industrial site is small and dark.

[Table 4] below compares model performance by performing several batch experiments by changing the batch size to 32, 64, and 128 based on epoch 200. The size of the input image is defined as 640x640, and the mAP, head, and helmet detection rates are highest when the batch size is 32.

The selected model experiment results were tested for various batch sizes and epochs, and as a result, batch 32 and epoch 200 showed the optimal detection rate for detecting workers' heads and personal protective equipment, so the system 100 of the present invention meets the above conditions. The model can be selected as the optimal model. As a result of verification and testing by applying the selected model, it is possible to detect and recognize workers' heads and personal protective equipment in various industrial field situations and conditions.

Next, the results of the face recognition classification experiment are as follows.

As previously explained, the MTCNN model is applied for face detection for identity verification, and the FaceNet model can be used for facial features. The process of identity verification is to store the image through a normalization process at a size of 160

The data set for face recognition learning consisted of 9 classes created by selecting the same person, and 9 classes created by 9 domestic workers of the same person taken with a smartphone, making a total of 18 classes. [Table 5] below is a data set of 203 for training, 44 for verification, and a total of 247 for 9 classes composed of smartphone captured videos.

Face detection and cropping are preprocessing processes for recognizing and classifying faces in the FaceNet model, and MTCNN can be used to detect objects and crop only the face part. Next, facial features detected by MTCNN can be used as input to the FaceNet model to express facial features as an embedding vector. The embedding vector can be used to determine whether the person is the person or not based on the distance from the embedding vector of the image to be checked for identity verification. Therefore, in order to find the optimal distance threshold for identity verification, we created a total of 100 face pairs, 50 pairs of the same person and 50 pairs of different people, and randomly selected 30 of them to detect faces and use FaceNet. After embedding the facial features detected using a learning model, an experiment was conducted to compare the embedding vectors at different distances. The accuracy of this experiment was calculated as 0.07, precision as 0.73, recall as 0.65, and harmonic mean as 0.69.

Looking at the face recognition values detected from distances 0.5 to 2.9 for epochs 50, 100, and 200 of images of 30 pairs of faces, when comparing distances using embedding vectors regardless of epoch, it is better to use a distance of 1.1 to 1.5. Since 21 items were detected in common at 1.4, the result was obtained that it is better to set a model to compare embedding vectors with the distance threshold set to 1.4.

To verify identity, an SVM classifier can be trained by inputting the embedding vector. The face image detected by MTCNN is input to FaceNet to generate a 128-dimensional embedding vector representing facial features and confirm the SVM identity. To derive the optimal SVM model, experiments were conducted on epochs 50 and 100, and it was found that accuracy, recall, and harmonic mean performance results were better at epoch 100.

When using the distance threshold of the embedding vector, a model with an epoch of 100 and a distance threshold of 1.4 was selected as the optimal model, and a model learned with an epoch of 100 using SVM was selected as the optimal model. When comparing the two results, it can be seen that the results of SVM show better performance.

Therefore, in the system 100 of the present invention, FaceNet was trained with epoch 100 and then used as input to SVM, and the model trained with epoch 100 was selected as the optimal model. As a result of an experiment with learning data of 110 faces of workers wearing hard hats captured with a camera, the accuracy was 0.98, precision was 1.0, and recall was 0.92.

As such, the system 100 proposed in the present invention has differences in the method of detecting personal protective equipment and the method of applying the object detection model compared to existing studies. Referring to [Table 6] below, it shows the difference from existing studies.

As a result, the system of the present invention selected the optimal model through experiments for each epoch and batch size for the YOLOv5s model to detect personal protective equipment, and derived and analyzed results for the selected model through a learning and verification process. First, in order to select the optimal epoch, the batch size was defined as 64 and through epoch experiments varying the epochs to 200, 500, 1000, and 2000, the optimal epoch was found to be 200 based on an Intersection over Union (IoU) of 0.5. This could be confirmed with the model. Second, as a result of detection through batch experiments on batches 32,64,128 at epoch 200, optimal performance was shown with mAP 0.959 when the batch size was 32 at epoch 200. Third, verification and testing of object detection and recognition in images was performed on the YOLOv5s model, which has the fastest real-time object detection, and it was confirmed that it recognized worker heads and personal protective equipment well in various industrial field environments.

Meanwhile, step-by-step experiments were conducted using MTCNN face detection and FaceNet face recognition methods to confirm worker identity. In the first step, the face was detected from the input image, normalized to a size of 160 Identity was confirmed using the extracted features as input to SVM. In three steps, the identity was confirmed using the embedding vector distance.

In an embodiment of the present invention, as a result of learning SVM on face pairs of 18 people (Class 18) of photographed data, experiments were conducted with epochs 50 and 100, and the differences between epochs 50 and 100 were similar in overall accuracy, precision, and recall. The harmonic average of the epoch 100 condition seemed to be a rather good result. Additionally, FaceNet was trained with epoch 100 and then used as input to SVM, and the model trained with epoch 100 was selected as the optimal model.

In this way, the YOLOv5s model and FaceNet model, which showed optimal performance through experiments under various conditions, such as a comparison experiment according to the batch size and epoch conditions of the YOLOv5 4 model, and an experiment comparing the accuracy of the face embedding vector value according to the epoch 50, 100, and 200 conditions, were compared. Derived. In the derived optimal model, it was confirmed whether the worker was wearing or not wearing personal protective equipment and his/her identity through facial recognition.

The system or device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and thus stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The above description is merely an exemplary description of the present invention, and various modifications may be made by those skilled in the art without departing from the technical spirit of the present invention. Accordingly, the embodiments disclosed in the specification of the present invention do not limit the present invention. The scope of the present invention should be interpreted in accordance with the claims below, and various equivalents and modifications that can replace them at the time of filing this application should be construed as being included in the scope of the present invention.

100: Deep learning-based worker personal protective equipment wearing and facial identification system
101: imaging unit 110: personal protective equipment detection unit
120: face recognition unit 130: model verification unit
140: Model learning unit 150: Personal protective equipment wearing identification unit
160: Worker identification unit

Claims (5)

A photography unit that acquires video data captured by workers;
A model learning unit that learns each model for recognizing the object of the personal protective equipment worn by the worker and the worker's face from the image data and then infers an optimal model algorithm;
A personal protective equipment wearing identification unit that detects the worker's personal protective equipment from actual image data by applying the model algorithm inferred through the model learning unit and identifies whether the personal protective equipment is worn by comparing it with a data set of pre-registered personal protective equipment; and
A worker identity identification unit that recognizes the worker's face from actual image data by applying the model algorithm inferred through the model learning unit and identifies the worker's identity by comparing it with a pre-registered worker's identity data set;
Deep learning-based worker personal protective equipment wearing and facial identification system including.
According to paragraph 1,
The model learning unit,
A personal protective equipment detection unit that detects objects of personal protective equipment from the image data using a YOLO (You Only Look Once) model using deep learning;
A face recognition unit that recognizes facial features from the image data using a MTCNN (Multi Tasking Convolutional Neural Networks) model using deep learning and a FaceNet model;
A model verification unit that adjusts parameters to minimize learning loss through performance analysis of the YOLO model, MTCNN model, and FaceNet model;
A deep learning-based worker personal protective equipment wearing and facial identification system comprising a.
According to paragraph 2,
The personal protective equipment detection unit,
A backbone that extracts feature maps from the image data and classifies classes according to cases of wearing and not wearing personal protective equipment,
Based on the extracted feature map, an anchor box is created in the area where the object of the personal protective equipment is expected to be located, and a final bounding box is created within the anchor box based on the class information. A convolution anchor box to set, and
A detection unit that detects the object of the personal protective equipment in the set boundary box
A deep learning-based worker personal protective equipment wearing and facial identification system comprising a.
According to paragraph 2,
The face recognition unit,
A configuration for extracting and cropping a face area from the image data using the MTCNN model,
A configuration for extracting and embedding facial features from the cropped image using the FaceNet model,
Configuration for measuring the distance between the embedded vectors to identify the identity of the face
A deep learning-based worker personal protective equipment wearing and facial identification system comprising a.
According to clause 3 or 4,
The model verification unit,
Deep learning-based worker personalization, characterized in that the performance of the values output from the YOLO model for personal protective equipment detection and the FaceNet model for face recognition are analyzed and the parameters of each model, such as epoch and batch size, are adjusted and applied. Protective equipment wearing and facial identification system.