KR102435581B1 - Attendance check system using face recognition and attendance check method using same - Google Patents

Abstract

The present invention includes a database unit, an image detection unit that detects a student's attendance face image and an input face image from the database unit, an image generation unit that generates a face image assuming a mask is worn, and an attendance analysis unit It is possible to provide an attendance check system using face recognition, characterized in that it is configured to include, and an attendance check method using the same. According to the above description, it is possible to increase the accuracy of face recognition for a person wearing a mask.

Description

Attendance check system using face recognition and attendance check method using same}

The present invention relates to an attendance check system using face recognition and a method for checking attendance using the same, and more particularly, face recognition that can improve the accuracy of face recognition even when wearing a mask as well as not wearing a mask. It relates to an attendance check system using the same and an attendance check method using the same.

In general, face recognition technology has the advantage of being simple and natural for a user to recognize, compared to other biometric recognition technologies such as iris recognition or vein recognition, and thus has been spotlighted as a major research subject.

As an example of such a face recognition technology, Republic of Korea Patent Publication No. 10-2020-0029659 discloses the steps of: receiving a video image of a subject's face to be identified; normalizing the video image; inputting a video image to a convolutional neural network (CNN) trained to extract a plurality of facial feature points, and deriving a feature map including facial feature points in the video image; applying global average pooling (GAP) to the feature map, and outputting global appearance features expressing the appearance features of the entire face of the subject in the video image; inputting a feature map to a pairwise related network (PRN) and extracting a plurality of local appearance features to form a relational pair; and extracting a relational local feature by embedding an identity identification feature in a relational pair.

On the other hand, recently, with the development of deep learning, an automatic attendance check system based on face recognition is being studied. However, since this face recognition technology is a technology for identifying a subject of a detected face by detecting a face region from a photographed image or a video image, the subject's face can be easily deformed by changes or occlusion of lighting, pose, and expression. It is difficult to determine that the pre-registered user and the subject are the same person based on the face region extracted from the photographed image or the video image.

Moreover, recently, many people are wearing masks due to corona, etc., and in this case, there is a difficulty in automatic attendance check based on face recognition because the problem of occlusion of the lower part of the face occurs frequently.

Republic of Korea Patent Publication No. 10-2020-0029659

An object of the present invention is to provide an attendance check system using face recognition that can not only increase the accuracy of face recognition for a person wearing a mask, but also solve the occlusion problem caused by wearing a mask, and an attendance check method using the same do.

According to one aspect of the present invention, the present invention, the attendance book image for the lecture, and a database unit in which the classroom images taken of the lecture are stored; an image detection unit for extracting faces from the attendance book image for the corresponding lecture from the database to detect attendance face images, and extracting faces from the lecture room images to detect input face images; In the presence face image and input face image detected by the image detection unit, a baseline is set on the horizontal line based on the nose, and the face image assuming that the mask is worn by partially processing the attendance face image or input face image with respect to the reference line. an image generating unit to generate; It is possible to provide an attendance check system through face recognition, characterized in that it comprises a; attendance analysis unit to check the attendance by matching the face image generated by the attendance face image and the image generator.

Here, the image detection unit may be configured to detect an attendance face image and an input face image using the Retinaface model.

In addition, the image generating unit may be configured to generate a face image by partially processing each of the attendance face image and the input face image.

Furthermore, the image generator may be configured to generate an upper face image obtained by extracting only a face image of an upper portion of the face with respect to the reference line, and a processed face image obtained by image-processing the lower area with respect to the reference line, respectively.

Meanwhile, the image generator may be configured to generate an upper face image and a processed face image through the ArcFace model.

The attendance analysis unit converts the attendance face image, the upper face image and processed face image generated from the attendance face image, the input face image, the upper face image and the processed face image generated from the input face image into the target space of R ⁵¹² , respectively. It may be configured to calculate the distance between vectors generated by embedding, and check the presence or absence of the calculated distance by combining and matching.

In detail, the attendance analysis unit may be configured to calculate the cosine distance (Angular/cosine Distance) between the vectors generated by embedding and to check attendance by matching based on the shortest distance.

According to another aspect of the present invention, the present invention, an image storage step of storing the attendance book image for the lecture, and the classroom images taken of the lecture in a database; a face detection step in which an image detection unit extracts a face from the attendance list image of the database for the corresponding lecture to detect attendance face images, and extracts the face from the lecture room image to detect input face images; It is assumed that the image generator sets a baseline on the horizontal line based on the nose in the presence face image and input face image detected by the image detection unit, and partially processes the input face image or attendance face image with respect to the reference line, and wears a mask. an image generating step of generating one face image; Attendance check step of checking attendance by matching the attendance face image and the face image generated by the image generator by the attendance analysis unit; provides a face recognition method using an attendance check system through face recognition, characterized in that it comprises a can do.

Here, in the face detection step, the image detection unit may detect an attendance face image and an input face image using the Retinaface model.

In addition, the image generating step, the image generating unit may be configured to generate a face image by partially processing each of the attendance face image and the input face image.

Also, in the image generating step, the image generating unit may generate an upper face image obtained by extracting only the upper face image of the upper region with respect to the reference line and a processed face image obtained by image processing the lower region with respect to the reference line, respectively.

Furthermore, in the image generating step, the image generating unit may generate an upper face image and a processed face image through the ArcFace model.

On the other hand, in the attendance check step, the attendance analysis unit separates the attendance face image, the upper face image and the processed face image generated from the attendance face image, the input face image, the upper face image and the processed face image generated from the input face image, respectively. It is possible to calculate the distance between vectors generated by embedding into the target space of R ⁵¹² , and check the attendance by combining and matching the calculated distance.

Here, in the attendance check step, the attendance analysis unit can check the attendance by calculating the cosine distance (Angular/cosine Distance) between the vectors generated by embedding and matching based on the shortest distance.

1 is an attendance check system through face recognition according to an embodiment of the present invention;
1 is a block diagram showing the configuration of an attendance check system through face recognition according to an embodiment of the present invention.
2 is a view showing an embodiment of the attendance check device in the case of recognizing the faces of students entering the classroom in the attendance check system through face recognition according to an embodiment of the present invention.
3 is a view showing an embodiment of an attendance check device in the case of recognizing the faces of students in a classroom in the attendance check system through face recognition according to an embodiment of the present invention.
4 is a diagram illustrating a case in which the image generator generates a face image from an attendance face image or an input image in the attendance check system through face recognition according to an embodiment of the present invention.
5 is a diagram illustrating a case in which the image generation unit generates a processed face image in the attendance check system through face recognition according to an embodiment of the present invention.
6 is a diagram illustrating a case in which the image generator generates an upper face image in the attendance check system through face recognition according to an embodiment of the present invention.
7 is a view for explaining a process of an attendance check method using the attendance check system through face recognition according to an embodiment of the present invention.
8 is a flowchart illustrating an attendance check method when an input face image is an image representing a state wearing a mask in the attendance check method using the attendance check system through face recognition according to an embodiment of the present invention.
9 shows a photo of an experiment in a classroom environment using the attendance check method using the attendance check system through face recognition according to an embodiment of the present invention.

Hereinafter, the terms used in the present invention have been selected as widely used general terms as possible while considering the functions in the present invention, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term. In the entire specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, terms such as "unit", "module", and "means" described in the specification mean a unit that processes at least one function or operation, which is implemented in hardware or software, or is a combination of hardware and software. can be implemented.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, similar reference numerals are attached to similar parts throughout the specification, and the same reference numbers used throughout the present specification refer to the same components. indicates. And, in the description of the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, the attendance check system (500, hereinafter referred to as the 'attendance check system') through face recognition according to an embodiment of the present invention is a company that requires individual access management, including the attendance check of students receiving lectures at universities, etc. It can be used in various fields such as time and attendance management, etc. Hereinafter, the attendance check system 500 of the present invention will be looked at as an embodiment that is used for the purpose of checking the attendance of students receiving lectures at universities, etc.

Referring to FIG. 1 , the attendance check system 500 according to an embodiment of the present invention includes a database unit 100 , an image detection unit 200 , an image generation unit 300 , and an attendance analysis unit 400 . may be included.

First, the database unit 100 is stored, including the attendance book image, and classroom images. Here, the attendance book image is input from a campus network server, etc., and includes face photos of students receiving a lecture, and is a pre-stored image composed of face images of students who listen to the lecture, and each student's face at this time is a mask from the past Indicates a state not to wear.

Next, the classroom image may be obtained by photographing students entering or leaving the classroom by the attendance check device 20 .

Here, the attendance check device 20 is a communication module (not shown) that transmits the image captured by the image capturing unit 10 including a CCTV or a photographing camera and the image capturing unit 10 to the attendance check system 500 . city) and the like.

On the other hand, the attendance check device 20 may obtain a classroom image in various embodiments.

Referring to FIG. 2 , the attendance check device 20 may be installed at the entrance of the classroom to photograph students entering the classroom, and may be configured to transmit an image obtained by shooting to the attendance check system 500 .

Referring to FIG. 3 as another embodiment, the attendance check device 20 is installed in a lecture room to photograph students receiving a lecture in the lecture room, and to transmit the captured lecture room image to the attendance check system 500. can

That is, the attendance check device 20 may photograph students and the classroom through the image capturing unit 10 installed at the entrance to the lecture room or the lecture room, and through this, it is possible to obtain a lecture room association image.

The image detection unit 200 serves to detect a face from an image and an image for a corresponding lecture from the database unit 100 .

Specifically, the image detection unit 200 may detect attendance face images from the attendance list image of the database unit 100 and detect input face images from the classroom image. At this time, the image detection unit 200 also detects the position of the nose of each corresponding student.

On the other hand, the image detection unit 200 can apply various models to detect the attendance face image and the input face image from the attendance book image and the lecture room image. It is preferable to use a Retinaface model with good detection performance, but do not limit

The image generating unit 300 serves to generate a matching image to determine attendance, and in detail, may be configured to generate an image in consideration of a case in which a target person wears a mask.

Accordingly, the image generating unit 300 sets a baseline on the horizontal line based on the nose in the presence face image or input face image detected by the image detection unit 200, and partially processes the attendance face image or input face image with respect to the reference line. By doing so, it may be configured to generate a face image assuming that a mask is worn.

Specifically, the image generating unit 300 may be configured to generate an upper face image in which only an image of the upper face is extracted with respect to a reference line, and a processed face image in which an image of a lower region is image-processed with white or the like with respect to the reference line, respectively.

4 to 6 are diagrams for explaining an image generated by the image generating unit 300 . First, FIG. 4 shows an attendance face image or an input face image, which is a target image of the image generating unit 300 . FIG. 5 shows a processed face image obtained by image-processing a lower region with respect to a reference line in the image of FIG. 4 . And FIG. 6 shows an upper face image obtained by extracting only the upper face image from the image of FIG. 4 with respect to the reference line.

Meanwhile, the image generating unit 300 may partially process at least one of an attendance face image and an input face image to generate a face image including a processed face image and an upper face image.

Specifically, the image generating unit 300 is preferably configured to generate a new face image by partially processing both the pre-stored attendance face image and the input face image acquired in the classroom with the recent activity image.

That is, the image generating unit 300 generates a processed face image and an upper face image from the attendance face image, and generates a processed face image and an upper face image from the input face image, respectively, as a result, an attendance face image and an attendance face A total of six images may be generated: an upper face image and a processed face image generated from the image, an input face image, and an upper face image and a processed face image generated from the input face image.

On the other hand, at this time, if the input face image acquired in the classroom is in a state of wearing a mask or the resolution is low enough to cause a problem in image processing, the image detection unit 200, the latest input face updated in the database unit 100 Attendance face images can be detected by searching for an input face image in a state not wearing a mask among images or other previously input face images. ), a processed face image and an upper face image can be generated from the input face image detected in .

The image generating unit 300 may generate an image through the ArcFace model in order to generate the above-described input face image, processed face image, and upper face image. Here, the ArcFace model is a face recognition model and a model capable of clear geometrical explanation, which is a well-known model and a detailed description thereof will be omitted.

The attendance analysis unit 400 serves to check the attendance by matching the attendance face image and the face images generated by the image generating unit 300 .

Specifically, the attendance analysis unit 400 is generated from the attendance face image generated by the image generation unit 300, the upper face image and the processed face image generated from the attendance face image, the input face image, and the input face image. It can be configured to calculate the distance between vectors generated by embedding six face images including the processed upper face image and processed face image into the target space of R ⁵¹² , and check attendance by matching based on the calculated distance. have.

In detail, the attendance analysis unit 400 calculates the first distance by calculating the distance between the vectors between the attendance face image and the input face image, and between the vectors between the attendance face image and the upper face image generated from the input face image. The second distance is calculated by calculating the distance, and the third distance is calculated by calculating the distance between the vectors between the attendance face image and the processed face image generated from the input face image. , it may be configured to match the third distance based on this combination. Here, a detailed description of the combination and matching of the first distance, the second distance, and the third distance will be described in a detailed embodiment to be described later.

At this time, the attendance analysis unit 400, by calculating the cosine distance (Angular/cosine Distance) between vectors generated by embedding as a matching criterion, matching based on the shortest distance may be configured to check attendance. Here, the shortest cosine distance between vectors may mean that the degree of similarity is as high as possible, and attendance is checked by matching based on the most similar image.

According to the above, the attendance check system 500 of the present invention includes an attendance face image, a processed face image and an upper face image in consideration of not only the face in a state not wearing a mask, but also a face in a state wearing a mask. Since attendance is checked by matching based on the distance between vectors generated by using face images and embedding these images into the target space, as a result, the matching is performed not only when the mask is not worn but also when the mask is worn. Accuracy can be improved and the occlusion problem that occurred when wearing a conventional mask can be solved.

Hereinafter, an attendance check method (hereinafter referred to as 'attendance check method') using the attendance check system through face recognition according to an embodiment of the present invention will be described. First, since the attendance check method of the present invention uses the above-described attendance check system 500, a detailed description of the configuration of the attendance check system 500 will be omitted. do.

First, referring to FIG. 7 , the attendance check method according to an embodiment of the present invention extracts a face from the attendance book image and the lecture room image stored in the database unit 100 through the image detection unit 200 to obtain an attendance face image and an input face. Each image is detected, and an upper face image and a processed face image are respectively generated by masking the detected attendance face image and input face image based on the reference line. Then, the angular cosine distance is calculated by embedding the attendance face image, the input face image, the upper face image and the processed face image generated from each into the target space, and the distance between the vectors is the shortest. Match attendance to check attendance.

On the other hand, as described above, the image creation target file can be set differently depending on the case in which the target student wears or does not wear a mask in the lecture room image.

Referring to the attendance check method according to the embodiment of the present invention with reference to FIG. 8, the attendance check method includes an image storage step (S110), a face detection step (S120), an image generation step (S130), and an attendance check It may be configured to include step (S140).

First, in the image storage step (S110), it may be configured by storing the attendance book image for the lecture and the classroom images photographed in the classroom in which the lecture is held in the database.

Here, the image storage may be configured such that the attendance book image and classroom images are received and stored from an on-campus server or the like, or the attendance book image and the photographed classroom images are transmitted and updated when a database is configured in the campus server and input. In addition, the database may classify the stored attendance book images and classroom images for each lecture, and may be stored for each setting code.

In the face detection step (S120), the image detection unit 200 detects the attendance face images by extracting the face from the attendance list image of the database for the lecture, and extracting the face from the lecture room image to detect the input face images respectively. can In this case, in the face detection step ( S120 ), the image detection unit 200 may also detect the position of the nose of each corresponding student.

Furthermore, in the face detection step (S120), as described above, the image detection unit 200 may detect an attendance face image and an input face image using the Retinaface model.

On the other hand, such face detection does not detect attendance face images and input face images from attendance list images and classroom images for all lectures stored in the database, but considers processing speed, etc. Attendance face image and input face image can be detected from attendance book images and classroom images.

In the image generation step (S130), a face image is generated assuming that a mask is worn from the presence face image detected by the image detection unit 200 and the input face image, and a reference line on the horizontal line is set based on the nose, A face image is generated by partially processing the input face image with respect to the reference line. Here, the reference line may be set based on the point at which the mask is worn on the nose, and as shown, the reference point may be set at 1/3 point or 1/2 point from the bottom for the entire length of the nose, but is not limited thereto. .

On the other hand, in the image generating step ( S130 ), an upper face image obtained by extracting only the upper face image of the upper region with respect to the reference line and a processed face image obtained by image processing the lower region with respect to the reference line may be generated, respectively, and the image generating unit (300) can be generated using the ArcFace model.

In detail, in the image generation step (S130), a processed face image and an upper face image are generated from the attendance face image, and a processed face image and an upper face image are generated from the input face image, respectively, as a result, the attendance face image and the attendance face A total of six images may be generated: an upper face image and a processed face image generated from the image, an input face image, and an upper face image and a processed face image generated from the input face image.

The attendance check step (S140) may be configured by the attendance analysis unit 400 matching the attendance face image with the face image generated by the image generating unit 300 to check attendance.

Specifically, in the attendance check step (S140), the images generated by the image generation unit 300 are respectively embedded into the target space of R ⁵¹² (S141), and the distance calculated by calculating the distance between the vectors generated by embedding in this way It is possible to check the attendance by matching based on (S142). At this time, the attendance check can check the attendance by calculating the cosine distance (Angular/cosine Distance) between the vectors generated by embedding so that the distance of the smallest value is matched.

In detail, as described above, in the attendance check step (S140), the first distance is calculated by calculating the distance between the vectors between the attendance face image and the input face image as described above, and generated from the attendance face image and the input face image The second distance is calculated by calculating the distance between vectors between The first distance, the second distance, and the third distance may be combined and configured to match based on this.

Hereinafter, a detailed embodiment of the attendance check system through face recognition of the present invention will be described.

The present invention proposes a method for recognizing a face wearing a mask, and is configured to design a deep learning system that detects a face from an image obtained through CCTV in a classroom and automatically checks attendance by comparing it with a face registered in an existing database can be

1.1 Face Detection

Object detection methods using deep learning are classified into CNN-based one-stage and two-stage detection methods. The method of detecting an object is largely composed of localization that finds the location of an object and classification that distinguishes the object. One-stage detection method performs two things simultaneously, and two-stage detection method performs sequentially. The one-stage method lacks accuracy compared to the two-stage method, but is characterized by high speed.

In face detection, a model using a one-stage based multi-task learning strategy called RetinaFace is used because the one-stage method shows sufficient accuracy. In RetinaFace, the Multi-Task Learning strategy not only explores the bounding box to detect a face, but also provides reliability, 5 landmarks marking both ends of the face's eyes, nose, and mouth, 3D location and It is a strategy that simultaneously predicts the corresponding relationship. State-of-the-art performance can be achieved with multiple face detection datasets, such as the WIDER FACE dataset.

1.2 Face Recognition

By learning embeddings that express faces with CNN for face recognition, the distance between vectors embedding the same person's face is short, and the distance between vectors embedding other people's faces is long. ArcFace optimizes Hypershpere's Geodesic Distance Margin by introducing Additive Angular Margin Loss, which improves the existing Loss functions such as Cosine Loss and Triplet Loss. These features enable ArcFace to achieve state-of-the-art performance in several face recognition benchmarks, including large image datasets.

2. The Proposed Scheme

The present invention can provide an automatic attendance check system through a face detection and recognition model.

The structure of the present invention is as shown in FIG. 7, and looking at this, first, a face is detected in the image through RetinaFace, and the face photo and detected face in the existing database, that is, the attendance list, are transferred to the target space of R ⁵¹² through ArcFace. embed By measuring the distance between two vectors generated by embedding as Angular Cosine Distance, the two closest photos are matched with each other. The Angular Cosine Distance between two vectors can be measured by Equation (1).

Equation 1

Also, the matching algorithm is shown in Table 1.

Table 1

는 크기(1,512)의 벡터이다. 출력은 중복값이 제거된 출석 정보를 담은 리스트 S로, 크기는 1×m이다. 먼저 임베딩 벡터들 사이의 유사도를 구해 m×m 크기의 행렬 F를 생성한다. 이 행렬은 모든 경우의 출석을 비교하기 위해 사용된다. 알고리즘은

과 같은 중복값이 발생하지 않을 때까지 모든 행(rows)을 반복하며, 중복값이 발생할 경우 해당 행(row)에 예외처리를 한다.The input to the algorithm is a set of embedded images, each

is a vector of size (1,512). The output is a list S containing attendance information with duplicate values removed, and the size is 1×m. First, the similarity between embedding vectors is calculated to generate a matrix F of size m×m. This matrix is used to compare attendance in all cases. Algorithm is

Repeats all rows until no duplicate values such as .

When the detected face wears a mask, an occlusion problem occurs in which the lower part of the face is covered. In this case, it is difficult to determine the identity between the photo of the attendance book and the detected face. As a way to solve this problem, we propose two image creation methods to be used as input of Arcface. The first method creates a new image by changing the lower part of the horizontal line passing through the nose to white in the bounding box detected through RetinaFace. The same characteristics are extracted from the lower end, and different characteristics are extracted only from the upper end, and the distance centered on the upper end is measured. The second method creates an image that cuts off the lower part of the horizontal line passing through the nose in the bounding box and leaves only the upper part. The distance is measured by extracting the features of only the upper part, excluding the part where the occlusion is predicted. In order to give weight to the features of the non-occlusion area, each distance is measured and combined with the distance of the existing bounding box.

3. Experiments

3.1 Experimental environment

RetinaFace uses ResNet-50 [9] as a backbone and a pre-trained model on the WIDER FACE dataset, and ArcFace uses ResNet-50 as a backbone and a pre-trained model on the MS Celeb 1M dataset.

For experimental data, a new dataset was created with a mask by selecting frontal photos from random 100 classes from the training dataset of The 'Celebrity Together' dataset. The existing database, that is, the data to be used like a photo of the attendance book is D1, the data to be used like the image detected in the CCTV environment is D2, and D3 is the image data created by putting a mask on the image of D2. Experiment 1 is called matching D1 and D2, and Experiment 2 is called matching D1 and D3.

3.2 Optimizing RetinaFace and ArcFace

The relationship between the size of the bounding box of the face detected through RetinaFace and the size of the photo in the database is identified. If you take 5 pictures of the same person at intervals of about 45 cm and calculate the distance between the same people in images with different resolutions, the value is around 0.2 for similarly sized photos, whereas the difference is around 0.4 for large differences in size. . Therefore, in the present invention, the photo of the attendance book is similar to the size of the bounding box measured through CCTV.

3.3 How to measure similarity

In the present invention, the three images for which the distance is to be measured can be named as follows. F stands for the entire face image, W stands for the image in which the lower part of the face image is changed to white (processed face image), and N stands for the upper face image (upper face image). Table 2 shows the results of measuring Rank-1 and matching accuracy by experimenting with embedding three images.

Table 2

In Experiment 1, the W image recorded poor performance compared to the F image, but in Experiment 2 wearing a mask, it recorded better performance. By combining these, a distance combining method as in Equation 2 may be proposed in order to increase the accuracy only when the mask is worn without reducing the accuracy when the mask is not worn.

Equation 2

로 측정될 수 있다.here,

can be measured as

3.4 Experimental results

Table 3 shows the experimental results of measuring Rank-1 and matching accuracy by combining the distance of Equation 2 with three images. The values of α, β, and γ are the same as above even when only two out of three are combined and measured.

Table 3

Looking at this, in the case of Experiment 1 not wearing a mask, when the present invention was applied, the result was not significantly different from that in the case of measuring the distance using only the bounding box in which the face was detected. However, it can be seen that the performance was improved in the case of Experiment 2, in which all masks were worn.

The matching accuracy of 80% Rank-1 and 88% in the F image was improved to 85% Rank-1 and 94% matching accuracy by combining the distances in the F, W, and N images, respectively. The results for 10 people in an actual classroom environment are as follows. Experiment 1 is named as matching the photo of the attendance sheet in an environment without a mask, and Experiment 2 in an environment wearing a mask. On the other hand, FIG. 9 shows a photograph tested in an actual classroom environment, and Table 4 shows Rank-1 and matching accuracy performed in the experimental environment of FIG. 9 .

Table 4

4. Conclusions

The present invention is to provide an automatic attendance check system through a CCTV image, and by using CNN in the image to detect and embed a face, compare it with a picture in an existing database to determine whether or not to attend. And to apply it to the real environment, a lightweight backbone network was used, and it was limited to an environment where a front face could be detected.

In addition, if you wear a mask, you can measure the distance with a vector embedded as an image of the lower part of the face in white and the image of the upper part of the face in order to judge the sameness well, and provide a method of combining with the distance obtained from the entire face image.

Accordingly, it is possible to increase the Rank-1 accuracy by 16% and the matching accuracy by 19% compared to the existing method in a mask-wearing situation while maintaining the accuracy of 100 people without a mask. Therefore, in the present invention, it is possible to erase the occlusion through the same processing method on the part with the occlusion, or measure the distance only for the part where the occlusion does not occur and combine it with the distance measured in the entire image to match other parts of the face with sunglasses, hat, etc. It can be expected to respond well to the occlusion problem caused by occlusion.

Although the present invention has been described with reference to the embodiment shown in the drawings, which is merely exemplary, it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

10: image recording unit
20: attendance check device
100: database unit
200: image detection unit
300: image generator
400: attendance analysis unit
500: attendance check system

Claims (14)

A database unit in which the attendance book image for the lecture and the lecture room images taken in the lecture room are stored;
an image detection unit for extracting a face from the attendance book image for a corresponding lecture from the database to detect attendance face images, and extracting a face from the lecture room image to detect input face images;
In the presence face image and the input face image detected by the image detection unit, a reference line on a horizontal line is set based on the nose, and a mask is worn by partially processing the attendance face image or the input face image with respect to the reference line an image generating unit generating a face image assuming ;
and an attendance analysis unit that matches the attendance face image and the face image generated by the image generator to check attendance.
The image generating unit,
It is configured to generate a face image by partially processing each of the attendance face image and the input face image,
The image generating unit,
an upper face image obtained by extracting only a face image of the upper part of the face in the upper region with respect to the reference line;
It is configured to generate each processed face image in which the image processing of the lower region with respect to the reference line is performed,
The attendance analysis unit,
The attendance face image, the upper face image and the processed face image generated from the attendance face image, the input face image, and the upper face image and the processed face image generated from the input face image are converted into the target space of R ⁵¹² , respectively. Attendance check system through face recognition, characterized in that it is configured to calculate the distance between the vectors generated by embedding, and check the attendance by combining and matching the calculated distance. The method of claim 1,
The image detection unit,
Attendance check system through face recognition, characterized in that it is configured to detect the attendance face image and the input face image using a Retinaface model. delete delete The method of claim 1,
The image generating unit,
Attendance check system through face recognition, characterized in that it is configured to generate the upper face image and the processed face image through the ArcFace model. delete The method of claim 1,
The attendance analysis unit,
Attendance check system through face recognition, characterized in that it is configured to check attendance by calculating the cosine distance (Angular/cosine Distance) between vectors generated by embedding and matching based on the shortest distance. an image storage step of storing the attendance book image for the lecture and the classroom images taken of the lecture room in a database;
a face detection step in which an image detection unit extracts a face from the attendance book image of the database for a corresponding lecture to detect attendance face images, and extracts a face from the lecture room image to detect input face images;
An image generating unit sets a baseline on a horizontal line based on the nose in the face image and the input face image detected by the image detection unit, and partially processes the input face image or the face image with respect to the reference line to form a mask An image generating step of generating a face image assuming a case of wearing;
Attendance check step of checking attendance by matching the attendance face image and the face image generated by the image generator by the attendance analysis unit;
The image creation step is
The image generating unit is configured to generate a face image by partially processing each of the attendance face image and the input face image,
The image creation step is
The image generating unit generates an upper face image in which only a face image of the upper region of the face is extracted with respect to the reference line, and a processed face image obtained by image processing of the lower region with respect to the reference line, respectively;
The attendance check step is,
The attendance analysis unit ^R512 each of the attendance face image, the upper face image and processed face image generated from the attendance face image, the input face image, and the upper face image and processed face image generated from the input face image A face recognition method using an attendance check system through face recognition, characterized in that it calculates the distance between vectors generated by embedding into the target space of , and checks the attendance by combining and matching the calculated distance. 9. The method of claim 8,
The face detection step is
The face recognition method using an attendance check system through face recognition, characterized in that the image detection unit detects the attendance face image and the input face image using a Retinaface model. delete delete 9. The method of claim 8,
The image creation step is
The face recognition method using the attendance check system through face recognition, characterized in that the image generator generates the upper face image and the processed face image through the ArcFace model. delete 9. The method of claim 8,
The attendance check step is,
A face recognition method using an attendance check system through face recognition, characterized in that the attendance analysis unit calculates the cosine distance (Angular/cosine Distance) between the vectors generated by embedding and checks the attendance by matching based on the shortest distance .

Images