KR20220043842A - Edge device for recognizing a face wearing a mask and system for recognizing a face wearing a mask - Google Patents

Abstract

A system for recognizing a face wearing a mask according to one aspect of the present invention comprises: a user face recognition unit for generating a partial feature vector by inputting an input image generated from a user image of a user requested for registration into a partial facial feature model and generating a full feature vector by inputting the input image into a full facial feature model; an array file generation unit for generating an array composed of the partial feature vector, the full feature vector, and user identification information for each user and merging the generated arrays to create an array file; and a face recognition server comprising an interface unit transmitting the partial facial feature model, the full facial feature model, and the generated array file to an edge device. The partial facial feature model extracts a partial facial image including a partial facial region from the input image and generates a first-dimensional partial feature vector based on the extracted partial facial image. The full facial feature model extracts a full facial image including an entire facial region from the input image and generates a second-dimensional full feature vector larger than the first dimension based on the extracted full facial image. Accordingly, the face recognition rate for a person wearing a mask can be improved.

Description

EDGE DEVICE FOR RECOGNIZING A FACE WEARING A MASK AND SYSTEM FOR RECOGNIZING A FACE WEARING A MASK}

The present invention relates to an edge device capable of recognizing a face and a face recognition system.

Face authentication technology is one of the fields of biometrics, which means a technology in which a machine automatically identifies and authenticates a person using the unique feature information contained in each person's face. It has superior security compared to the authentication method, and has been widely used in various fields recently.

A general face authentication system transmits a face image taken from a device installed in an access gate, etc. to the server, and the server performs face recognition and user authentication according to face recognition, and transmits the authentication result to the device to determine whether to open the access gate. .

Although the general face authentication system is the same person, when a face is photographed in a different environment or a face is photographed while wearing a mask, there is a problem in that it cannot distinguish the same person.

The present invention is to solve the above-mentioned problems, and it is a technical task to provide an edge device for mask-wearing face recognition and a mask-wearing face authentication system capable of accurate face recognition even when wearing a mask.

The mask wearing face recognition system according to an aspect of the present invention generates a partial feature vector by inputting an input image generated from a user image of a registered user into a partial facial feature model, and inputs the entire feature vector to the full facial feature model. A user face recognition unit that generates, an array file generation unit that generates an array consisting of the partial feature vector, the entire feature vector, and user identification information for each user and merges the generated arrays to generate an array file, and the partial facial feature and a face recognition server including an interface unit for transmitting a model, the entire facial feature model, and the generated array file to an edge device. The partial facial feature model extracts a partial facial image including a partial facial region from the input image, and generates a first-dimensional partial feature vector based on the extracted partial facial image, and the full facial feature model uses the input image. An entire face image including the entire face region is extracted from the image, and an entire feature vector of a second dimension larger than the first dimension is generated based on the extracted entire face image.

A mask-wearing edge device for face recognition according to another aspect of the present invention includes a mask detection model, a partial facial feature model, a full face feature model, and a user partial feature vector of each of a plurality of users, a user full feature vector, An interface unit for receiving an array file including user identification information, a photographing unit for acquiring a photographed image of a target user to be authenticated, and inputting an input image generated from the photographed image into a mask detection model to detect a mask area, When the mask region is detected, the input image is input to the partial facial feature model to generate a target partial feature vector. If the mask region is not detected, the input image is input to the full facial feature model to obtain a target full feature vector. a target face recognition unit that generates, and compares each of the user partial feature vectors of a plurality of users with the target partial feature vector, or compares each of the user total feature vectors of a plurality of users with the target full feature vector It includes an authentication unit that determines whether to authenticate or not.

According to the present invention, it is possible to train a partial facial feature model based on a face image wearing a mask, and train a full facial feature model based on a face image without a mask. The present invention can improve the facial recognition rate for a person wearing a mask by authenticating a target user using a partial facial feature model and a full facial feature model.

In addition, the present invention detects a face and a mask through one deep neural network, thereby reducing the amount of association caused by face and mask detection and improving the operation speed.

In addition, the present invention can accurately extract a face image by integrating the feature maps generated by the convolution operation units of different layers through a neural network.

1 is a block diagram schematically showing the configuration of a mask wearing face authentication system according to an embodiment of the present invention.
2 is a block diagram schematically showing the configuration of a face recognition server according to an embodiment of the present invention.
FIG. 3 is a block diagram showing the configuration of an entire face image extractor constituting the entire facial feature model of FIG. 2 .
4 is a block diagram showing the configuration of a full feature vector extractor constituting the entire facial feature model of FIG. 2 and a partial feature vector extractor constituting the partial face feature model of FIG. 2 .
FIG. 5 is a block diagram showing the configuration of a first unit included in the face image processing unit of FIG. 4 .
6 is a block diagram illustrating the configuration of a second unit included in the face image processing unit of FIG. 4 .
7 is a block diagram illustrating the configuration of the mask detection model of FIG. 2 .
8 is a view showing an example of coordinates of a whole face region, a coordinate of a mask region, and coordinates of a partial face region.
9 is a view showing an example in which a general face recognition model does not recognize the same person.
10 is a diagram showing an example in which an overlapping area is generated when a learning image is arranged in a vector space according to a distance between the learning images.
11 is a view showing an example of arranging a learning image on a two-dimensional angular plane.
12 is a diagram exemplarily showing that the training images are spaced apart from each other by providing a margin angle between the training images on a two-dimensional angular plane.
13 is a block diagram schematically showing the configuration of an edge device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The meaning of the terms described in this specification should be understood as follows.

The singular expression is to be understood as including the plural expression unless the context clearly defines otherwise, and the terms "first", "second", etc. are used to distinguish one element from another, The scope of rights should not be limited by these terms.

It should be understood that terms such as “comprise” or “have” do not preclude the possibility of addition or existence of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

The term “at least one” should be understood to include all possible combinations from one or more related items. For example, the meaning of “at least one of the first, second, and third items” means 2 of the first, second, and third items as well as each of the first, second, or third items. It means a combination of all items that can be presented from more than one.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of a face recognition system wearing a mask according to an embodiment of the present invention, and FIG. 2 is a block diagram schematically showing the configuration of a face recognition server according to an embodiment of the present invention.

1 and 2 , the mask wearing face authentication system 100 according to an embodiment of the present invention includes a face recognition server 110 and a plurality of edge devices 120 .

The face recognition server 110 generates a face recognition model, and extracts a feature vector from the user's image input from the user terminal 130 using the generated face recognition model. The face recognition server 110 generates an array file for authentication of the target user by using the feature vector. Then, the face recognition server 110 transmits the generated array file to the edge device 120 so that the edge device 120 can authenticate the target user.

To this end, the face recognition server 110 includes a user registration unit 210, an input image generation unit 215, a user face recognition unit 220, face recognition models 225, 230, 240, as shown in FIG. It includes an array file generating unit 250 , a training unit 280 , and an interface unit 270 .

The user registration unit 210 receives one or more user images from the user terminal 130 of the user who wishes to register. When the user image is received, the user registration unit 210 checks whether the user is the same person as the user image, and if it is determined that the user is the same person, obtains access permission information granted to the user and the user database 212 together with the user image. register in

In an embodiment, the user registration unit 210 may receive the user's identification information from the user terminal 130 together with the user image. For example, the user registration unit 210 may receive the user's identification information, such as the user's ID, name, phone number, or the user's employee number, together with the corresponding user image. According to this embodiment, the user registration unit 210 may register the user's identification information and the user's access access information together with the corresponding user image in the user database 212 .

Meanwhile, when receiving a plurality of user images from the user terminal 130 , the user registration unit 210 may induce different user images to be input. For example, the user registration unit 210 may induce the user to input a user image photographed in a different environment, a user image photographed in a different illuminance, or a user image wearing a mask through the user terminal 130 . In this way, the user registration unit 210 can improve the accuracy of face recognition by receiving a plurality of user images taken according to different environments, different illuminance, or whether a mask is worn from a single user.

The input image generating unit 215 generates an input image to be used for face recognition from the user image input by the user registration unit 210 . Specifically, the input image generator 215 generates a plurality of user images having different resolutions from one user image by downsampling or upsampling one user image to a predetermined step. For example, the input image generator 215 may generate a plurality of user images having different resolutions by downsampling one user image.

In one embodiment, the input image generator 215 generates a downsampled user image by applying a Gaussian Pyramid to the user image, or upsampled by applying a Laplacian Pyramid to the user image. You can create custom images.

When a plurality of user images having different resolutions are generated, the input image generator 215 generates, as input images, a plurality of images obtained by moving a window having a predetermined pixel size on the user image for each user image. do. The input image generating unit 215 inputs the generated plurality of input images to the user face recognition unit 220 .

The user face recognition unit 220 is generated by the input image generator 215 to the face recognition model trained by the training unit 280 , specifically, the full facial feature model 230 and the partial facial feature model 240 . Input a plurality of input images.

The user face recognition unit 220 obtains the entire feature vector for the entire face region through the full facial feature model 230 . Also, the user face recognition unit 220 obtains a partial feature vector for a partial face region through the partial facial feature model 240 .

The full facial feature model 230 may include a full face image extractor 232 for extracting a full face image from the input image and a full feature vector extractor 234 for extracting a full feature vector from the full face image.

Hereinafter, with reference to FIGS. 3 to 6 , the user face recognition unit 220 uses the full face image extractor 232 and the full feature vector extractor 234 included in the full face feature model 230 to obtain an input image. The contents of extracting the entire face image and the entire feature vector from

FIG. 3 is a block diagram showing the configuration of an entire face image extractor constituting the entire facial feature model of FIG. 2 .

Referring to FIG. 3 , the full face image extraction unit 232 is configured based on a convolutional neural network (CNN) and extracts the entire face image including the entire face region from the input image. This whole face image extraction unit 232 includes n convolution operation units 310, 320, 330, 340, 350, n-1 sampling units 315, 325, 335, 345, and an integrated detection unit 360. do.

In an embodiment, the full face image extraction unit 232 may include five convolution operation units 310 , 320 , 330 , 340 , and 350 . In FIG. 3 , for convenience of explanation, the entire face image extraction unit 232 is illustrated as including five convolution operation units 310 , 320 , 330 , 340 , and 350 , but the present invention is not limited thereto. The full face image extraction unit 232 may include less than 4 convolutional operation units or may include 6 or more convolutional operation units.

In one embodiment, the full face image extraction unit 232 may include four sampling units (315, 325, 335, 345). In FIG. 3 , for convenience of explanation, the entire face image extracting unit 232 is illustrated as including the sampling units 315 , 325 , 335 , and 345 , but the present invention is not limited thereto. The entire face image extraction unit 232 may include fewer than three sampling units or may include five or more sampling units.

Each of the first to fifth convolution operation units 310 , 320 , 330 , 340 , 350 applies a convolution filter to an input image to generate a feature map, and applies an activation function to the generated feature map, so that the feature map is non-linear. reflect the characteristics. In this case, the convolution filters applied to the first to fifth convolution operation units 310 , 320 , 330 , 340 and 350 may be different filters.

In an embodiment, the activation function used in the first to fifth convolution operation units 310 , 320 , 330 , 340 , 350 outputs positive values among pixel values of the feature map as it is, and negative values are predetermined. It may be an activation function that outputs a value reduced by the size. Here, the activation function refers to a function that gives a weight to a plurality of input information and outputs a completed result value by combining them.

Each of the first to fourth sampling units 315 , 325 , 335 , 345 is disposed between the first to fifth convolution operation units 310 , 320 , 330 , 340 , and 350 so that k (k is greater than 1 and greater than n). The dimension of the feature map is reduced by applying a sampling filter to the feature map generated by the small integer)-th convolution operation unit, and the reduced-dimensional feature map is input to the k+1-th convolution operation unit.

Specifically, the first sampling unit 315 is disposed between the first convolution operation unit 310 and the second convolution operation unit 320 , and applies a sampling filter to the first feature map output from the first convolution operation unit 310 . to extract a feature value from the first feature map. In an embodiment, the first sampling unit 315 may extract a maximum value among pixel values included in a region corresponding to the sampling filter on the first feature map as a feature value of the first feature map. According to this embodiment, the first sampling unit 315 may be implemented as a max pooling layer, and the dimension of the first feature map may be reduced through the max pooling layer. The first sampling unit 315 inputs the reduced-dimensional feature map to the second convolution operation unit 320 .

The second sampling unit 325 is disposed between the second convolution operation unit 320 and the third convolution operation unit 330 , and applies a sampling filter to the second feature map output from the second convolution operation unit 320 to obtain a second Extract feature values from feature maps. In an embodiment, the second sampling unit 325 may extract a maximum value among pixel values included in a region corresponding to the sampling filter on the second feature map as a feature value of the second feature map. According to this embodiment, the second sampling unit 325 may be implemented as a max pooling layer, and the dimension of the second feature map may be reduced through the max pooling layer. The second sampling unit 325 inputs the reduced-dimensional feature map to the third convolution operation unit 330 .

The third sampling unit 335 is disposed between the third convolution operation unit 330 and the fourth convolution operation unit 340 , and applies a sampling filter to the third feature map output from the third convolution operation unit 330 to obtain a third Extract feature values from feature maps. In an embodiment, the third sampling unit 335 may extract a maximum value among pixel values included in a region corresponding to the sampling filter on the third feature map as a feature value of the third feature map. According to this embodiment, the third sampling unit 335 may be implemented as a max pooling layer, and the dimension of the third feature map may be reduced through the max pooling layer. The third sampling unit 335 inputs the reduced-dimensional feature map to the fourth convolution operation unit 340 .

The fourth sampling unit 345 is disposed between the fourth convolution operation unit 340 and the fifth convolution operation unit 350 , and applies a sampling filter to the fourth feature map output from the fourth convolution operation unit 340 to obtain a fourth Extract feature values from feature maps. In an embodiment, the fourth sampling unit 345 may extract a maximum value among pixel values included in a region corresponding to the sampling filter on the fourth feature map as a feature value of the fourth feature map. According to this embodiment, the fourth sampling unit 345 may be implemented as a max pooling layer, and the dimension of the fourth feature map may be reduced through the max pooling layer. The fourth sampling unit 345 inputs the reduced-dimensional feature map to the fifth convolution operation unit 350 .

The integrated detection unit 360 performs a neural network using the feature maps generated by each of the at least two convolution operation units 310, 320, 330, 340, and 350 among the n convolution operation units 310, 320, 330, 340, and 350. Integrate using the network. For example, as shown in FIG. 3 , the integrated detection unit 360 includes a first feature map generated by the first convolution operation unit 310 , a third feature map generated by the third convolution operation unit 330 , and a fifth The fifth feature map generated by the convolution operation unit 350 may be integrated using a neural network. At this time, the first feature map generated by the first convolution operation unit 310 , the third feature map generated by the third convolution operation unit 330 , and the fifth feature map generated by the fifth convolution operation unit 350 are neural networks Layers in the network may be different from each other. The first feature map generated by the first convolution operation unit 310 corresponds to the lowest layer, the third feature map generated by the third convolution operation unit 330 corresponds to the middle layer, and the fifth convolution operation unit 350 ) may correspond to the highest layer. That is, the integrated detector 360 may integrate feature maps generated in different layers using a neural network.

The integrated detector 360 reduces the dimension by applying the first dimension reduction filter to the integrated feature map using the neural network. In an embodiment, the first dimensionality reduction filter may be set as a filter capable of reducing the feature map in two dimensions.

The integrated detector 360 calculates a first probability value of whether the entire face region is included in the corresponding input image by applying a predetermined classification function to the two-dimensional output data. In an embodiment, when the calculated first probability value is equal to or greater than the first threshold value, the integrated detector 360 may determine that the entire face region is included in the corresponding input image.

When the first probability value is equal to or greater than the first threshold, the integrated detector 360 reduces the dimension by applying the second dimension reduction filter to the integrated feature map using the neural network. In one embodiment, the second dimension reduction filter may be set as a filter capable of reducing the feature map in four dimensions, and four values output in four dimensions may be determined as coordinates of the entire face region on the corresponding input image. . At this time, the coordinates of the entire face area are the coordinates of the upper left vertex and the lower right vertex when the area including the entire face is displayed as a rectangular bounding box as shown in Fig. 8(a). It may be defined or it may be defined by the coordinates of the upper right vertex and the coordinates of the lower left vertex.

The integrated detector 360 extracts the entire face image from the input image determined to include the entire face region by using the calculated coordinates of the entire face region.

Meanwhile, the integrated detector 360 reduces the dimension by applying the third dimension reduction filter to the integrated feature map using the neural network. In an embodiment, the 3D reduction filter may be set as a filter capable of reducing in 10 dimensions, and 10 values output in 10 dimensions may be determined as landmark coordinates on a corresponding input image. In this case, the landmark coordinates may mean coordinates of two eyes, coordinates of a nose, and coordinates of two mouths on the input image. The two coordinates of the mouth may mean coordinates for the left tail of the mouth and coordinates for the right tail of the mouth.

As described above, the entire face image extracting unit 232 according to the present invention may implement a neural network of one face detection unit. The full face image extraction unit 232 may integrate the feature maps generated by the convolution operation units of different layers once again through the neural network. For example, the full face image extraction unit 232 includes a first feature map generated by the first convolution operation unit 310 of the lower layer, a second feature map generated by the third convolution operation unit 330 of the middle layer, and The fifth feature map generated by the fifth convolution operation unit 350 of the high layer may be integrated through the neural network.

In addition, the full face image extractor 232 may determine the presence or absence of the entire face region, coordinates of the entire face region, and landmark coordinates of the entire face using the integrated feature map. The full face image extractor 232 may extract the full face image by using the coordinates of the entire face region.

The full face image extraction unit 232 according to the present invention does not consist of a plurality of face detection units executing a neural network, but may be implemented as a face detection unit composed of a single deep network. Accordingly, the total face image extractor 232 according to the present invention has a small amount of computation, so that the computation speed can be increased. On the other hand, the full face image extraction unit 232 according to the present invention integrates the feature maps generated by the convolution operation units of different layers once again through a neural network, similar to using a plurality of face detection units having different depths. accuracy can be maintained.

Although not shown in FIG. 3 , the full face image extractor 232 may further include an all face image aligner (not shown). The entire face image aligning unit may align the entire face image by using the landmark coordinates output from the integrated detection unit 360 . Specifically, the entire face image aligning unit may align the entire face image by performing at least one of rotation, translation, enlargement, and reduction on the entire face image by using the landmark coordinates for the extracted entire face image. The reason for aligning the face images using the full face image alignment unit is to improve the face recognition performance by imparting consistency to the entire face image to be provided as an input to the full feature vector extraction unit 234 .

In an embodiment, the full face image aligner may resize the entire face image extracted by the integrated detection unit 360 to the same resolution as the full face image used in the full feature vector extractor 234 . . Then, the full face image alignment unit moves the landmark coordinates calculated by the integrated detection unit 360 based on the reference landmark coordinates for the full face image of the resolution used in the full feature vector extraction unit 234 to move the entire face Images can be rotated, translated, enlarged or reduced.

Referring back to FIG. 2 , when the full face image is input from the full face image extracting unit 232 , the full feature vector extraction unit 234 extracts the entire feature vector from the entire face included in the corresponding full face image. Hereinafter, the entire feature vector extractor 234 according to the present invention will be described in more detail with reference to FIGS. 4 to 6 .

4 is a block diagram showing the configuration of the full feature vector extraction unit constituting the full facial feature model of FIG. 2 and the partial feature vector extraction unit constituting the partial facial feature model of FIG. It is a block diagram showing the configuration of the first unit, and FIG. 6 is a block diagram showing the configuration of the second unit included in the face image processing unit of FIG. 4 .

4 to 6 , the entire feature vector extracting unit 234 includes a plurality of face image processing units 410a to 410n and a feature vector generating unit 420 .

The plurality of face image processing units 410a to 410n generate output data by image processing the input data. At this time, the first face image processing unit 410a among the plurality of face image processing units 410a to 410n receives the entire face image output from the full face image extraction unit 232 as an input image, and the n+1th face image processing unit Output data of the n-th face image processing unit 410n is input to (410n+1) as an input image.

For example, the first face image processing unit 410a image-processes the entire face image to generate output data, and uses the generated output data twice. It may be input to the face image processing unit 410b. The first face image processing unit 410b image-processes the output data output from the first face image processing unit 410a to generate new output data, and repeats the generated new output data three times. It may be input to the face image processing unit 410c.

Since the functions and detailed configurations of the plurality of face image processing units 410a to 410n shown in FIG. 4 are the same, the first face image processing unit 410a among the plurality of face image processing units 410a to 410n will be exemplarily described below. Explain. Hereinafter, for convenience of description, the first face image processing unit 410a will be referred to as the face image processing unit 410 .

As shown in FIG. 4 , the face image processing unit 410 performs a convolution operation on input data (either a face image or output data of a previous face image processing unit) to generate a feature map. It is composed of a second unit 414 that gives weight to the feature map generated by the unit 412 , and an operation unit 416 .

The first unit 412 includes a normalization unit 510 , a first convolution operation unit 520 , and a non-linearization unit 530 as shown in FIG. 5 .

The normalization unit 510 normalizes the entire face images input from the full face image extraction unit 232 in batch units. Batch means the unit of number of all face images to be processed at once. The reason that the normalization unit 510 performs normalization in batches is that when normalization is performed in batches, the mean and variance for each face image may be different from the mean and variance for the entire batch. This characteristic acts as a kind of noise. This is because overall performance can be improved.

In addition, through batch normalization, the learning complexity increases and gradient vanishing or exploding occurs due to the internal covariance shift phenomenon, in which the distribution of inputs for each layer of the network changes inconsistently. because it can be prevented.

The first convolution operation unit 520 generates a first feature map by applying a first convolution filter to the entire face image normalized by the normalization unit 510 . In an embodiment, the first convolution operation unit 520 may generate a first feature map by applying a first convolution filter having a size of 3*3 pixels and a stride value of 1 to the face image. As such, the first convolution operation unit 520 according to the present invention applies the first convolution filter having a size of 3*3 pixels and a stride value of 1 to the face image, so that the resolution of the first feature map can be maintained high. do.

The non-linearity unit 530 applies the activation function to the first feature map to impart a non-linear characteristic to the first feature map. In an embodiment, the non-linearization unit 530 outputs an activation function to the first feature map by outputting the same positive pixel value among the pixel values of the first feature map and reducing the size of the negative pixel value to the first feature map. By applying it, a non-linear characteristic can be imparted to the first feature map.

In FIG. 5 , it has been described that the non-linearization unit 530 imparts a non-linear characteristic to the first feature map generated by the first convolution operation unit 520 . However, in a modified embodiment, the normalization unit 510 may additionally normalize the first feature maps generated by the first convolution operation unit 520 in units of batches. According to this embodiment, the normalization unit 510 provides the normalized first feature map to the non-linearization unit 530, and the non-linearization unit 530 applies an activation function to the normalized first feature map to obtain the normalized first feature map. Non-linear characteristics can be given to the feature map.

Meanwhile, in the above-described embodiment, it has been described that the first unit 412 includes only the first convolution operation unit 520 . However, in a modified embodiment, the first unit 412 may further include a second convolution operation unit 540 as shown in FIG. 5 .

Specifically, the second convolution operation unit 540 generates a second feature map by applying a second convolution filter to the first feature map to which the nonlinear characteristic is given by the nonlinearity unit 530 . In an embodiment, the second convolutional filter may be a different filter from the first convolutional filter. The second convolutional filter may be a filter having the same size as the first convolutional filter but different stride values. As an example, the second convolutional filter may be a filter having a size of 3*3 pixels and having a stride value of 2.

According to this embodiment, the normalization unit 510 may additionally normalize the second feature maps generated by the second convolution operation unit 540 in units of batches.

Meanwhile, although not shown in FIG. 5 , the first unit 412 may further include a pre-normalization unit. The pre-normalizer may normalize the pixel value of each pixel included in the face image input from the entire face image extractor 232 . For example, the pre-normalizer may normalize the face image by subtracting 127.5 from each pixel value of the face image and then dividing the subtraction result by 128. The pre-normalizer may provide the pre-normalized input face image to the normalizer 510 .

Referring back to FIG. 4 , the second unit 414 gives weights to the second feature map generated by the first unit 412 . In the present invention, the reason for weighting the second feature map through the second unit 414 is that in the case of convolution operation, all channels of the input image are multiplied by the convolution filter and then summed, both important and non-important channels. Since the data is entangled, the sensitivity (Sensitivity) of the data is lowered. This is to assign a weight to the second feature map according to its importance for each channel.

The second unit 414 includes a sampling unit 610 , a weight reflecting unit 620 , and an upscaling unit 630 as shown in FIG. 4 .

First, the sampling unit 610 reduces the dimension by subsampling the second feature map input from the first unit 412 . In an embodiment, the sampling unit 610 may reduce the dimension of the second feature map by applying a global pooling filter to the second feature map. For example, when the dimension of the second feature map is H*W*C, the sampling unit 610 may reduce the dimension of the second feature map to 1*1*C through subsampling of the second feature map.

The weight reflection unit 620 gives a weight to the second feature map whose dimension has been reduced by the sampling unit 610 . To this end, as shown in FIG. 6 , the weight reflection unit 620 may include a dimension reduction unit 622 , a first non-linearization unit 624 , a dimension increase unit 626 , and a second non-linearization unit 628 . can

The dimension reduction unit 622 reduces the dimension of the sub-sampled second feature map by connecting the sub-sampled second feature map to one layer. For example, when the dimension of the second feature map output from the sampling unit 610 is 1*1*C, the dimension reduction unit 622 reduces the dimension of the second feature map to 1*1*C/r. Here, r denotes a reduction rate, and may be determined according to the number of feature vectors desired to be extracted.

The first non-linearization unit 624 applies a first activation function to the second feature map, the dimension of which has been reduced by the dimension reduction unit 622 , to impart a non-linear characteristic to the second feature map with the reduced dimension. In an embodiment, the first non-linearization unit 624 reduces the dimension by applying a first activation function that outputs a positive pixel value as it is and outputs a negative pixel value as 0 among pixel values of the second feature map. A non-linear characteristic may be imparted to the second feature map.

The dimension increase unit 626 increases the dimension of the second feature map to which the non-linear characteristic is given by the first non-linearization unit 624 . For example, when the dimension of the second feature map to which the non-linear characteristic is given is 1*1*c/r, the dimension increasing unit 626 increases the dimension of the second feature map back to 1*1*C.

The second non-linearization unit 628 applies a second activation function to the second feature map whose dimension has been increased by the dimension increase unit 626, thereby re-applying a non-linear characteristic to the second feature map with an increased dimension. In one embodiment, the second activation function may be a different function from the first activation function. For example, the second non-linearization unit 628 applies a second activation function that causes positive pixel values among pixel values of the second feature map with increased dimensions to converge to a predetermined value and outputs negative pixel values to 0. By doing so, a non-linear characteristic can be imparted to the second feature map having an increased dimension.

The weight reflection unit 620 applies a weight to the second feature map through the dimension reduction unit 622, the first non-linearization unit 624, the dimension increase unit 626, and the second non-linearization unit 628, and , it is possible to limit model complexity and support generalization by limiting the gating mechanism by creating a bottleneck section through the dimension reduction unit 622 and the dimension increase unit 626 .

The upscaling unit 630 upscales the second feature map weighted by the weight reflecting unit 620 to the same dimension as the face image input to the second unit 414 . In an embodiment, when the dimension of the face image input to the second unit 414 is H*W*C, the upscaling unit 620 sets the dimension of the second feature map to which the weight is assigned to H*W*C upscaling to

Referring back to FIG. 4 , the operation unit 416 adds the upscaled second feature map output through the second unit 414 with the face image input to the first unit 412 . In the present invention, the reason for adding the upscaled second feature map output from the second unit 414 to the face image input to the first unit 412 through the operation unit 416 in the present invention is when the depth increases in the convolutional neural network. This is to avoid the Vanish Problem.

The feature vector generation unit 420 merges the feature map output from the last face image processing unit 410n among the plurality of face image processing units 410a to 410n into one layer to reduce the dimension to generate a predetermined number of feature vectors. create In an embodiment, the feature vector generator 420 may output 128 or more feature vectors from the feature map output from the face image processor 410n. For example, the feature vector generator 420 included in the total feature vector extractor 234 may output 512 total feature vectors from the feature map output from the face image processor 410n.

Referring back to FIG. 2 , the partial facial feature model 240 includes a full face image extractor 242 that extracts a full face image from an input image, and a partial face image extractor 244 that extracts a partial face image from the full face image. ) and a partial feature vector extraction unit 245 for extracting a partial feature event from the partial face image.

In an embodiment, the partial facial feature model 240 includes a full feature vector extractor 243 that extracts a full feature vector from the full face image extracted through the full face image extractor 242, and a full feature vector and part A partial feature vector determiner 246 for determining a final partial feature vector based on the feature vector may be further included.

Since the full face image extractor 242 and the full feature vector extractor 243 are substantially the same as the full face image extractor 232 and the full feature vector extractor 234 included in the full face feature model 230 , , a detailed description thereof will be omitted.

The partial face image extraction unit 244 extracts a partial face image including a partial face region from the entire face image.

In an embodiment, the partial face image extraction unit 244 may extract a partial face region including eye coordinates from the entire face image as a partial face image. For example, the partial face image extraction unit 244 may extract an image disposed at the top of the entire face image as a partial face image. The partial face image extracting unit 244 may extract an image arranged within a predetermined range within the entire face region detected by the full face image extracting unit 242 as a partial face image.

In another embodiment, the partial face image extractor 244 may extract a partial face image based on the mask area detected by the mask detection model 225 . For example, the partial face image extraction unit 244 may extract an image excluding a mask area from the entire face image as a partial face image. As another example, the partial face image extraction unit 244 may extract an image disposed at the top of the mask area from the full face image as a partial face image. The partial face image extraction unit 244 may extract an image arranged within a predetermined range from the coordinates of the mask area detected by the mask detection model 225 as a partial face image. As shown in Fig. 8(c), the partial face image extraction unit 244 extracts an image arranged in a rectangular bounding box having a predetermined size and disposed at the top from the area including the mask as a partial face image. can

Meanwhile, the partial face image extraction unit 244 may determine landmark coordinates including coordinates of two eyes and a coordinate of a nose on the partial face image.

Although not shown in the drawings, the partial face image extraction unit 244 may further include a partial face image alignment unit (not shown). The partial face image aligning unit may align the partial face image by using landmark coordinates for the partial face image. Specifically, the partial face image aligning unit may align the partial face image by performing at least one of rotation, translation, enlargement, and reduction on the partial face image using the landmark coordinates of the extracted partial face image. The reason for aligning the face images by using the partial face image alignment unit is to improve the face recognition performance by imparting consistency to the entire face image to be provided as an input to the partial feature vector extraction unit 245 .

In one embodiment, the partial face image alignment unit resizes the partial face image extracted by the partial face image extraction unit 244 to the same resolution as the partial face image used in the partial feature vector extraction unit 245. can Then, the partial face image alignment unit moves the landmark coordinates calculated by the partial face image extraction unit 244 based on the reference landmark coordinates for the partial face image of the resolution used in the partial feature vector extraction unit 245 by moving the Partial face image can be rotated, translated, enlarged or reduced.

When a partial face image is input from the partial face image extraction unit 244 , the partial feature vector extractor 245 extracts a first partial feature vector from the partial face included in the partial face image.

Since the partial feature vector extractor 245 has similar functions and detailed operations to the full feature vector extractor 234 included in the full facial feature model 230, a detailed description thereof will be omitted. However, as described above, the full feature vector extractor 234 outputs 512 full feature vectors after receiving the entire face image including the eyes, nose, and mouth, whereas the partial feature vector extractor 245 generally uses the mouth Partial face images including eyes and nose are inputted, and 256 first partial feature vectors, which are less than 512, may be output.

In an embodiment, the partial feature vector extractor 245 may determine the first partial feature vector as a partial feature vector for the input image. However, it is not necessarily limited thereto.

In the partial facial feature model 240 according to the present invention, the partial feature vector determiner 246 may be additionally configured to increase the reliability of the partial feature vector. The partial feature vector determiner 246 may calculate a second partial feature vector based on the first partial feature vector extracted from the partial face image and the full feature vector extracted from the full face image. The partial feature vector determiner 246 may determine the calculated second partial feature vector as a partial feature vector for the input image.

Specifically, the partial feature vector determiner 246 may assign a first weight to the first partial feature vector extracted from the partial face image. In this case, the first weight may have a value greater than 0.5 and less than 1. As an example, the first weight may have 0.6.

Also, the partial feature vector determiner 246 may reduce the dimension of the entire feature vector extracted from the entire face image, and may assign a second weight to the reduced dimension of the entire feature vector.

The first partial feature vector extracted from the partial face image may include vectors of a first dimension, and the full feature vector extracted from the full face image may include vectors of a second dimension larger than the first dimension. For example, the first partial feature vector extracted from the partial face image may include 256 vectors, and the full feature vector extracted from the full face image may include 512 vectors.

The partial feature vector determiner 246 may reduce the total feature vector extracted from the full face image from the second dimension to the first dimension so that it has the same dimension as the first partial feature vector extracted from the partial face image. there is.

As an example, the partial feature vector determiner 246 may reduce the dimensions of all 512-dimensional feature vectors by 1/2, and convert them into 256-dimensional total feature vectors.

Then, the partial feature vector determiner 246 assigns a second weight to the entire feature vector with a reduced dimension. In this case, the second weight may be smaller than the first weight, and the sum of the first weight and the first weight may be 1. As an example, the first weight may have 0.6, and the second weight may have 0.4.

The partial feature vector determiner 246 may calculate the second partial feature vector by summing the first partial feature vector to which the first weight is given and the entire feature vector to which the second weight is assigned. The partial feature vector determiner 246 may determine the calculated second partial feature vector as a partial feature vector for the input image.

Next, the mask detection model 225 detects a mask region from the input image. Hereinafter, the configuration of the mask detection model 225 will be described in more detail with reference to FIG. 7 .

7 is a block diagram illustrating the configuration of the mask detection model of FIG. 2 .

Referring to FIG. 7 , the mask detection model 225 is configured based on a convolutional neural network (CNN) to detect a mask region from an input image. The mask detection model 225 includes m convolution operation units 710 , 720 , 730 , 740 , 750 , m-1 sampling units 715 , 725 , 735 , 745 , and an integrated detection unit 760 .

In an embodiment, the mask detection model 225 may include five convolution operation units 710 , 720 , 730 , 740 , and 750 . In FIG. 7 , for convenience of explanation, the mask detection model 225 is illustrated as including five convolution operation units 710 , 720 , 730 , 740 , and 750 , but is not limited thereto. The mask detection model 225 may include less than 4 convolutional operation units or may include 6 or more convolutional operation units.

In an embodiment, the mask detection model 225 may include four sampling units 715 , 725 , 735 , and 745 . In FIG. 7 , for convenience of explanation, the mask detection model 225 is illustrated as including the sampling units 715 , 725 , 735 , and 745 , but is not limited thereto. The mask detection model 225 may include less than 3 sampling units or may include 5 or more sampling units.

Each of the first to fifth convolution operation units 710 , 720 , 730 , 740 , and 750 applies a convolution filter to an input image to generate a feature map, and applies an activation function to the generated feature map, so that the feature map is non-linear. reflect the characteristics. In this case, the convolution filters applied to the first to fifth convolution operation units 710 , 720 , 730 , 740 and 750 may be different filters.

In an embodiment, the activation function used in the first to fifth convolution operation units 710 , 720 , 730 , 740 , 750 outputs positive values among pixel values of the feature map as it is, and negative values are predetermined. It may be an activation function that outputs a value reduced by the size. Here, the activation function refers to a function that gives a weight to a plurality of input information and outputs a completed result value by combining them.

Each of the first to fourth sampling units 715 , 725 , 735 , and 745 is disposed between the first to fifth convolution operation units 710 , 720 , 730 , 740 , and 750 , so that k (k is greater than 1 and greater than n). The dimension of the feature map is reduced by applying a sampling filter to the feature map generated by the small integer)-th convolution operation unit, and the reduced-dimensional feature map is input to the k+1-th convolution operation unit. The functions and detailed operations of the first to fourth sampling units 715 , 725 , 735 , and 745 shown in FIG. 7 are substantially the same as those of the first to fourth sampling units 315 , 325 , 335 and 345 shown in FIG. 3 . , so a detailed description thereof will be omitted.

The integrated detection unit 760 uses the feature maps generated by each of the at least two convolution operation units 710, 720, 730, 740, and 750 among the m convolution operation units 710, 720, 730, 740, and 750350 through a neural network. Integrate using the network. For example, as shown in FIG. 7 , the integrated detection unit 760 includes a first feature map generated by the first convolution operation unit 710 , a third feature map generated by the third convolution operation unit 730 , and a fifth The fifth feature map generated by the convolution operation unit 750 may be integrated using a neural network. At this time, the first feature map generated by the first convolution operation unit 710 , the third feature map generated by the third convolution operation unit 730 , and the fifth feature map generated by the fifth convolution operation unit 750 are neural networks Layers in the network may be different from each other. The first feature map generated by the first convolution operation unit 710 corresponds to the lowest layer, the third feature map generated by the third convolution operation unit 730 corresponds to the middle layer, and the fifth convolution operation unit 750 ) may correspond to the highest layer. That is, the integrated detector 760 may integrate feature maps generated in different layers using a neural network.

The integrated detector 760 reduces the dimension by applying the first dimension reduction filter to the integrated feature map using the neural network. In an embodiment, the first dimensionality reduction filter may be set as a filter capable of reducing the feature map in two dimensions.

The integrated detector 760 calculates a second probability value of whether a mask region is included in the corresponding input image by applying a predetermined classification function to the two-dimensional output data. In an embodiment, when the calculated second probability value is equal to or greater than the second threshold value, the integrated detector 760 may determine that the mask region is included in the corresponding input image.

When the second probability value is equal to or greater than the second threshold, the integrated detector 760 reduces the dimension by applying the second dimension reduction filter to the integrated feature map using the neural network. In an embodiment, the second dimension reduction filter may be set as a filter capable of reducing the feature map in four dimensions, and four values output in four dimensions may be determined as coordinates of a mask region on a corresponding input image. . In this case, the coordinates of the mask area are defined as the coordinates of the upper left vertex and the coordinates of the lower right vertex when the area including the mask is displayed as a rectangular bounding box as shown in FIG. 8(b). , may be defined as the coordinates of the upper right vertex and the coordinates of the lower left vertex.

Referring back to FIG. 2 , the array file generating unit 250 includes the full feature vectors extracted from the full face images of each of the plurality of users by the user face recognition unit 220 and the partial face images of each of the plurality of users. An array is created for each user by using partial feature vectors extracted from , and the generated arrays are merged into one file to create an array file.

The array file generator 250 may store the generated array file in an array file database (not shown).

In an embodiment, the array generated by the array file generating unit 250 includes full feature vectors extracted from each user's full face image, partial feature vectors extracted from each user's partial face image, and each user's It may be composed of a key value. In this case, the user's key value includes each user's identification information and each user's access right information. As described above, each user's identification information may be defined as each user's ID, name, phone number, or employee number, etc., and each user's access right information includes information on each floor to which each user can access. may include

In an embodiment, the array file generator 250 may generate an array file for each location where the edge device 120 is installed. For example, the first array file may be composed of arrays of users granted access to the first floor, and the second array file may be composed of arrays of users granted access to the second floor. there is. To this end, the array file generating unit 230 may generate each user's arrays by dividing them by regions to which each user can access. For example, when the first user has permission to access the first floor and the third floor, the array file generator 230 generates a first array including access permission information for the first floor for the first user and a second A second array including access permission information for the third floor can be separately created.

The reason that the array file generating unit 250 according to the present invention generates an array file for each place where the edge device 120 is installed is that when the edge device 120 that authenticates the user's face is installed for each place, a specific place This is because only the array file including the access permission information for the corresponding place needs to be transmitted to the edge device 120 installed in the .

In the above-described embodiment, it has been described that the array file generating unit 250 generates an array file for each location, but in a modified embodiment, the array file generating unit 250 is installed in all places where the edge device 120 is installed. It is also possible to generate one array file including permission information for the data and transmit the generated array file to all edge devices 120 .

The edge device registration unit 260 registers information on a plurality of edge devices 120 installed in each location in the edge device database 262 . According to an embodiment, the edge device registration unit 260 may store identification information of each edge device 120 in the edge device database 262 by mapping the identification information of each edge device to a place where each edge device is installed. Here, the identification information of the edge device 120 may include a manufacturer and serial number of the edge device 120 .

On the other hand, the edge device registration unit 260 may receive the authentication record from the edge device 120 through the interface unit 270 every predetermined period, and store the received access record in the edge device database 262 .

The access authority information management unit 255 changes access authority information assigned to each user or adds new access authority information. In an embodiment, the access right information management unit 255 may separately grant access right information to each user or grant access right information to an organizational unit to which each user belongs.

Meanwhile, the face recognition server 110 according to an embodiment of the present invention may further include a scheduler 265 . The scheduler 265 performs a function of collectively registering new users whenever a predetermined period arrives or a predetermined event occurs. For example, although it has been described that the user registration unit 210 performs a registration procedure for a new user when a registration request occurs from a user, when the face recognition server 110 includes the scheduler 265, at a predetermined time unit or in advance When a predetermined event occurs, the scheduler 265 initiates the operations of the user registration unit 210 , the input image generation unit 215 , and the user face recognition unit 220 so that a new user registration procedure is automatically performed.

The interface unit 270 encrypts the full facial feature model 230 , the partial facial feature model 240 , the mask detection model 225 , and the array file in a predetermined manner and transmits it to each edge device 120 . In one embodiment, the interface unit 270 encrypts the full facial feature model 230, the partial facial feature model 240, the mask detection model 225, and the array file using a public key-based encryption algorithm to each It can be transmitted to the edge device 120 .

Meanwhile, the interface unit 270 may transmit the encrypted array file to the edge device 120 according to a protocol agreed with the edge device 120 . In addition, the interface unit 270 may receive an authentication record from each edge device 120 every predetermined period and provide it to the edge device 120 .

The facial recognition model training unit 280 generates a full facial feature model 230, a partial facial feature model 240, and a mask detection model 225 based on the convolutional neural network, and the generated full facial feature model 230, A partial facial feature model 240 and a mask detection model 225 are trained. Specifically, the face recognition model training unit 280 continuously trains the convolutional neural network constituting the full face feature model 230 , the partial face feature model 240 , and the mask detection model 225 to obtain an optimal face recognition model. create

To this end, the face recognition model training unit 280 includes a full face image extraction training unit 282 that trains the entire face image extraction unit 232, and a full feature vector extraction training unit that trains the entire feature vector extraction unit 234. (284), a mask detection training unit 286 for training the mask detection model 225 and a partial feature vector extraction training unit 288 for training the partial feature vector extraction unit 245 and feature vector extraction units 234 and 245 ) includes an error reducing unit 290 for reducing the error.

The whole face image extraction training unit 282 trains the entire face image extraction unit 232 using the first learning image. The first learning image may include a face image that does not wear a mask.

The entire face image extraction training unit 282 inputs a plurality of first training images having a predetermined size, and calculates a first probability value and facial region coordinates to include a face region in the first training image. The full face image extraction training unit 282 feeds back the calculated first probability value and the facial region coordinates to the full face image extraction unit 232 according to a Back Propagation algorithm applied to the entire face image extraction unit 232. The filter coefficients and weights of convolutional filters are updated.

In one embodiment, the full face image extraction training unit 282 additionally calculates landmark coordinates during training of the full face image extraction unit 232, and combines the calculated landmark coordinates with a first probability value and face region coordinates. The filter coefficients and weights of the convolutional filters applied to the full face image extractor 232 may be updated by feeding back the full face image extractor 232 together through the backpropagation algorithm.

The entire feature vector extraction training unit 284 trains the entire feature vector extraction unit 234 using the first learning image. Specifically, the entire feature vector extraction training unit 284 extracts the entire feature vector from each first training image by inputting a plurality of first training images to the entire feature vector extraction unit 234 in a predetermined batch unit.

The total feature vector extraction training unit 284 predicts a probability value that the corresponding first learning image will be included in a specific class by applying all the extracted feature vectors to a predetermined classification function, and predicts the predicted probability value (hereinafter referred to as 'prediction value') The filter coefficients and weights of the convolutional filters applied to the entire feature vector extractor 234 are updated by calculating the error between the and the actual value and feeding the result back to the full feature vector extractor 234 using a backpropagation algorithm.

The mask detection training unit 286 trains the mask detection model 225 using the second training image. The second learning image may include a face image wearing a mask.

The mask detection training unit 286 inputs a plurality of second training images having a predetermined size, and calculates a second probability value to include a mask region in the second training image and mask region coordinates. The mask detection training unit 286 feeds back the calculated second probability value and the mask region coordinates to the mask detection model 225 according to a back propagation algorithm, so that the filter coefficients of the convolution filters applied to the mask detection model 225 and update the weights.

The partial feature vector extraction training unit 288 trains the partial feature vector extraction unit 245 using the second learning image. Specifically, the partial feature vector extraction training unit 288 extracts a partial feature vector from each second training image by inputting a plurality of second training images to the partial feature vector extraction unit 245 in a predetermined batch unit.

The partial feature vector extraction training unit 288 predicts a probability value that the corresponding second learning image will be included in a specific class by applying the extracted partial feature vectors to a predetermined classification function, and predicts a predicted probability value (hereinafter referred to as a 'prediction value') The filter coefficients and weights of the convolutional filters applied to the partial feature vector extractor 245 are updated by calculating the error between the and the actual value and feeding the result back to the partial feature vector extractor 245 using a backpropagation algorithm.

On the other hand, the face recognition model training unit 280 according to the present invention further reduces the error generated when extracting the entire feature vector or the partial feature vector through the error reducing unit 290, thereby further reducing the full feature vector extracting unit 234 and the partial features. The performance of the vector extractor 245 may be further improved.

The error reduction unit 290 includes a predicted value based on the entire feature vectors extracted through the entire feature vector extraction unit 234 in the process where the entire feature vector extraction training unit 284 trains the entire feature vector extraction unit 234 and It reduces the error between the actual values.

Specifically, the error reduction unit 290 calculates an error between the predicted value and the actual value based on the total feature vectors extracted from the first training image by the total feature vector extraction unit 234 . The error reduction unit 290 arranges each first training image on a two-dimensional angular plane so that the error can be reduced, and the entire feature vector extraction training unit 284 uses a probability value according to the arrangement result to convert the entire feature vector extraction unit (234) to be able to train.

In addition, the error reduction unit 290 is based on the partial feature vectors extracted through the partial feature vector extraction unit 245 in the process where the partial feature vector extraction training unit 288 trains the partial feature vector extraction unit 245 . It reduces the error between the predicted value and the actual value.

Specifically, the error reduction unit 290 calculates an error between the predicted value and the actual value based on the partial feature vectors extracted from the second training image by the partial feature vector extraction unit 245 . The error reduction unit 290 arranges each second training image on a two-dimensional angular plane so that the error can be reduced, and the partial feature vector extraction training unit 288 uses the probability value according to the arrangement result. (245) to be able to train.

The reason why the face recognition model training unit 280 according to the present invention trains the entire feature vector extractor 234 and the partial feature vector extractor 245 to reduce errors through the error reduction unit 290 is shown in FIG. As shown in the figure, since an error such as not being able to distinguish that the face is the same person when the lighting or environment in which the face is photographed changes even though it is the same person occurs, the error of face recognition can be reduced through this error reduction unit 290. This is to train the entire feature vector extracting unit 234 and the partial feature vector extracting unit 245 to extract the feature vectors that are present.

The error reduction unit 290 according to an embodiment of the present invention includes a face image arrangement unit 292 and a probability calculation unit 294 .

The face image arrangement unit 292 includes a plurality of feature vectors (full feature vectors or partials) extracted by the full or partial feature vector extraction units 234 and 245 for the training image (including the first training image or the second training image). Each training image is arranged on a two-dimensional angular plane based on the feature vector). Specifically, the face image arrangement unit 292 calculates the cosine similarity between training images included in different classes, and calculates a reference angle that is a separation angle between each training image according to the cosine similarity, thereby dividing the training images into two-dimensional angles. placed on a flat surface.

The face image arrangement unit 292 arranges each of the second training images on a two-dimensional angular plane based on the plurality of partial feature vectors extracted by the partial feature vector extraction unit 245 for the second training image. In this case, the face image arrangement unit 292 may assign a preset first weight to the partial feature vectors. The first weight may be greater than 0.5 and less than 1. For example, the partial feature vector extraction unit 245 may extract 256 partial feature vectors. In this case, the face image arrangement unit 292 applies a first weight, for example, 0.6, to the 256-dimensional partial feature vector. can

The face image arrangement unit 292 arranges each of the first training images on a two-dimensional angular plane based on the plurality of total feature vectors extracted by the full feature vector extraction unit 234 for the first training image. In this case, the face image arrangement unit 292 may reduce the dimension of all the feature vectors to 1/2 and give a preset second weight. The second weight may be smaller than the first weight. That is, the first weight may be greater than the second weight. For example, the total feature vector extraction unit 234 may extract 512 total feature vectors. In this case, the face image arrangement unit 292 converts the 512-dimensional total feature vectors to 1/2 of the 256-dimensional total feature vectors. , and a second weight, for example, 0.4, may be given to the transformed 256-dimensional entire feature vector.

On the other hand, in the present invention, when the face image arrangement unit 292 arranges the learning images in the vector space according to the distance between the respective learning images calculated based on the feature vectors of the respective learning images, each learning image is arranged in the vector space as shown in FIG. There is a limit in that it is difficult to clearly distinguish the same person from another person during learning because an overlapping area 1000 between the images is bound to occur.

Therefore, in the present invention, the face image arrangement unit 292 calculates the angle between the learning images included in different classes through the cosine similarity, and arranges each learning image on a two-dimensional angular plane based on the calculated angle. will do

The probability calculation unit 294 varies the margin angle to be added to the reference angle calculated by the face image arrangement unit 292 on the two-dimensional angular plane, and determines the probability that each learning image is included in the corresponding class for each variable margin angle. Calculate.

Specifically, as shown in FIG. 11 , the probability calculating unit 294 changes the margin angles P1 and P2 added to the reference angles θ1 and θ2 between the respective training images while varying the learning images having overlapping characteristics. to be spaced apart on a two-dimensional angular plane. In an embodiment, the margin angles P1 and P2 may be determined according to a learning rate within a range greater than 0 and less than 90 degrees.

For example, when the learning rate increases, the margin angle increases accordingly, and when the learning rate decreases, the margin angle may be set to decrease accordingly. In this case, the probability calculator 294 may vary the margin angle by a predetermined reference unit.

When the margin angle is added to the reference angle by the probability calculator 294, it can be seen that the training images having overlapping features in the vector space are arranged to be spaced apart from each other as shown in FIG. 12 .

The probability calculator 294 calculates a probability that each training image is included in a corresponding class for each margin angle added to the reference angle. The probability calculating unit 294 provides the calculated probability value to the full or partial feature vector extraction training units 284 and 288, so that the total or partial feature vector extraction training units 284 and 288 are calculated by the probability calculating unit 294. It enables the whole or partial feature vector extracting units 234 and 245 to be trained on the basis of the obtained probability value. That is, the total or partial feature vector extraction training units 284 and 288 calculate the coefficients and weights of the convolution filters applied to the full or partial feature vector extraction units 234 and 245 based on the probability value calculated by the probability calculator 294 . By updating at least one of the full or partial feature vector extraction units 234 and 245 are learned.

In an embodiment, the probability calculator 294 may calculate a probability that each training image is included in a corresponding class for each margin angle using Equation 1 below.

In Equation 1, x denotes a reference angle, p denotes the margin angle, and n denotes the number of classes.

In one embodiment, the probability calculation unit 294 performs a predetermined test in the full or partial feature vector extraction units 234 and 245 trained by the full or partial feature vector extraction training units 284 and 288 based on the probability value. When a face image is applied, the margin angle can be continuously varied until the error between the predicted value and the actual value is less than or equal to the reference value.

That is, the probability calculating unit 294 is a margin angle at a point in time when the error between the predicted value and the actual value calculated when a predetermined test face image is applied to the trained full or partial feature vector extracting units 234 and 245 becomes less than or equal to the reference value. is determined as the final margin angle. In this case, the error between the predicted value and the actual value may be calculated using a cross entropy function.

When the error reduction is performed through the error reduction unit 290 as described above, the same person can be accurately classified even if the photograph is taken in different environments, lighting, or wearing a mask.

In the above-described embodiment, the face recognition model training unit 280 may be implemented as software in the form of an algorithm and mounted on the face recognition server 110 .

Referring back to FIG. 1 , the edge device 120 is disposed at a specific place and distributed by the face recognition server 110 with a full facial feature model 230 , a partial facial feature model 240 , and a mask detection model 225 . Recognizes the face of the target user who wants to enter the place using

In the present invention, the reason that the edge device 120 performs face recognition and authentication of the target user without the face recognition server 110 performing face recognition and authentication of the target user is that the face recognition and authentication of the target user are performed. In the case of performing in the recognition server 110, when a failure occurs in the face recognition server 110 or the network, not only face recognition and authentication cannot be performed, but also, as the number of users increases, the expansion of the expensive face recognition server 110 is required. because it becomes

Accordingly, the present invention applies an edge computing method to perform face recognition and authentication of a target user in the edge device 120, thereby providing a face recognition service normally even if a failure occurs in the face recognition server 110 or the network It is possible to improve the reliability of service provision, and even if the number of users increases, there is no need to expand the expensive face recognition server 110, thereby reducing the cost of constructing the face recognition system 100.

Hereinafter, the configuration of the edge device 120 according to the present invention will be described in more detail with reference to FIG. 13 .

13 is a block diagram schematically showing the configuration of an edge device according to an embodiment of the present invention.

Referring to FIG. 13 , the edge device 120 according to an embodiment of the present invention includes a photographing unit 1310 , an input image generation unit 1320 , a target face recognition unit 1330 , a mask detection model 1340 , and a part It includes a facial feature model 1342 , a full facial feature model 1345 , an authentication unit 1350 , an interface unit 1360 , an array file update unit 1370 , and a memory 1380 .

The photographing unit 1310 generates a photographed image by photographing the target user when the target user to be authenticated approaches. The photographing unit 1310 transmits the generated photographed image to the input image generation unit 1320 .

The input image generation unit 1320 generates an input image to be used for face recognition from the target user's photographed image transmitted from the photographing unit 1310 . Specifically, the input image generating unit 1320 down-sampling or up-sampling the captured image of one target user to a predetermined stage to generate a plurality of images of the target user having different resolutions from the captured image of one target user. there is.

When a plurality of target user images having different resolutions are generated, the input image generator 1320 inputs a plurality of images obtained by moving a window having a predetermined pixel size on the target user image for each target user image. create as an image The input image generating unit 1320 inputs the generated input image to the target face recognition unit 1330 .

When the target user's input image is received from the input image generating unit 1320, the target face recognition unit 1330 converts the received target user's input image into the mask detection model 1340 distributed from the face recognition server 110, partial face A target feature vector is extracted using the feature model 1342 and the full facial feature model 1345 .

To this end, the target face recognition unit 1330 includes a mask detection unit 1332 , a partial face recognition unit 1334 , and a full face recognition unit 1336 .

The mask detector 1332 inputs the input image input from the input image generator 1320 to the mask detection model 1340 to determine the presence or absence of a mask region and the location of the mask region.

When it is determined by the mask detector 1332 that the mask region is included in the input image, the partial face recognition unit 1334 obtains a target partial feature vector for the target user through the partial facial feature model 1342 .

Specifically, the partial face recognition unit 1334 extracts the target partial face image from the input image by using the mask area coordinates.

In an embodiment, the partial face recognition unit 1334 may extract an image excluding the mask area from the input image as the target partial face image.

In another embodiment, the partial face recognition unit 1334 may extract an image disposed at the top of the mask area from the input image as the target partial face image. The partial face recognition unit 1334 may extract an image disposed within a predetermined range from the coordinates of the mask area detected by the mask detection unit 1332 as the target partial face image. For example, the partial face recognition unit 1334 may extract, as a target partial face image, an image disposed at the top of the region including the mask and disposed in a rectangular bounding box having a predetermined size.

Also, the partial face recognition unit 1334 may determine landmark coordinates including coordinates of two eyes and a coordinate of a nose on the partial face image.

The partial face recognition unit 1334 generates first target partial feature vectors from the target partial face image extracted through the partial facial feature model 1342 . In this case, the partial face recognition unit 1334 generates first-dimensional first target partial feature vectors from the target partial facial image extracted through the partial facial feature model 1342 . In an embodiment, the partial face recognition unit 1334 may generate a 256-dimensional first target partial feature vector through the partial facial feature model 1342 .

In an embodiment, the partial face recognition unit 1334 may determine the first target partial feature vector as the target partial feature vector for the target user.

In another embodiment, the partial face recognition unit 1334 further extracts a target full face image from the input image, and uses the target full feature vector and the first target partial feature vector extracted from the target full face image to perform the target partial feature It is also possible to determine a vector.

Specifically, the partial face recognition unit 1334 inputs the input image input from the input image generation unit 1320 into the partial facial feature model 1342 to determine the presence or absence of a face region, coordinates of the face region, and landmark coordinates, , it is possible to extract the entire target face image using the coordinates of the face region. In this case, the target full face image may correspond to an image of the entire face area including the mask area.

The partial face recognition unit 1334 may generate target full feature vectors from the target full face image extracted through the partial facial feature model 1342 . In this case, the partial face recognition unit 1334 may generate second-dimensional target full feature vectors from the target full face image. In this case, the second dimension may be larger than the first dimension. In an embodiment, the partial face recognition unit 1334 may generate a 512-dimensional target full feature vector.

The partial face recognition unit 1334 may assign a first weight to the first target partial feature vector. In this case, the first weight may have a value greater than 0.5 and less than 1. As an example, the first weight may have 0.6.

Also, the partial face recognition unit 1334 may reduce the dimension of the entire target feature vector, and may assign a second weight to the reduced dimension of the entire target feature vector. In this case, the second weight may be smaller than the first weight, and the sum of the first weight and the first weight may be 1. As an example, the first weight may have 0.6, and the second weight may have 0.4.

The partial face recognition unit 1334 may calculate the second target partial feature vector by summing the first target partial feature vector to which the first weight is assigned and the entire target feature vector to which the second weight is assigned. The partial face recognition unit 1334 may determine the second target partial feature vector as the target partial feature vector for the target user.

On the other hand, when it is determined by the mask detector 1332 that the mask region is not included in the input image, the partial face recognition unit 1334 does not operate.

The full face recognition unit 1336 obtains a target full feature vector for the target user through the full face feature model 1345 .

Specifically, the full face recognition unit 1336 inputs the input image input from the input image generation unit 1320 into the full face feature model 1345 to determine the presence or absence of a face region, coordinates of the face region, and landmark coordinates, , extracts the entire target face image using the coordinates of the face region.

The full face recognition unit 1336 generates target full feature vectors from the target full face image extracted through the full face feature model 1345 . In this case, the full face recognition unit 1336 generates second-dimensional target full feature vectors from the target full face image extracted through the full face feature model 1345 . The second dimension may be larger than the first dimension.

In an embodiment, the full face recognition unit 1336 may generate a 512-dimensional target full feature vector through the full face feature model 1345 . The full face recognition unit 1336 may determine the 512-dimensional target full feature vector as the target total feature vector for the target user.

The authenticator 1350 authenticates the target user by comparing the target feature vector obtained by the target face recognition unit 1330 with the array file received from the face recognition server 110 .

Specifically, the authenticator 1350 compares the target feature vector with an array file composed of a plurality of arrays having user partial feature vectors of each of a plurality of users, total user feature vectors, and identification information of each user to target the target feature vector. You can authenticate users.

To this end, the authentication unit 1350 includes a partial feature vector comparison unit 1352 , a full feature vector comparison unit 1354 , and an authentication decision unit 1356 .

When the target partial feature vector is extracted by the partial face recognition unit 1334 , the partial feature vector comparison unit 1352 compares the extracted target partial feature vector with user partial feature vectors of each of the plurality of users.

Specifically, the partial feature vector comparison unit 1352 may calculate a degree of similarity (hereinafter, referred to as 'partial similarity') between each of the user partial feature vectors and the target partial feature vector. The partial feature vector comparison unit 1352 may calculate the partial similarity through a Euclidean distance or a cosine distance between the target partial feature vector and the user partial feature vectors.

The total feature vector comparison unit 1354 compares the entire target feature vector extracted by the entire face recognition unit 1336 with the total user feature vectors of each of the plurality of users.

Specifically, the total feature vector comparison unit 1354 may calculate a similarity (hereinafter, referred to as 'total similarity') between each of the user total feature vectors and the target total feature vector. The total feature vector comparison unit 1354 may calculate the partial similarity through a Euclidean distance or a cosine distance between the total target feature vector and the user total feature vectors.

The authentication determining unit 1356 may determine whether to authenticate the target user based on at least one of a partial similarity and an overall similarity.

When the target user wears a mask, the authentication determiner 1356 may determine whether to authenticate the mask wearing target user by using the partial similarity.

In an embodiment, the authentication determiner 1356 may determine that the user mapped to the array having the greatest partial similarity with the target partial feature vector is the similar user most similar to the target user. In this case, if the degree of partial similarity with the similar user is equal to or greater than the first threshold value, the authentication determination unit 1356 authenticates the target user as a user with legitimate authority, and accordingly, the target user may be permitted to enter the corresponding place. On the other hand, if the partial similarity is less than the first threshold, the authentication determiner 1356 may reject the authentication of the target user, re-photograph the face of the target user, and repeat the authentication process again.

In one embodiment, the authentication determining unit 1356 determines whether to authenticate the mask wearing target user by using the overall similarity between the entire user feature vectors and the target total feature vectors in order to increase the authentication reliability for the mask wearing target user. can decide

Specifically, the target face recognition unit 1330 may provide the authentication unit 1350 with the target partial feature vector and the entire target feature vector for the mask-wearing target user. In this case, the authenticator 1350 calculates a partial similarity between each of the user partial feature vectors and the target partial feature vector of the mask wearing target user through the partial feature vector comparison unit 1352 , and the overall feature vector comparison unit 1354 ) It is possible to calculate an overall similarity between each of the user overall feature vectors and the mask wearing target user's target overall feature vectors.

The authentication determiner 1356 may extract the first user mapped to the array having the greatest partial similarity with the target partial feature vector. Also, the authentication determiner 1356 may extract at least two or more second users mapped to an array having a high overall similarity with the target overall feature vectors. For example, the authentication determiner 1356 may extract one first user and three second users.

The authentication determining unit 1356 may determine whether the same person as the first user exists among at least two or more second users, and if there is, determine that the corresponding user is a similar user most similar to the target user.

In this case, if the degree of partial similarity with the similar user is equal to or greater than the first threshold value, the authentication determination unit 1356 authenticates the target user as a user with legitimate authority, and accordingly, the target user may be permitted to enter the corresponding place. On the other hand, if the partial similarity is less than the first threshold, the authentication determiner 1356 may reject the authentication of the target user, re-photograph the face of the target user, and repeat the authentication process again.

Meanwhile, when the target user does not wear a mask, the authentication determiner 1356 may determine whether to authenticate the general target user using the overall similarity.

In an embodiment, the authentication determiner 1356 may determine that the user mapped to the array having the greatest overall similarity with the target overall feature vectors is the similar user most similar to the target user. In this case, if the total similarity with the similar user is equal to or greater than the second threshold, the authentication determination unit 1356 authenticates the target user as a user having a legitimate authority, and accordingly, the target user may be permitted to enter the corresponding place. On the other hand, if the overall similarity is less than the first threshold, the authentication determiner 1356 may reject authentication of the target user, re-photograph the face of the target user, and repeat the authentication process again. Meanwhile, the second threshold value may be different from the first threshold value and may be greater than the first threshold value.

The mask detection model 1340 , the partial facial feature model 1342 , and the full facial feature model 1345 are generated and distributed by the face recognition server 110 , and may be updated at predetermined intervals. For example, the edge device 120 is the face recognition server 110 whenever the mask detection model 1340, the partial facial feature model 1342, and the full facial feature model 1345 are updated by the face recognition server 110. A new mask detection model 1340, partial facial feature model 1342, and full facial feature model 1345 are distributed from the mask detection model 1340, partial facial feature model 1342, and full facial feature model 1345 may be updated with a new mask detection model 1340 , a partial facial feature model 1342 , and a full facial feature model 1345 .

The array file update unit 1370 receives, from the face recognition server 110, a new array file composed of a plurality of arrays including feature vectors extracted from the optimal reference image at each predetermined update cycle, and converts the existing array file into a new array file. change to

When the array file update unit 1370 receives the array file from the face recognition server 110 through the interface unit 1350, it uploads it to the first memory 1382, and the authentication unit 1350 uses it to authenticate the target user. make it possible In particular, the array file update unit 1370 according to the present invention may dynamically load the array file.

Specifically, when the array file is loaded in the first memory 1382 and the array file update unit 1370 receives the new array file from the face recognition server 110, the array file update unit 1370 transfers the new array file to the second memory 1384. and, when the loading of the new ray file in the second memory 1384 is completed, the array file loaded in the first memory 1382 is replaced with the new array file loaded in the second memory 1384 .

The reason why the array file update unit 1370 dynamically loads the array file as described above is that the array file update unit 1370 updates the new array file while the authentication unit 1350 performs authentication processing for the target user. This is to enable the edge device 120 to perform face recognition in real time based on the newly updated array file.

An array file used by the authenticator 1350 is loaded into the first memory 1382 , and a newly received array file is loaded into the second memory 1384 . When the loading of the new array file into the second memory 1384 is completed, the array file written in the first memory 1382 by the array file update unit 1370 is replaced with the new array file.

The interface unit 1360 mediates data transmission/reception between the edge device 120 and the face recognition server 110 . Specifically, the interface unit 1360 receives the mask detection model 1340 , the partial facial feature model 1342 , and the full facial feature model 1345 from the face recognition server 110 . The interface unit 1360 receives the array file from the face recognition server 110 and loads it into the first memory 1382 or the second memory 1384 through the array file update unit 1370 . In addition, the interface unit 1360 periodically transmits the authentication record by the authentication unit 1350 to the face recognition server 110 .

In an embodiment, the array file, the mask detection model 1340 , the partial facial feature model 1342 , and the full facial feature model 1345 may be updated at predetermined intervals through the interface unit 1360 .

As described above, according to the present invention, only the mask detection model 1340 for face recognition, the partial facial feature model 1342 and the full facial feature model 1345, and the array file are stored in the edge device 120 according to the present invention. Because no face image or personal information is stored, even if the edge device 120 is hacked, there is no fear of leakage of the user's personal information, thereby enhancing security.

Referring back to FIG. 1 , the user terminal 130 transmits a user image for newly registering a user to the face recognition server 110 together with the user's identification information. In an embodiment, the user terminal 130 is equipped with a face registration agent (not shown) capable of interworking with the face recognition server 110 , and the user executes the face registration agent on the user terminal 130 . It is possible to transmit an image or a pre-photographed image of the face of the user to the face recognition server 110 together with user identification information.

In an embodiment, the user terminal 130 may request to register a plurality of user images for each user. In this case, the plurality of images requested for registration by each user may be pictures taken in different environments or pictures taken under different lighting conditions.

The user terminal 130 may use any type without limitation as long as it can request user registration by transmitting a user image to the face recognition server 110 . For example, the user terminal 130 may be implemented as a smart phone, a laptop computer, a desktop computer, or a tablet PC.

The mask wearing face authentication system 100 according to an embodiment of the present invention determines whether a mask is worn from an input image, and when wearing a mask, a target using a face recognition model learned using a partial face image and a full face image Perform user authentication. Through this, the mask wearing face authentication system 100 according to an embodiment of the present invention can improve the face recognition rate for a person wearing a mask.

Those skilled in the art to which the present invention pertains will understand that the above-described present invention may be embodied in other specific forms without changing the technical spirit or essential characteristics thereof.

For example, the configuration of the face recognition server shown in FIG. 2 and the configuration of the edge device shown in FIG. 13 may be implemented in the form of a program. When the configuration of the face recognition server and the configuration of the edge device according to the present invention are implemented as a program, each configuration shown in FIGS. 2 and 13 is implemented as a code, and codes for implementing a specific function are implemented as a single program Alternatively, a plurality of programs may be divided and implemented.

Those skilled in the art to which the present invention pertains will be able to understand that the above-described present invention may be embodied in other specific forms without changing the technical spirit or essential characteristics thereof.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: face recognition system 110: face recognition server
120: edge device 130: user terminal
210: user register 215: input image generator
220: user face recognition unit 225: mask detection model
230: full facial feature model 240: partial facial feature model
250: array file generation unit 280: face recognition model training unit

Claims (18)

a user face recognition unit generating a partial feature vector by inputting an input image generated from the user image of the user requested for registration into the partial facial feature model, and inputting the input image into the full facial feature model to generate a full feature vector;
an array file generator for generating an array composed of the partial feature vector, the entire feature vector, and user identification information for each user, and merging the generated arrays to generate an array file; and
and a face recognition server including an interface unit for transmitting the partial facial feature model, the full facial feature model, and the generated array file to an edge device,
The partial facial feature model extracts a partial facial image including a partial facial region from the input image, and generates a first-dimensional partial feature vector based on the extracted partial facial image,
The full facial feature model extracts a full face image including the entire face region from the input image, and a mask for generating a full feature vector of a second dimension larger than the first dimension based on the extracted full face image Wearable face recognition system. According to claim 1, wherein the partial facial feature model,
a full face image extraction unit for extracting an entire face region including eye coordinates, nose coordinates, and mouth coordinates from the input image as a full face image; and
and a partial face image extraction unit for extracting a partial face region including eye coordinates from the full face image as a partial face image. According to claim 2, wherein the partial facial feature model,
a partial feature vector extraction unit for extracting a first partial feature vector from the partial face image;
a full feature vector extractor for extracting all feature vectors from the entire face image; and
A second partial feature vector is calculated based on the first partial feature vector extracted by the partial feature vector extracting unit and the entire feature vector extracted by the full feature vector extraction unit, and the calculated second partial feature vector is used in the Mask wearing face recognition system, characterized in that it further comprises a partial feature vector determiner to determine the partial feature vector for the input image. The method of claim 3, wherein the partial feature vector determiner comprises:
assigning a first weight to the first partial feature vector, reducing the entire feature vector from the second dimension to the first dimension, and assigning a second weight to the total feature vector reduced in the first dimension; A mask wearing face recognition system, characterized in that the first partial feature vector to which the first weight is assigned and the total feature vector to which the second weight is assigned are summed to calculate a second partial feature vector. 5. The method of claim 4,
The first weight is greater than the second weight, the mask wearing face recognition system characterized in that. The method of claim 2, wherein the full face image extraction unit,
n pieces (n is an integer greater than or equal to 3) that are sequentially arranged to generate a feature map by applying a convolution filter to the input images, and apply an activation function to the generated feature map to give a non-linear characteristic to the feature map convolution operators of ; and
Feature maps generated by each of at least two or more convolution operation units among the n convolution operation units are integrated using a neural network, and based on the integrated feature map, the presence or absence of the entire face region, the coordinates of the entire face region, and the total A face recognition system for wearing a mask, comprising an integrated detector that determines the coordinates of the eyes, the nose, and the mouth of the face. According to claim 1,
The first dimension is 256 dimensions, and the second dimension is a mask wearing face recognition system, characterized in that 512 dimensions. According to claim 1,
Mask wearing face recognition system, characterized in that it further comprises a mask detection model for detecting the presence or absence of the mask region and the position of the mask region from the input image. an interface unit for receiving a mask detection model, a partial facial feature model, a full facial feature model, and an array file including a partial user feature vector, a full user feature vector, and user identification information of each of a plurality of users from the face recognition server;
a photographing unit that acquires a photographed image of a target user to be authenticated;
A mask region is detected by inputting an input image generated from the photographed image into a mask detection model, and when the mask region is detected, the input image is input to a partial facial feature model to generate a target partial feature vector, and the mask region a target face recognition unit for generating a target full feature vector by inputting the input image into a full facial feature model if this is not detected; and
Authentication for determining whether to authenticate the target user by comparing each of the user partial feature vectors of a plurality of users with the target partial feature vector, or comparing each of the user total feature vectors of a plurality of users with the target overall feature vector Edge device for mask-wearing face recognition comprising a part. The method of claim 9, wherein the target face recognition unit,
a mask detection unit inputting the input images into the mask detection model to determine the presence or absence of a mask area and a position of the mask area;
When the mask is detected by the mask detector, a partial face recognition unit that extracts a target partial face image from the input image through the partial facial feature model, and determines a target partial feature vector for the extracted target partial face image ; and
If the mask is not detected by the mask detector, extracting a target full face image from the input image through the full face feature model, and full face recognition for determining a target full feature vector for the extracted target full face image Edge device for face recognition wearing a mask, characterized in that it comprises a part. 11. The method of claim 10,
The partial face recognition unit extracts a target full face image including a full face area from the input image, and extracts a target partial face image including a partial face area except for the detected mask area from the target full face image. An edge device for mask-wearing face recognition. 12. The method of claim 11,
The partial face recognition unit extracts a first target partial feature vector from the target partial face image, extracts a full feature vector from the target full face image, and based on the first target partial feature vector and the full feature vector, Edge device for face recognition wearing a mask, characterized in that determining the target partial feature vector for the target user. 13. The method of claim 12,
The partial face recognition unit determines a value obtained by adding a first target partial feature vector to which a first weight is assigned and a total feature vector to which a second weight smaller than the first weight is assigned as a target partial feature vector for the target user. Edge device for face recognition wearing a mask, characterized in that. 11. The method of claim 10, wherein the mask detection model,
n pieces (n is an integer greater than or equal to 3) that are sequentially arranged to generate a feature map by applying a convolution filter to the input images, and apply an activation function to the generated feature map to give a non-linear characteristic to the feature map convolution operators of ; and
Feature maps generated by each of at least two or more convolution operation units among the n convolution operation units are integrated using a neural network, and the presence or absence of the mask area and the location of the mask area are determined based on the integrated feature map Edge device for face recognition wearing a mask, characterized in that it comprises an integrated detection unit. 10. The method of claim 9,
The edge device for face recognition wearing a mask, characterized in that the user partial feature vector and the target partial feature vector are 256 dimensions, and the user overall feature vector and the target total feature vector are 512 dimensions. 10. The method of claim 9,
a partial feature vector comparator for calculating a partial similarity between the target partial feature vector, each of the user partial feature vectors of a plurality of users, and the target partial feature vector when the target partial feature vector is generated by the target face recognition unit;
a total feature vector comparison unit for calculating a total similarity between each of the user total feature vectors of a plurality of users and the total target feature vector when the target full feature vector is generated by the target face recognition unit; and
and an authentication determiner configured to determine whether to authenticate the target user based on at least one of the partial similarity and the overall similarity. 17. The method of claim 16,
The target face recognition unit, when a mask region is detected in the input image, generates a target partial feature vector and a target full feature vector for a target user wearing a mask,
The authentication determining unit, edge device for face recognition wearing a mask, characterized in that it determines whether to authenticate the mask wearing target user based on the partial similarity with the target partial feature vector and the total similarity with the entire target feature vector . 18. The method of claim 17,
The authentication determining unit extracts a first user having the highest partial similarity, extracts at least two or more second users in an order of increasing the overall similarity, and selects the same person as the first user from among the at least two or more second users. The edge device for face recognition wearing a mask, characterized in that if the presence and the partial similarity of the first user is greater than or equal to a first threshold, authenticating the mask wearing target user as a legitimate user.

Images