KR20200079095A - Edge Device for Face Recognition - Google Patents

Abstract

According to one embodiment of the present invention, an edge device for face recognition can perform face recognition and authentication processing on an edge device by distributing a face recognition model created on a central server to each edge device. The edge device for face recognition includes: a first photographing part obtaining a photographed image of a target user which is an authentication target; a face recognition part extracting a target face image from input images created from the photographed image by inputting the input images into a face recognition model, and then, creating a target characteristic vector from the extracted target face image; an authentication part authenticating the target user by comparing the target characteristic vector to an array file comprising a plurality of arrays having identification information of the user and characteristic vectors extracted from the face image of the user; and an interface part receiving the array file and the face recognition model from a central server.

Description

Edge Device for Face Recognition

The present invention relates to face recognition, and more particularly, to a device for face recognition.

Face Recognition (Face Recognition) technology is one of the fields of biometrics (Biometrics) refers to a technology that automatically identifies and authenticates a person by using unique feature information contained in each person's face. It is widely used in various fields in recent years because of its superior security compared to the authentication method of.

The general face recognition system determines whether the access gate is opened by transmitting a face image photographed from a device installed at the access gate to the server, the server performs user authentication according to the face recognition and face recognition, and transmits the authentication result to the device. .

In the case of the general face recognition system as described above, since not only the face recognition function but also the authentication function according to the result of the face recognition are implemented in the server, when a face recognition system is applied to a place where there are a large number of users, high performance and high cost There is a problem in that the server is required to increase the construction cost of the face recognition system.

In addition, in the case of a general face recognition system, since the face recognition function is concentrated on the server, when a failure occurs in the server or the network, there is a problem in that it is impossible to provide the face recognition service itself.

In addition, in the case of a general face recognition system, since the face image photographed from the device has to be transmitted to a server through a network, the face image may be leaked to the outside through hacking, etc., thereby making it vulnerable to personal information security.

In addition, in the case of the general face recognition system, despite being the same person, there is a problem in that it is not possible to distinguish the same person when the face is photographed in different environments or when the face is photographed in different illuminance.

The present invention is to solve the above-mentioned problems, and by providing a face recognition model generated by the central server to each edge device by providing an edge device for face recognition that can perform face recognition and authentication processing to the edge device. This is a technical task.

In addition, another technical problem of the present invention is to provide an edge device for face recognition that can perform face recognition and authentication processing without storing a user's face image and personal information.

In addition, another object of the present invention is to provide an edge device for face recognition that can easily update an array file corresponding to a face image of a new user.

An edge device for face recognition according to an aspect of the present invention for achieving the above object includes a first photographing unit acquiring a photographed image of a target user to be authenticated; A face recognition unit for inputting the input images generated from the photographed image into a face recognition model, extracting a target face image from the input image, and generating a target feature vector from the extracted target face image; An authentication unit authenticating the target user by comparing the target feature vector with an array file composed of a plurality of arrays having user identification information and feature vectors extracted from the face image of the user; And an interface unit that receives the array file and the face recognition model from a central server.

According to the present invention, since the face recognition model generated by the central server is distributed to each edge device, face recognition and authentication processing is performed on the edge device, so even if the face recognition system is applied to a large number of users, the high performance and high cost There is an effect that the construction cost of the face recognition system can be reduced because no server is required.

Further, according to the present invention, since face recognition and authentication processing is performed on the edge device by being distributed to each edge device, even if a server or network failure occurs, the face recognition service can be continuously provided, thereby improving service reliability. Since the face image photographed on the edge device is not transmitted to the server, the possibility that the face image is leaked to the outside is blocked in advance, thereby improving the security of personal information.

In addition, according to the present invention, since only the face recognition model and the array file for face recognition are stored in the edge device, and the user's face image or personal information is not stored, there is no fear of leaking the user's personal information even if the edge device is hacked. It has the effect of strengthening security.

In addition, the present invention has an effect that it is easy to add a new user because only the array file corresponding to the face image of the new user needs to be updated on the edge device without changing the hardware or the face recognition model when adding a new user.

1 is a block diagram schematically showing the configuration of a face recognition system according to an embodiment of the present invention.
2 is a block diagram schematically showing the configuration of a central server according to an embodiment of the present invention.
3A is a diagram exemplarily showing a method of obtaining a plurality of user images having different resolutions by downsampling the user image.
3B is a diagram illustrating landmark coordinates in a face image by way of example.
4A to 4D are block diagrams showing a configuration of a face image extraction unit constituting a face recognition model.
5 is a block diagram schematically showing the configuration of a feature vector extraction unit according to the present invention.
6 is a block diagram showing the configuration of the first unit included in the face image processing unit.
7 is a block diagram showing the configuration of the second unit included in the face image processing unit.
8 is a diagram showing an example in which the general face recognition model does not recognize the same person.
9 is a diagram illustrating an example in which overlapping areas are generated when a learning image is placed in a vector space according to the distance between the learning images.
10 is a diagram showing an example in which a learning image is arranged on a two-dimensional angular plane.
FIG. 11 is a diagram exemplarily showing that the learning images are separated by providing a margin angle between the learning images on a 2D angular plane.
12 is a view showing an example in which the same images taken in different environments are accurately classified when error reduction is performed by the error reduction unit.
13 is a block diagram schematically showing the configuration of an edge device according to a first embodiment of the present invention.
14 is a diagram exemplarily showing a method of authenticating a target user by an authentication unit.
15 is a block diagram schematically showing the configuration of an edge device according to a second embodiment of the present invention.
16A and 16B are diagrams showing an example of a depth image generated by the second photographing unit.

The meaning of the terms described in this specification should be understood as follows.

It should be understood that a singular expression includes a plurality of expressions unless the context clearly defines otherwise, and the terms "first", "second", etc. are used to distinguish one component from another component, The scope of rights should not be limited by these terms.

It should be understood that terms such as “include” or “have” do not preclude the existence or addition possibility of one or more other features or numbers, steps, actions, components, parts or combinations thereof.

It should be understood that the term “at least one” includes all possible combinations from one or more related items. For example, the meaning of “at least one of the first item, the second item, and the third item” means 2 of the first item, the second item, and the third item, as well as the first item, the second item, and the third item, respectively. Any combination of items that can be presented from more than one dog.

Hereinafter, the configuration of the face recognition system according to the present invention will be described in more detail with reference to FIG. 1.

1 is a block diagram schematically showing the configuration of a face recognition system according to an embodiment of the present invention. 1, the face recognition system 100 according to an embodiment of the present invention includes a central server 110 and a plurality of edge devices 120.

The central server 110 generates a face recognition model, and uses the generated face recognition model to generate an array file for authentication of the target user using a feature vector extracted from the user's face information input from the user terminal 130 ( Array File). The central server 110 transmits the generated array file to the edge device 120 so that the edge device 120 can authenticate the target user.

To this end, the central server 110 according to the present invention, as shown in Figure 2, the user registration unit 210, the input image generation unit 215, the face recognition unit 220, the face recognition model 225, the array It includes a file generating unit 230, a face recognition model training unit 240, and an interface unit 250.

The user registration unit 210 receives one or more user images from the user terminal 130 of the user who wishes to register. When a user image is received, the user registration unit 210 checks whether the user is the same person as the user image, and if it is determined that the user is the same person, obtains the access permission information given to the user, and the user database 212 together with the user image To register.

In one embodiment, the user registration unit 210 may receive identification information of the user from the user terminal 130 together with the user image. For example, the user registration unit 210 may receive the user's identification information such as the user's ID, name, phone number, or user's employee number together with the corresponding user image. According to this embodiment, the user registration unit 210 may register the user's identification information and the user's access information to the user database 212 along with the corresponding user image.

Meanwhile, when a plurality of user images are input from the user terminal 130, the user registration unit 210 may induce different user images to be input. For example, the user registration unit 210 may induce a user to input a user image photographed in another environment or a user image photographed in different illuminance through the user terminal 130. As described above, the user registration unit 210 can improve the accuracy of face recognition by receiving a plurality of user images photographed in different environments or different illuminances from a single user.

The input image generation unit 215 generates an input image to be used for face recognition from the user image input by the user registration unit 210. Specifically, the input image generation unit 215 generates a plurality of user images having different resolutions from one user image by downsampling or upsampling one user image to a predetermined step. For example, the input image generation unit 215 may generate a plurality of user images 400a to 400n having different resolutions by downsampling one user image 400 as illustrated in FIG. 3A.

In one embodiment, the input image generation unit 215 generates a downsampled user image by applying a Gaussian Pyramid to the user image, or upsampled by applying a Laplacian Pyramid to the user image. You can create user images.

When a plurality of user images having different resolutions are generated, the input image generation unit 215 displays a window 405 having a predetermined pixel size on the user image 400 for each user image, as shown in FIG. 3B. A plurality of images obtained while moving is generated as an input image. The input image generation unit 215 inputs the generated plurality of input images to the face recognition unit 220.

The face recognition unit 220 inputs a plurality of input images generated by the input image generation unit 215 into the face recognition model 225 trained by the face recognition model training unit 250 to face the face including the face region. An image is acquired, and a feature vector is extracted from the acquired face image.

In one embodiment, the face recognition model 225 may include a face image extraction unit 227 for extracting the face image from the input image and a feature vector extraction unit 229 for extracting the feature vector from the face image.

Hereinafter, with reference to FIGS. 4 to 7, the face recognition unit 220 uses the face image extraction unit 227 and the feature vector extraction unit 229 included in the face recognition model 225 and the face image from the input image. The content of extracting the feature vector will be described in detail.

4A to 4D are block diagrams showing a configuration of a face image extraction unit constituting a face recognition model. The face image extraction unit 227 according to the present invention is configured based on a convolutional neural network (CNN) and extracts a face image including a face region from an input image. As illustrated in FIG. 4A, the face image extraction unit 227 may include a first face detection unit 310, a second face detection unit 320, a third face detection unit 330, and a face image alignment unit ( 340).

The first face detection unit 310 extracts a feature of each input image by applying a convolution operation to the input image input by the face recognition unit 220, and based on the extracted feature, a face region on the input image 1 Extract it gradually.

To this end, the first face detection unit 310 includes n convolution calculation units 311a to 311c, a sampling unit 313, first and second dimensional reduction units 315a and 315b, as shown in FIG. 4B, and And a first probability value calculating unit 317.

In one embodiment, the first face detection unit 310 may include three convolution calculation units 311a to 311c. In FIG. 4B, for convenience of description, the first face detection unit 310 includes three convolution calculation units 311a to 311c, but this is only one example, and the first face detection unit 310 is four or more. It may include a convolution operator or one or two convolution operators.

Each of the first to third convolution calculation units 311a to 311c generates a feature map by applying a convolution filter to the input image, and reflects nonlinear characteristics to the feature map by applying an activation function to the generated feature map. In this case, the convolution filters applied to the first to third convolution operation units 311a to 311c may be different filters from each other.

In one embodiment, the activation function used in the first to third convolution operation units 311a to 311c outputs a positive value among pixel values of the feature map and a negative value is reduced by a predetermined size. It can be an activation function to output. Here, the activation function refers to a function that combines weighted values of a plurality of input information and outputs a completed result.

The sampling unit 313 extracts the feature values from the feature map by applying a sampling filter to the feature map output from the first convolution operation unit 311a. In one embodiment, the sampling unit 313 may extract a maximum value among pixel values included in a region corresponding to the sampling filter on the feature map as a feature value of the feature map. According to this embodiment, the sampling unit 313 may be implemented as a Max Pooling layer, and the dimension of the feature map is reduced through the Max pooling layer. The sampling unit 313 inputs the feature map with reduced dimensions to the second convolution operation unit 311b.

The first dimensional reduction unit 315a reduces the dimension of the feature map output from the third convolution calculation unit 311c by applying the first dimensional reduction filter to the feature map output from the third convolution calculation unit 311c. In one embodiment, the first dimension reduction filter may be set as a filter capable of reducing the feature map in two dimensions.

The first probability value calculating unit 317 applies a predetermined classification function to the 2D output data output by the first dimensional reduction unit 315a to determine a first probability value of whether a face region is included in the corresponding input image. To calculate. In one embodiment, if the calculated first probability value is greater than or equal to the first threshold, the first probability value calculating unit 317 may determine that the face region is included in the input image.

The second dimensional reduction unit 315b applies a second dimensional reduction filter to the feature map output from the third convolution operation unit 311c when the first probability value calculated by the first probability value calculation unit 317 is greater than or equal to the first threshold value. By doing so, the dimension of the feature map output from the third convolution operation unit 311c is reduced. In one embodiment, the second dimensional reduction filter may be set as a filter capable of reducing the feature map to four dimensions, and the second dimensional reduction unit 315b sets four values output in four dimensions on the corresponding input image. It is determined by the coordinates of the face area. At this time, the coordinates of the face region are defined as the coordinates of the upper left vertex and the lower right vertex when the area containing the face is displayed as a rectangular bounding box, or the coordinates of the upper right vertex and the lower left vertex. It can be defined as the coordinates of.

The second face detection unit 320 receives input images determined to include a face region by the first face detection unit 310 and coordinates of a face region on the input images, and on the input images The feature of the first sub input images is extracted by applying a convolution operation to the first sub input images corresponding to the coordinates of the face region, and the face region is secondarily extracted on the first sub input images based on the extracted feature. do.

To this end, as shown in FIG. 4C, the second face detection unit 320 includes n convolutional calculation units 321a to 321c, second to third sampling units 323a and 323b, and a first dimensional increase unit 325. ), the third and fourth dimension reduction units 327a and 327b, and the second probability value calculation unit 328.

In one embodiment, the second face detection unit 320 may include three convolution calculation units 321a to 321c. In FIG. 4C, for convenience of description, the second face detection unit 320 includes three convolution calculation units 321a to 321c, but this is only one example, and the second face detection unit 320 is the first face The number of convolution calculation units 311a to 312c included in the detection unit 320 may be included.

Each of the fourth to sixth convolution operation units 321a to 312c generates a feature map by applying a convolution filter to the input image, and reflects nonlinear characteristics to the feature map by applying an activation function to the generated feature map. In one embodiment, the activation functions used by the fourth to sixth convolution calculation units 321a to 312c output positive values of pixel values of the feature map and negative values of values reduced by a predetermined size. It may be an activation function.

The second sampling unit 323a extracts feature values from the feature map by applying a sampling filter to the feature map output from the fourth convolution operator 321a, and the third sampling unit 323b is the fifth convolution operator 321b ) To extract feature values from the feature map by applying a sampling filter to the feature map. In one embodiment, the second and third sampling units 323a and 323b may extract a maximum value among pixel values included in a region corresponding to a sampling filter on each feature map as a feature value of the feature map. . According to this embodiment, the second and third sampling units 323a and 323b may be implemented as a maxpooling layer, and the dimension of each feature map is reduced through the maxpooling layer.

The first dimension increasing unit 325 increases the dimension of the feature map using a plurality of nodes so that the feature map output from the sixth convolution operation unit 321c has a dimension of a predetermined size. In one embodiment, the first dimensional increase unit 325 may increase the dimension such that the feature map output from the sixth convolution operation unit 321c has a size of 128*128 or a size of 256*256.

The first dimensional increase unit 325 reflects the non-linear characteristic in the feature map with increased dimension by applying an activation function to the feature map with increased dimension. In one embodiment, the first dimension increasing unit 325 applies the activation function to output a positive value of the pixel values of the feature map as it is and a negative value reduced by a predetermined size to apply the feature map. Can reflect nonlinear characteristics.

The third dimension reduction unit 327a reduces the dimension of the feature map output from the first dimension increase unit 325 by applying a third dimension reduction filter to the feature map output from the first dimension increase unit 325. In one embodiment, the third dimension reduction filter may be set as a filter capable of reducing the feature map in two dimensions.

The second probability value calculating unit 329 applies a predetermined classification function to the 2D output data output by the 3D reducing unit 327a to determine whether a face region is included in the corresponding first sub-input image. 2 Calculate the probability value. In one embodiment, when the calculated second probability value is greater than or equal to the second threshold value greater than the first threshold value, the second probability value calculating unit 328 may determine that the face region is included in the corresponding first sub-input image.

When the second probability value calculated by the second probability value calculation unit 329 is greater than or equal to the second threshold, the fourth dimension reduction unit 327b applies a fourth dimension reduction filter to the feature map output from the first dimension increase unit 325. By applying, the dimension of the feature map output from the first dimension increasing unit 325 is reduced. In one embodiment, the fourth dimension reduction filter may be set as a filter capable of reducing the feature map to four dimensions, and the fourth dimension reduction unit 327b may apply four values output in four dimensions to the corresponding first sub It is determined by the coordinates of the face area on the input image. At this time, the coordinates of the face area are defined as the coordinates of the upper left corner and the lower right corner when the area containing the face is displayed as a rectangular bounding box, or the coordinates of the upper right corner and the lower left corner. Can be.

The third face detection unit 330 receives first sub-input images determined by the second face detection unit 310 to include the face region and coordinates of the face region on the first sub-input images. On the first sub-input images, features of the second sub-input images are extracted by applying a convolution operation to the second sub-input images corresponding to the coordinates of the face region, and the second sub-input image is based on the extracted features. Face regions are extracted tertiarily on the field.

To this end, the third face detection unit 330 includes n+1 convolution calculation units 331a to 331d, fourth to sixth sampling units 333a to 333c, and a second dimensional increase unit as shown in FIG. 4D. 327), a fifth to sixth dimensional reduction units 337a to 337c, and a third probability value calculating unit 339.

In one embodiment, the third face detection unit 330 may include four convolution calculation units 331a to 331d. In FIG. 4D, for convenience of description, the third face detection unit 330 includes four convolution calculation units 331a to 331d, but this is only one example, and the third face detection unit 330 is the second face If the number of convolution calculation units that is greater than or equal to the number of convolution calculation units 321a to 322c included in the detection unit 320 is included, the number may not be limited.

Each of the seventh to tenth convolution operation units 331a to 331d applies a convolution filter to the input image to generate a feature map, and applies an activation function to the generated feature map to reflect the nonlinear characteristics in the feature map. In one embodiment, the activation functions used by the seventh to tenth convolutional calculation units 331a to 331d output positive values of pixel values of the feature map and negative values of values reduced by a predetermined size. It may be an activation function.

The fourth sampling unit 333a extracts feature values from the feature map by applying a sampling filter to the feature map output from the seventh convolution operation unit 331a, and the fifth sampling unit 333b is an eighth convolution operation unit 331b ) To extract the feature values from the feature map by applying a sampling filter to the feature map, and the sixth sampling unit 333c applies the sampling filter to the feature map output from the ninth convolution operation unit 331c to apply the feature. Feature values are extracted from the map. In one embodiment, the fourth to sixth sampling units 333a to 333c may extract a maximum value among pixel values included in a region corresponding to a sampling filter on each feature map as a feature value of the feature map. . According to this embodiment, the fourth to sixth sampling units 333a to 333c may be implemented as a maxpooling layer, and the dimension of each feature map is reduced through the maxpooling layer.

The second dimension increase unit 335 increases the dimension of the feature map by using a plurality of nodes so that the feature map output from the tenth convolution operation unit 331d has a dimension of a predetermined size. In one embodiment, the second dimensional increase unit 335 may increase the dimension such that the feature map output from the tenth convolution operation unit 331d has a size of 128*128 or a size of 256*256.

The second dimensional increase unit 335 reflects the non-linear characteristic to the feature map with increased dimension by applying an activation function to the feature map with increased dimension. In one embodiment, the second dimension increasing unit 335 applies a feature map by applying an activation function that outputs a positive value as it is among the pixel values of the feature map and a negative value by a predetermined size. Can reflect nonlinear characteristics.

The fifth dimension reduction unit 337a reduces the dimension of the feature map output from the second dimension increase unit 335 by applying a fifth dimension reduction filter to the feature map output from the second dimension increase unit 335. In one embodiment, the fifth dimension reduction filter may be set as a filter capable of reducing the feature map in two dimensions.

The third probability value calculating unit 339 applies a predetermined classification function to the 2D output data output by the fifth dimensional reduction unit 337a to determine whether a face region is included in the corresponding second sub-input image. 3 Calculate the probability value. In one embodiment, the third probability value calculating unit 339 determines that the face region is included in the second sub-input image if the calculated third probability value is greater than or equal to the third threshold value greater than the second threshold value.

When the third probability value calculated by the third probability value calculation unit 339 is greater than or equal to the third threshold, the sixth dimension reduction unit 337b applies a sixth dimension reduction filter to the feature map output from the second dimension increase unit 335. By applying, the dimension of the feature map output from the second dimension increasing unit 335 is reduced. In one embodiment, the sixth dimension reduction filter may be set as a filter capable of reducing the feature map to four dimensions, and the sixth dimension reduction unit 337b may apply four values output in four dimensions to the corresponding second sub It is determined by the coordinates of the face area on the input image. At this time, the coordinates of the face area are defined as the coordinates of the upper left corner and the lower right corner when the area containing the face is displayed as a rectangular bounding box, or the coordinates of the upper right corner and the lower left corner. Can be.

The sixth dimensional reduction unit 337b extracts the face image on the second sub-input image determined to include the face region using the calculated face region coordinates.

When the third probability value calculated by the third probability value calculation unit 339 is greater than or equal to the third threshold, the seventh dimension reduction unit 337c applies a seventh dimension reduction filter to the feature map output from the second dimension increase unit 335. By applying, the dimension of the feature map output from the second dimension increasing unit 335 is reduced. In one embodiment, the seventh dimension reduction filter may be set as a filter capable of reducing the feature map to ten dimensions, and the seventh dimension reduction unit 337c may apply ten values output in ten dimensions to the corresponding second sub Determined by landmark coordinates on the input image. At this time, the landmark coordinates are the coordinates of the two eyes on the second sub-input image 350 (420, 425), the coordinates of the nose 430, and the coordinates of the two mouths 440, 450 as shown in FIG. 3C. ), and the coordinates of the two mouths 440 and 450 mean the coordinates 440 for the left tail of the mouth and the coordinates 450 for the right tail of the mouth.

As described above, according to the present invention, the face image extraction unit 300 is implemented as the first to third face detection units 310 to 330, and the first face detection unit 310 is provided to the second face detection unit 320. It is composed of a network of shallow depths, and the second face detection unit 320 is composed of a network of shallow depths compared to the third face detection unit 330, so that the face image extraction unit 300 is entirely shallow-to -Deep structure to be formed in stages. Through this, it is possible to improve the accuracy of face image extraction and at the same time reduce the complexity, thereby gaining a benefit in terms of face recognition speed.

The face image alignment unit 340 aligns the face image using the landmark coordinates output from the third face detection unit 330. Specifically, the face image alignment unit 340 aligns the face image by performing at least one of rotation, translation, and enlargement and reduction on the face image using landmark coordinates of the extracted face image. The reason for aligning the face images using the face image alignment unit 340 in the present invention is to improve the face recognition performance by giving consistency to the face images to be provided as input to the feature vector extraction unit 229.

In one embodiment, the face image alignment unit 340 resizing the face image extracted by the third face detection unit 330 to the same resolution as the face image used in the feature vector extraction unit 229. , By moving the landmark coordinates calculated by the third face detection unit 330 based on the reference landmark coordinates for the face image of the resolution used by the feature vector extracting unit 229, rotate, translate the face image, It can be enlarged or reduced.

Referring to FIG. 2 again, when a face image is input from the face image extractor 227, the feature vector extractor 229 extracts the feature vector from the face included in the face image. Hereinafter, a feature vector extracting unit according to the present invention will be described in more detail with reference to FIG. 5.

5 is a block diagram schematically showing the configuration of a feature vector extraction unit according to the present invention. As illustrated in FIG. 5, the feature vector extraction unit according to an embodiment of the present invention includes a plurality of face image processing units 510a to 510n and a feature vector generation unit 520.

The plurality of face image processing units 510a to 510n process image data of the input data to generate output data. At this time, a face image output from the face image extraction unit 227 is input to the first face image processing unit 510a among the plurality of face image processing units 510a to 510n, and an n+1th face image processing unit 510n The output data of the nth face image processing unit 510n is input as +1) as an input image.

For example, the first face image processor 510a processes the face image to generate output data, and generates the output data twice?? Input to the face image processing unit 510b. The first face image processing unit 510b image-processes the output data output from the first face image processing unit 510a to generate new output data, and generates the new output data 3 times?? Input to the face image processing unit 510c.

Since the functions and detailed configurations of the plurality of face image processing units 510a to 510n shown in FIG. 5 are the same, hereinafter, as an example of the first face image processing unit 510a among the plurality of face image processing units 510a to 510n Explain. Hereinafter, for convenience of description, the first face image processing unit 510a will be referred to as a face image processing unit 510.

As shown in FIG. 5, the face image processing unit 510 performs a convolution operation on input data (which is an output data of a face image or a previous face image processing unit) to generate a feature map, a first unit 512, a first It is composed of a second unit 514 for assigning weights to the feature map generated by the unit 512, and a calculation unit 516.

Hereinafter, the configuration of the first unit 512 will be described in more detail with reference to FIG. 6. 6 is a block diagram showing the configuration of the first unit included in the face image processing unit. As shown in FIG. 6, the first unit 512 includes a normalization unit 600, a first convolution operation unit 610, and a nonlinearization unit 620.

The normalization unit 600 normalizes face images input from the face image extraction unit 227 in a batch unit. The arrangement refers to the number of face images to be processed at one time. The reason that the normalization unit according to the present invention performs normalization in a batch unit is that when the normalization is performed in a batch unit, the average and variance for each face image may be different from the average and variance for the entire batch. This is because the overall performance can be improved.

In addition, the complexity of learning increases and the gradient vanishing or exploding occurs due to the internal covariance shift phenomenon, in which the distribution of input is inconsistently changed for each layer of the network through batch normalization. It is because it can prevent.

The first convolution operation unit 610 applies the first convolution filter to the face image normalized by the normalization unit 600 to generate a first feature map. In one embodiment, the first convolution operator 610 may generate a first feature map by applying a first convolution filter having a 3*3 pixel size and a stride value of 1 to the face image. As described above, since the first convolution operation unit 610 according to the present invention applies a first convolution filter having a 3*3 pixel size and a stride value of 1 to the face image, it is possible to preserve a high resolution of the first feature map. do.

The non-linearization unit 620 imparts a non-linear characteristic to the first feature map by applying an activation function to the first feature map. In one embodiment, the non-linearization unit 620 outputs the same positive pixel value among the pixel values of the first feature map, and the negative pixel value decreases the size to output the activation function to the first feature map. By applying, it is possible to impart nonlinear characteristics to the first feature map.

In FIG. 6, it has been described that the nonlinearization unit 620 imparts nonlinear characteristics to the first feature map generated by the first convolution operation unit 610. However, in the modified embodiment, the normalization unit 600 may further normalize the first feature maps generated by the first convolution operation unit 610 in a batch unit. In accordance with this embodiment, the normalization unit 600 provides the normalized first feature map to the nonlinearization unit 620, and the nonlinearization unit 620 applies the activation function to the normalized first feature map to normalize the first It is possible to impart nonlinear characteristics to the feature map.

Meanwhile, in the above-described embodiment, it has been described that the first unit 512 includes only the first convolution operation unit 610. However, in the modified embodiment, the first unit 512 may further include a second convolution operation unit 630 as illustrated in FIG. 6.

Specifically, the second convolution operation unit 630 generates a second feature map by applying a second convolution filter to the first feature map to which the nonlinear characteristic is assigned by the nonlinearization unit 620. In one embodiment, the second convolution filter may be a different filter from the first convolution filter. The second convolution filter may be a filter having the same stride value but different stride values. As an example, the second convolution filter may be a filter having a 3*3 pixel size and a stride value of 2.

According to this embodiment, the normalization unit 600 may further normalize the second feature maps generated by the second convolution operation unit 630 in a batch unit.

Meanwhile, although not illustrated in FIG. 6, the first unit 512 may further include a pre-normalization unit. The pre-normalization unit may normalize pixel values of each pixel included in the face image input from the face image extraction unit. For example, the pre-normalization unit may normalize the face image by subtracting 127.5 from each pixel value of the face image, and then subtracting the subtraction result value by 128. The pre-normalization unit may provide the pre-normalized input face image to the normalization unit 600.

Referring again to FIG. 5, the second unit 514 weights the second feature map generated by the first unit 512. In the present invention, the reason for weighting the second feature map through the second unit 514 is that, in the case of convolution, all channels of the input image are multiplied by the convolution filter, and then all important channels and non-critical channels are added. Since the sensibility of data is reduced because of entanglement, the second feature map is weighted according to the importance of each channel.

Hereinafter, the configuration of the second unit 514 will be described in more detail with reference to FIG. 7. 7 is a block diagram showing the configuration of the second unit included in the face image processing unit. As shown in FIG. 7, the second unit 514 includes a sampling unit 710, a weight reflecting unit 720, and an upscaling unit 730.

First, the sampling unit 710 reduces the dimension by sub-sampling the second feature map input from the first unit 512. In one embodiment, the sampling unit 710 may reduce the dimension of the second feature map by applying a global pooling filter to the second feature map. For example, when the dimension of the second feature map is H*W*C, the sampling unit 710 may reduce the dimension of the second feature map to 1*1*C through subsampling of the second feature map.

The weight reflector 720 assigns weights to the second feature map whose dimensions are reduced by the sampling unit 710. To this end, as illustrated in FIG. 7, the weight reflecting unit 720 includes a dimensional reduction unit 722, a first nonlinearization unit 724, a dimensional increase unit 726, and a second nonlinearization unit 728. Can.

The dimension reduction unit 722 reduces the dimension of the sub-sampled second feature map by connecting the sub-sampled second feature map to one layer. For example, when the dimension of the second feature map output from the sampling unit 710 is 1*1*C, the dimension reduction unit 722 reduces the dimension of the second feature map to 1*1*C/r. Here, r means a reduction rate, and may be determined according to the number of feature vectors to be extracted.

The first non-linearization unit 724 imparts a non-linear characteristic to the second feature map with reduced dimensions by applying the first activation function to the second feature map with reduced dimensions by the dimension reduction unit 722. In one embodiment, the first nonlinearization unit 724 reduces the dimension by applying a first activation function that outputs a positive pixel value of the pixel values of the second feature map as it is and a negative pixel value of 0. Nonlinear characteristics may be given to the second feature map.

The dimension increasing unit 726 increases the dimension of the second feature map to which the nonlinear characteristics are assigned by the first nonlinearization unit 724. For example, when the dimension of the second feature map to which the nonlinear characteristic is assigned is 1*1*c/r, the dimension increasing unit 726 increases the dimension of the second feature map back to 1*1*C.

The second non-linearization unit 728 applies the second activation function to the second feature map whose dimension is increased by the dimension increase unit 726 to give the non-linear characteristic to the second feature map whose dimension is increased. In one embodiment, the second activation function may be a different function from the first activation function. For example, the second non-linearization unit 728 applies a second activation function that causes a positive pixel value to converge to a predetermined value among pixel values of a second feature map whose dimension is increased, and outputs a negative pixel value to 0. By doing so, it is possible to impart a nonlinear characteristic to the second feature map having an increased dimension.

As such, the weight reflecting unit 720 according to the present invention is the second feature map through the dimension reducing unit 722, the first non-linearizing unit 724, the dimensional increasing unit 726, and the second non-linearizing unit 728). By assigning a weight to the, and creating a bottleneck section through the dimensionality reducing unit 722 and the dimensionality increasing unit 726, it is possible to limit model complexity and support generalization.

The upscaling unit 730 upscales the second feature map weighted by the weight reflector 720 to the same dimension as the face image input to the second unit 514. In one embodiment, when the dimension of the face image input to the second unit 514 is H*W*C, the upscaling unit 720 sets the dimension of the weighted second feature map to H*W*C. Upscaling to

Referring back to FIG. 5, the calculation unit 516 adds the upscaled second feature map output through the second unit 514 to the face image input to the first unit 512. In the present invention, the reason for summing the upscaled second feature map output from the second unit 514 through the operation unit 516 with the face image input to the first unit 512 is characteristic when the depth is deep in the convolutional neural network. This is to prevent this vain problem.

The feature vector generation unit 520 reduces a dimension by merging the feature maps output from the last face image processing units 510n among a plurality of face image processing units 510a to 510n into one layer to reduce a dimension, thereby generating a predetermined number of feature vectors. To create. In one embodiment, the feature vector generator 520 may output 128 or more feature vectors from the feature map output from the face image processor 510n. For example, the feature vector generation unit 520 may output 512 feature vectors from the feature map output from the face image processing unit 510n.

Referring to FIG. 2 again, the array file generation unit 230 generates an array for each user using the feature vector generated by the face recognition unit 220 and merges the generated arrays into one file. Create an array file. The array file generation unit 230 may store the generated array file in an array file database (not shown).

In one embodiment, the array generated by the array file generator 230 may be composed of a plurality of feature vectors obtained from each user's face image and a key value of each user. At this time, the key value of the user includes identification information of each user and access permission information of each user. As described above, the identification information of each user may be defined by each user's ID, name, telephone number, or employee number, and each user's access permission information may include information on each floor that each user can access. It can contain.

In one embodiment, the array file generation unit 230 may generate an array file for each location where the edge device 120 is installed. For example, the first array file may consist of arrays of users granted access to the first layer, and the second array file may consist of arrays of users granted access to the second layer. have. To this end, the array file generation unit 230 may also create arrays of each user separately for each user access area. For example, when the first user has permission to access the first and third floors, the array file generation unit 230 includes the first array and the first array including access permission information for the first user. A second array including access permission information for the third floor may be separately generated.

The reason why the array file generation unit 230 according to the present invention generates an array file for each place where the edge device 120 is installed is when the edge device 120 for authenticating a user's face is installed for each place, a specific place This is because it is only necessary to transmit the array file including the access permission information for the corresponding place to the edge device 120 installed in the array file and the management of the array file in the edge device 120 becomes easy.

In the above-described embodiment, the array file generation unit 230 is described as generating an array file for each location, but in a modified embodiment, the array file generation unit 230 is installed at all locations where the edge device 120 is installed. One array file including authority information for the generated object may be generated, and the generated array file may be transmitted to all edge devices 120.

The edge device registration unit 240 registers information of a plurality of edge devices 120 installed at each location in the edge device information database 242. In one embodiment, the edge device registration unit 240 may map identification information of each edge device 120 to a place where each edge device is installed and store the edge device information in the edge device information database 242. Here, the identification information of the edge device 120 may include the manufacturer and serial number of the edge device 120.

Meanwhile, the edge device registration unit 240 may receive an authentication record from the edge device 120 every predetermined period through the interface unit 270, and store the received access record in the edge device information database 242.

The access permission information management unit 250 changes access permission information assigned to each user or adds new access permission information. In one embodiment, the access permission information management unit 250 may separately grant access permission information for each user or grant access permission information in an organizational unit to which each user belongs.

Meanwhile, the central server 110 according to the present invention may further include a scheduler 260. The scheduler 260 performs a function of registering new users collectively whenever a predetermined period arrives or a predetermined event occurs. For example, in the above-described embodiment, it has been described that the user registration unit 210 performs a registration procedure of a new user when a registration request is generated from the user, but it is predetermined when the central server 110 includes the scheduler 260. When the time unit or a predetermined event occurs, the scheduler 260 starts the operations of the user registration unit 210, the input image generation unit 215, and the face recognition unit 220 so that the new user registration procedure is automatically performed. can do.

The interface unit 270 encrypts the array file generated by the array file generation unit 230 in a predetermined manner and transmits it to each edge device 120. In one embodiment, the interface unit 270 may encrypt the array file using a public key-based encryption algorithm and transmit it to each edge device 120.

Meanwhile, the interface unit 270 may transmit the encrypted array file to the edge device 120 according to the protocol agreed with the edge device 120.

Also, the interface unit 270 may receive an authentication record from each edge device 120 every predetermined period of time and provide it to the edge device 120.

The face recognition model training unit 280 generates a face recognition model 225 based on the convolutional neural network and trains the generated face recognition model 225. Specifically, the face recognition model training unit 280 continuously trains the convolutional neural network constituting the face recognition model 225 to generate an optimal face recognition model.

To this end, the face recognition model training unit 280 is a face image extraction training unit 282 for training the face image extraction unit 227, a feature vector extraction training unit 284 for training the feature vector extraction unit 229, And an error reduction unit 286 for reducing the error of the feature vector extraction unit 229.

The face image extraction training unit 282 trains the first to third face detection units 310 to 330 constituting the face image extraction unit 227 using a learning image. Specifically, the face image extraction training unit 282 inputs a plurality of training images having a predetermined size to the first face detection unit 310 having a structure as shown in FIG. 4B to include a face region in the training image. The first probability value and the face region coordinates are calculated, and the calculated first probability value and the face region coordinates are fed back to the first face detection unit 310 according to a back propagation algorithm to the first face detection unit 310. The filter coefficients and weights of the applied convolution filters are updated.

In FIG. 4B, since the first face detection unit 310 has been described as calculating only the first probability value for the corresponding image to include the face region and the coordinates of the face region on the corresponding image, the face image extraction training unit 282 calculates the calculated first It has been described that the probability value and the coordinates of the face region are fed back to the first face detection unit 310 using a backpropagation algorithm to update the filter coefficients and weights of the convolution filters applied to the first face detection unit 310.

However, in another embodiment, the face image extraction training unit 282 receives landmark coordinates from the first face detection unit 310 during training of the first face detection unit 310 to improve the accuracy of landmark coordinate extraction. Filters of the convolution filters applied to the first face detection unit 310 by additionally calculating and feeding back the calculated landmark coordinates to the first face detection unit 310 through a back propagation algorithm together with the first probability value and the face region coordinates Coefficients and weights may be updated.

According to this embodiment, the first face detection unit 310 outputs from the third convolution operation unit 311c by applying a dimensionality reduction filter to the feature map output from the third convolution operation unit 311c to obtain landmark coordinates A dimension reduction unit (not shown) for reducing the dimension of the feature map to 10 dimensions may be further included. At this time, the 10 values output in 10 dimensions are determined by the coordinates of the two eyes, which are landmarks, the coordinates of the nose, the coordinates of the left mouth tail, and the coordinates of the right mouth tail.

In addition, the face image extraction training unit 282 inputs a plurality of learning images having a predetermined size to the second face detection unit 320 having a structure as shown in FIG. 4C to include a face region in the learning image. 2 Probability values and face region coordinates are calculated, and the calculated second probability values and face region coordinates are fed back to the second face detection unit 320 using a backpropagation algorithm, and the convolution filters applied to the second face detection unit 320 Filter coefficients and weights are updated. At this time, the learning image input to the second face detection unit 320 may be selected as the learning image determined to include the face region by the first face detection unit 310.

In FIG. 4C, since the second face detection unit 320 has been described as calculating only the probability that the corresponding image includes a face region and coordinates of the face region on the corresponding image, the face image extraction training unit 282 calculates the calculated second probability value and It has been described that the face region coordinates are fed back to the second face detection unit 320 using a back propagation algorithm to update the filter coefficients and weights of the convolution filters applied to the second face detection unit 320.

However, in another embodiment, the face image extraction training unit 282 may also perform landmark coordinates from the second face detection unit 320 during training of the second face detection unit 320 to improve the accuracy of landmark coordinate extraction. Is calculated, and the calculated landmark coordinates are fed back to the second face detection unit 320 through the backpropagation algorithm together with the second probability value and the face region coordinates, thereby generating the convolution filters applied to the second face detection unit 320. The filter coefficients and weights may be updated.

According to this embodiment, the second face detection unit 320 applies the dimensionality reduction filter to the feature map output from the first dimensional increase unit 325 to obtain landmark coordinates, the first dimensional increase unit 325 A dimension reduction unit (not shown) for reducing the dimension of the feature map output from the 10 dimension may be further included. At this time, the 10 values output in 10 dimensions are determined by the coordinates of the two eyes, which are landmarks, the coordinates of the nose, the coordinates of the left mouth tail, and the coordinates of the right mouth tail.

In addition, the face image extraction training unit 282 inputs a plurality of learning images having a predetermined size to the third face detection unit 330 having a structure as shown in FIG. 4D to include a face region in the learning image. The third probability value, the face area coordinates, and the coordinates of the landmark are calculated, and the calculated third probability value, the face area coordinates, and the coordinates of the landmark are fed back to the third face detection unit 330 using a backpropagation algorithm. 3 The filter coefficients and weights of the convolution filters applied to the face detection unit 330 are updated. In this case, the learning image input to the third face detection unit 330 may be selected as a learning image determined by the second face detection unit 320 to include a face region.

The feature vector extraction training unit 284 trains the feature vector extraction unit 229 having a configuration as shown in FIGS. 5 to 7 using a learning image. Specifically, the feature vector extracting training unit 284 inputs a plurality of training images into a feature vector extracting unit 229 having a structure as shown in FIGS. 5 to 7 in a predetermined arrangement unit, thereby making features from each learning image. Extract the vector.

The feature vector extraction training unit 284 applies the extracted feature vectors to a predetermined classification function to predict a probability value in which the corresponding learning image will be included in a specific class, and between the predicted probability value (hereinafter referred to as'prediction value') and the actual value. By calculating the error and feeding the result back to the feature vector extraction unit 229 using a back propagation algorithm, the filter coefficients and weights of the convolution filters applied to the feature vector extraction unit 229 are updated.

On the other hand, the face recognition model training unit 280 according to the present invention can further improve the performance of the feature vector extraction unit 229 by further reducing the error generated during feature vector extraction through the error reduction unit 286.

Specifically, the error reduction unit 286 includes the predicted values and actual values based on the feature vectors extracted through the feature vector extraction unit 229 in the process of the feature vector extraction training unit 284 training the feature vector extraction unit 229. Reduce the error between values. Specifically, the error reduction unit 286 calculates the error between the predicted value and the actual value based on the feature vectors extracted by the feature vector extraction unit 229 from the training image, and the two-dimensional angle of each learning image so that the error can be reduced. Placed on a plane and using a probability value according to the placement result, the feature vector extraction training unit 284 can train the feature vector extraction unit 229.

The reason for learning the feature vector extraction unit 279 so that the face recognition model training unit 280 according to the present invention is to reduce the error through the error reduction unit 286 is the case of the general face recognition model as shown in FIG. 8. In spite of being the same person, if the lighting or environment where the face was photographed changes, an error such as inability to distinguish the same person occurs, and thus the feature vector that can reduce the error of face recognition through the error reducing unit 286 It is for training the feature vector extraction unit 229 to be extracted.

The error reduction unit 286 according to an embodiment of the present invention includes a face image arrangement unit 287 and a probability calculation unit 289 as illustrated in FIG. 2.

The face image arranging unit 287 arranges each learning image on a two-dimensional angular plane based on a plurality of feature vectors extracted by the feature vector extraction unit 229 for the learning image. Specifically, the face image arranging unit 287 calculates cosine similarities between learning images included in different classes, and calculates a reference angle that is a separation angle between each learning images according to the cosine similarities, thereby providing two-dimensional angles of the learning images. It is placed on a flat surface.

In the present invention, when the face image arranging unit 287 arranges the learning images in the vector space according to the distance between the respective learning images calculated based on the feature vectors of the respective learning images, the learning images are shown in FIG. 9. There is a limitation in that it is difficult to clearly distinguish between the same person and others when learning because the area 900 overlapping with each other is caused.

Accordingly, in the present invention, the face image arranging unit 287 calculates an angle between the training images included in different classes through cosine similarity, and places each training image on a two-dimensional angle plane based on the calculated angle. Is to do.

The probability calculating unit 289 changes the margin angle to be added to the reference angle calculated by the face image placement unit 287 on the 2D angular plane, and determines the probability that each learning image is included in the corresponding class for each variable margin angle. Calculate.

Specifically, as illustrated in FIG. 10, the probability calculating unit 289 varies the margin angles P1 and P2 added to the reference angles θ ₁ and θ ₂ between the learning images, and learns to overlap each other. Images are spaced apart on a two-dimensional angular plane. In one embodiment, the margin angles P1 and P2 may be determined according to a learning rate within a range greater than 0 and less than 90 degrees.

For example, if the learning rate increases, the margin angle may increase accordingly, and when the learning rate decreases, the margin angle may also decrease. At this time, the probability calculating unit 288 may vary the margin angle by a predetermined reference unit.

When the margin angle is added to the reference angle by the probability calculating unit 289, it can be seen that, as illustrated in FIG. 11, learning images having features overlapping each other in the vector space are arranged to be spaced apart from each other.

The probability calculation unit 289 calculates the probability that each learning image is included in the corresponding class for each margin angle added to the reference angle, and provides the calculated probability value to the feature vector extraction training unit 284 to extract the feature vector extraction training unit 284 ) Allows the feature vector extraction unit 229 to be trained based on the probability value calculated by the probability calculation unit 289. That is, the feature vector extraction training unit 284 updates the feature vector extraction unit by updating at least one of the coefficients and weights of the convolution filters applied to the feature vector extraction unit 229 based on the probability values calculated by the probability calculation unit 288. (279).

In one embodiment, the probability calculator 289 may calculate the probability that each learning image is included in the corresponding class for each margin angle using Equation 1 below.

In Equation 1, x denotes a reference angle, p denotes the margin angle, and n denotes the number of classes.

In one embodiment, the probability calculation unit 289 is a predetermined test in the feature vector extraction unit 229 trained by the feature vector extraction training unit 284 based on the probability value calculated by the probability calculation unit 289 When the face image is applied, the margin angle can be continuously varied until the error between the predicted value and the actual value becomes less than the reference value.

That is, the probability calculating unit 289 applies the predetermined test face image to the trained feature vector extracting unit 229 to determine the margin angle at the time when the error between the predicted value and the actual value calculated is less than or equal to the reference value as the final margin angle. Decide. At this time, the error between the predicted value and the actual value can be calculated using a cross entropy function.

When the error reduction is performed through the error reduction unit 286 as described above, as illustrated in FIG. 12, even when photographed in different environments or different lighting, the same person can be accurately classified.

In the above-described embodiment, the face image extraction training unit 282, the feature vector extraction training unit 284, and the error reduction unit 286 constituting the face recognition model training unit 280 are implemented by algorithm-type software. It can be mounted on the central server 110.

Referring back to FIG. 1, the edge device 120 recognizes the face of a target user who wishes to enter the corresponding place by using the face recognition model 225 which is disposed at a specific place and distributed by the central server 110. And performs a function of authenticating the access of the target user based on the recognition result.

In the present invention, the reason why the central server 110 does not perform the face recognition and authentication of the target user and the edge device 120 performs the face recognition and authentication of the target user is because the face recognition and authentication of the target user is performed by the central server. This is because, in the case of performing in 110, face recognition and authentication cannot be performed when a failure occurs in the central server 110 or the network, and as the number of users increases, the expansion of the expensive central server 110 is required.

Accordingly, the present invention applies the edge computing (Edge Computing) method to perform the face recognition and authentication of the target user in the edge device 120 to provide a normal face recognition service even if a failure occurs in the central server 110 or the network. It can improve the reliability of service provision, and even if the number of users increases, it is possible to reduce the cost of constructing the face recognition system 100 because there is no need to increase the expensive central server 110.

Hereinafter, the configuration of the edge device 120 according to the present invention will be described in more detail with reference to FIG. 13.

13 is a block diagram schematically showing the configuration of an edge device according to a first embodiment of the present invention. As illustrated in FIG. 13, the edge device 120 according to the first embodiment of the present invention includes a first photographing unit 1210, an input image generating unit 1250, a face recognition unit 1300, and an authentication unit 1310. ), a face recognition model 1320, an array file update unit 1330, a memory 1340, and an interface unit 1350.

When the target user to be authenticated approaches, the first photographing unit 1210 photographs the target user to generate a photographed image. The first photographing unit 1210 transmits the generated photographed image to the input image generating unit 1250.

The input image generating unit 1250 generates an input image to be used for face recognition from the target user's captured image transmitted from the first photographing unit 1210. Specifically, the input image generation unit 1250 generates images of a plurality of target users having different resolutions from the shot images of one target user by down-sampling or up-sampling a shot image of one target user to a predetermined step.

For example, the input image generator 1250 may generate a down-sampled target user image by applying a Gaussian pyramid to the target user's image, or an up-sampled target user image by applying a Laplacian pyramid to the target user image. .

When a plurality of target user images having different resolutions are generated, the input image generation unit 1250 inputs a plurality of images obtained by moving a predetermined pixel size window on the target user image for each target user image. It is created as an image. The input image generation unit 1250 inputs the generated input image to the face recognition unit 1300.

When the input image of the target user is received from the input image generation unit 1250, the face recognition unit 1300 inputs the received target user's input image into the face recognition model 1320 distributed from the central server 110 to target the face. An image is extracted, and a target feature vector is extracted from the extracted target face image. In particular, the face recognition model 1320 distributed from the central server 110 may be reduced in error by learning through the above-described error reduction unit 286.

In addition, the face recognition model 1320 may be updated every predetermined period. As an example, the edge device 120 receives a new face recognition model 1320 from the central server 110 whenever the face recognition model 1320 is updated by the central server 110, thereby pre-distributing the face recognition model. 1320 may be updated with a new face recognition model 1320.

Since the face recognition model 1320 used for target face image extraction and target feature vector extraction is the same as the face recognition model 225 illustrated in FIGS. 4 to 8, a detailed description thereof will be omitted.

In addition, the method for the face recognition unit 1300 to extract the target face image and the target feature vector from the input image of the target user using the face recognition model 1320 is the face recognition unit 220 included in the central server 110. Since it is the same as extracting the face image and feature vector using the face recognition model 225, a detailed description thereof will be omitted.

The authentication unit 1310 compares the target feature vector obtained by the face recognition unit 1300 with the feature vectors included in the array file received from the central server 110 to determine whether the target user is a legitimate user who can access the corresponding place. Whether it is authenticated. Hereinafter, a method of authenticating the target user by the authentication unit 1310 will be described in detail.

First, the authentication unit 1310 calculates a first result value squared by subtracting a target feature vector from each feature index included in the corresponding array for each index included in each array included in the array file. The authentication unit 1310 calculates a second result value by summing the first result values calculated for each index, and calculates a third result value obtained by subtracting the second result value from a predetermined reference value.

The authentication unit 1310 determines that the user mapped to the array having the largest third result value among the arrays included in the array file is the user most similar to the target user. At this time, when the third result value is greater than or equal to a predetermined threshold, the authentication unit 1310 authenticates the target user as a user with legitimate authority, and accordingly, the target user may be allowed to access the corresponding place.

In one embodiment, the threshold may be set differentially according to the security level of the place where the edge device 120 is installed. For example, when the edge device 120 is installed in an area to which a high security level is applied, the threshold may be set high, and when the edge device 120 is installed in an area to which a low security level is applied, the threshold value may be set low. You can.

Hereinafter, the method for authenticating whether the target user is a legitimate user who can access the floor by comparing the target feature vector obtained by the face recognition unit 1300 with the feature vectors included in the array file is as follows. For example, it will be described.

14 is a diagram exemplarily showing a method of authenticating a target user by an authentication unit. As shown in FIG. 14, in the array file, an array including feature vectors of each user is arranged for each row. For example, feature vectors for the first user are sequentially arranged in the first row, and feature vectors for the second user are sequentially arranged in the second row. At this time, the feature vectors of each user are arranged in one row according to the index order.

As shown in FIG. 14A, the authentication unit 1310 calculates a difference value between feature vectors and target feature vectors of each array of the array file 1410 for each index, and the difference value calculated as shown in FIG. 14B. The first result value is calculated by squaring them, and the second result value is calculated by summing all the first result values for each array as illustrated in FIG. 14C.

Thereafter, as illustrated in FIG. 14D, the authentication unit 1310 calculates a third result value by subtracting the second result value from a predetermined reference value (eg, 1), and is the largest of the calculated third result values. The user mapped to the array corresponding to the value of 0.310528 is determined as the user most similar to the target user. In addition, since the third result value determined as the largest value is greater than a predetermined threshold (eg, 0.15), the authentication unit 1310 finally authenticates the target user as a user mapped to the corresponding array.

When the method of authenticating the target user by the authentication unit 1310 is expressed by an equation, it can be expressed as Equation 2 below.

In Equation 2, Z represents a third result value, R represents a predetermined reference value, Xi represents a feature vector corresponding to the i-th index among n feature vectors, and Yi represents an i-th index among n feature vectors. Target feature vector corresponding to.

The face recognition model 1320 is generated and distributed by the central server 110, and the face recognition model 1320 is a face that is updated whenever the face recognition model 225 is trained and updated by the central server 110. It is replaced by the recognition model 225. In this case, the face recognition model 1320 may be received from the central server 110 through the interface unit 1350.

When the array file update unit 1330 receives the array file from the central server 110 through the interface unit 1350, uploads it to the first memory 1342, and the authentication unit 1310 uses this to authenticate the target user. Make it possible. In particular, the array file update unit 1330 according to the present invention can dynamically load the array file.

Specifically, when an array file is loaded in the first memory 1342, the array file update unit 1330 transfers the new array file to the second memory 1344 when a new array file is received from the central server 110. When loading, and the loading of the new ray file to the second memory 1344 is completed, the array file loaded in the first memory 1342 is replaced with the new array file loaded in the second memory 1344.

The reason why the array file update unit 1330 dynamically loads the array file as described above is because the authentication unit 1310 performs authentication processing for the target user and the array file update unit 1330 is a new array. This is to enable the edge device 120 to perform face recognition in real time based on the newly updated array file by allowing the file to be updated.

The array file used by the authentication unit 1310 is loaded into the first memory 1342, and the newly received new array file is loaded into the second memory 1344. When the loading of the new array file to the second memory 1344 is completed, the array file recorded in the first memory 1342 by the array file update unit 1330 is replaced with the new array file.

The interface unit 1350 mediates data transmission and reception between the edge device 120 and the central server 110. Specifically, the interface unit 1350 receives the face recognition model 1320 from the central server 110 and receives the array file from the central server 110 to receive the array file from the first memory 1342 through the array file update unit 1330. ) Or the second memory 1344. In addition, the interface unit 1350 periodically transmits the authentication record by the authentication unit 1330 to the central server 110.

In one embodiment, the array file and the face recognition model 1320 may be updated every predetermined period through the interface unit 1350.

As described above, according to the present invention, since the edge recognition device 120 stores only the face recognition model 1320 and the array file for face recognition, the edge device 120 is not stored. Even if it is hacked, there is no fear of leaking the user's personal information, thereby enhancing security.

In the case of the edge device 120 according to the first embodiment described above, it has been described that the photographed image generated by the first photographing unit 1210 is directly input to the input image generating unit 1250. However, according to this embodiment, even when a photograph including a face of a target user is photographed by the first photographing unit 1210, the first photographing unit 1210 generates a normal photographed image to input the image generator 1250. By passing to, there is a problem that a user who is not a target user may perform authentication using a picture of the target user.

Hereinafter, the edge device 120 according to the second embodiment capable of solving the above-described problems will be described in detail.

15 is a block diagram showing the configuration of an edge device according to a second embodiment of the present invention. The edge device according to the second embodiment illustrated in FIG. 15 further comprises a second imaging unit 1370 and a authenticity determination unit 1380 compared to the edge device according to the first embodiment illustrated in FIG. 13. It is distinguished from edge devices according to one embodiment. Hereinafter, for convenience of description, a description of a configuration having the same function as the edge device according to the first embodiment is omitted, and the newly added second photographing unit 1510 and the authenticity determination unit 1520 and the newly added Only the first photographing unit 1210 whose function is changed due to the configuration will be described.

The first photographing unit 1210 photographs an object to be photographed to generate a photographed image. The first photographing unit 1210 transmits the generated photographed image to the authenticity determination unit 1520.

The second photographing unit 1510 photographs an object to be photographed to generate a depth image. The second photographing unit 1510 is before or after a predetermined time from the time when the photographing object is photographed by the first photographing unit 1210 or when the photographing object is photographed in the first photographing unit 1210. You can shoot the subject.

In one embodiment, the second photographing unit 1510 may be implemented as an IR camera capable of generating a depth image by photographing a photographing target.

As described above, the reason why the edge device 120 according to the second embodiment photographs a shooting target through the second shooting unit 1510 to generate a depth image is that the actual face of the shooting target is generated by the second shooting unit 1510. This is because different types of depth images are generated when a picture including a face to be photographed is photographed.

For example, when a photograph including a face of a photographing target is photographed by the second photographing unit 1510, while a depth image having the form shown in FIG. 16A is generated, the photographing target is photographed by the second photographing unit 1510 When the actual face of is photographed, a second depth image having the form shown in FIG. 16B is generated.

The second photographing unit 1510 transmits the generated depth image to the authenticity determination unit 1520.

The authenticity determination unit 1520 determines whether the photographing object photographed by the second photographing unit 1510 is a photograph or a face of an actual photographing object using the depth image transmitted from the second photographing unit 1510.

Specifically, the authenticity determination unit 1520 extracts depth data from the depth image received from the second imaging unit 1510, and determines whether the depth image is the actual face of the object to be photographed through binary classification. . In one embodiment, the authenticity determination unit 1520 may improve classification accuracy for real faces and photos through training based on a deep learning algorithm.

The authenticity determination unit 1520 transmits the photographed image received from the first photographing unit 1210 to the input image generator 1250 when it is determined that the photographing target photographed by the second photographing unit 1510 is an actual face. . Meanwhile, the authenticity determination unit 1520 inputs the photographed image received from the first photographing unit 1210 when the photographing target photographed by the second photographing unit 1510 is not an actual face. Without sending to, it can be output by using the text or voice type alarm message that the authentication process has failed or notify the system operator that there was an abnormal access attempt.

As described above, according to the present invention, by accurately determining whether the photographed subject is a real face or a photograph, authentication processing is not performed when the photograph is taken, so that a user without a legitimate authority attempts to authenticate using a photograph of another person Attempts can be blocked at the source, thereby improving security.

Referring back to FIG. 1, the user terminal 130 transmits a user image for newly registering a user together with the user's identification information to the central server 110. In one embodiment, the user terminal 130 is equipped with a face registration agent (not shown) capable of interworking with the central server 110, and the user executes the face registration agent on the user terminal 130. The face image or the pre-image may be transmitted to the central server 110 together with the user identification information.

In one embodiment, the user terminal 130 may request to register a plurality of user images for each user. At this time, the plurality of images requested for registration for each user may be photographs taken in different environments or photographs taken under different lighting.

If the user terminal 130 can request user registration by transmitting a user image to the central server 110, any of them can be used without limitation. For example, the user terminal 130 may be implemented as a smartphone, laptop, desktop or tablet PC.

Those skilled in the art to which the present invention pertains will understand that the above-described present invention can be implemented in other specific forms without changing its technical spirit or essential features.

For example, the configuration of the central server shown in FIG. 2 and the configuration of the edge device shown in FIG. 13 may be implemented in a program form. When the configuration of the central server and the configuration of the edge device according to the present invention is implemented as a program, each configuration shown in FIGS. 2 and 13 is implemented as code, and codes for implementing a specific function are implemented as one program or , It may be implemented by dividing a plurality of programs.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modifications derived from the meaning and scope of the claims and equivalent concepts are included in the scope of the present invention. do.

100: face recognition system 110: central server
120: edge device 130: user terminal
210: user registration unit 215: input image generation unit
220: face recognition unit 225: face recognition model
230: array file generation unit 240: face recognition model training unit
250: interface

Claims (18)

A first photographing unit acquiring a photographed image of a target user to be authenticated;
A face recognition unit for inputting the input images generated from the photographed image into a face recognition model, extracting a target face image from the input image, and generating a target feature vector from the extracted target face image;
An authentication unit authenticating the target user by comparing the target feature vector with an array file composed of a plurality of arrays having user identification information and feature vectors extracted from the face image of the user; And
And an interface unit receiving the array file and the face recognition model from a central server. According to claim 1,
The authentication unit,
From the feature vector included in each array, the target feature vector is subtracted by the same index to calculate a squared first result value, and a second result value obtained by summing the first result values calculated for each index in the corresponding array is used. Edge device for face recognition, characterized in that to authenticate the target user. According to claim 2,
The authentication unit,
The third result value is calculated by subtracting the second result value from a predetermined reference value for each array, and a user mapped to the largest array of the third result values among the plurality of arrays included in the array file Edge device for face recognition, characterized in that it is determined to be the user most similar to the target user. According to claim 2,
The authentication unit,
An edge for face recognition characterized in that a third result value is calculated by subtracting the second result value from a predetermined reference value for each array, and authenticating the target user when the calculated third result value is greater than or equal to a predetermined threshold value. device. According to claim 4,
The threshold value is an edge device for face recognition characterized in that the differential is set in accordance with the security level of the place where the edge device is installed. According to claim 1,
When the array file is loaded in the first memory, when the new array file is received from the central server through the interface unit, the new array file is loaded into the second memory, and loading of the new array file into the second memory is performed. And an array file update unit that replaces the array file loaded in the first memory with a new array file loaded in the second memory when completed. According to claim 1,
And an input image generating unit generating up-sampling or down-sampling the photographed image photographed through the first photographing unit to generate a plurality of input images having different resolutions. According to claim 1,
The face recognition model,
A face image extraction unit extracting the target face image from the input image;
A plurality of face image processing units for processing the input data to generate output data; And
And a feature vector generator for merging the output data output from the last face image processor among the plurality of face image processors into a single layer to generate a predetermined number of feature vectors,
The target face image extracted by the face image extraction unit is input to the first face image processing unit of the plurality of face image processing units, and the n+1th face image processing unit is the n-th face as the input image. An edge device for face recognition, characterized in that output data of the image processing unit is input. The method of claim 8,
The face image extraction unit extracts the target face image from the input images by using two or more face detection units having neural network networks of different depths,
The two or more face detection units are sequentially arranged in the order of increasing depth, so that the number of input images input to the n-th face detection unit is reduced compared to the number of input images input to the n-1-th face detection unit. Edge device for face recognition. The method of claim 8,
The face image extraction unit,
A feature map of a plurality of input images is generated by using a first neural network network having a first depth, and a plurality of first sub-input images including a face region among the plurality of input images based on the feature map A first face detection unit that primarily screens them;
A feature map of the plurality of first sub-input images is generated using a second neural network network having a second depth deeper than the first depth, and the plurality of features are generated based on the feature map generated through the second neural network. A second face detection unit for secondarily selecting a plurality of second sub input images including a face region among the first sub input images; And
A feature map of the plurality of second sub-input images is generated using a third neural network network having a third depth deeper than the second depth, and the plurality of the feature maps are generated based on the feature map generated through the third neural network. And a third face detection unit that selects the target face image including the face region from among the second sub-input images. The method of claim 8,
The face image extraction unit,
Obtain landmark coordinates from the target face image, and perform at least one of rotation, translation, enlargement, and reduction on the target face image to match the obtained landmark coordinates with a predetermined reference landmark coordinate. An edge device for face recognition, further comprising a face image alignment unit for aligning the target face image. The method of claim 8,
The face image processing unit,
A normalization unit that normalizes the input data;
A first convolution operator that generates a first feature map by applying a first convolution filter to the normalized input data; And
A face characterized by including a non-linearization unit that gives a non-linear characteristic to the first feature map by outputting a positive value as it is among the pixel values of the first feature map and outputting a negative value by reducing its size. Edge device for recognition. The method of claim 12,
The face image processing unit,
Further comprising a second convolution operation unit for generating a second feature map by applying a second convolution filter to the first feature map to which the nonlinear characteristic is assigned,
The first convolution filter and the second convolution filter are face recognition edge devices characterized in that the filter having the same size and different stride values. The method of claim 8,
The face image processing unit,
A sampling unit sub-sampling the generated feature map to reduce the dimension of the feature map;
A weight reflecting unit reflecting a weight in the sub-sampled feature map; And
And an upscaling unit for upscaling the feature map in which the weight is reflected to the same dimension as the feature map input to the sampling unit. The method of claim 14,
The weight reflecting unit,
A dimension reduction unit that reduces the dimension by connecting the sub-sampled feature maps to one layer;
A first non-linearization unit which outputs a positive value as it is and outputs a negative value as 0 among the pixel values of the feature map in which the dimension is reduced, thereby giving a non-linear characteristic to the feature map in which the dimension is reduced;
A dimension increasing unit that increases a dimension of the feature map to which the nonlinear characteristic is given; And
Among the pixel values of the feature map having the increased dimension, a positive value converges to a predetermined value and a negative value is output as 0 to include a second non-linearization unit that gives non-linear characteristics to the feature map having the increased dimension. Edge device for face recognition, characterized in that. According to claim 1,
The face recognition model is learned through an error reduction algorithm,
The error reduction algorithm places the learning images on a two-dimensional angular plane based on a plurality of feature vectors obtained by inputting a learning image into the face recognition model, and adds them to a reference angle between learning images included in different classes. The edge device for face recognition, characterized in that by calculating a probability that each learning image is to be included in each class for each variable margin angle by varying the margin angle to be, and learning the face recognition model based on the calculated probability. . According to claim 1,
The array file and the face recognition model are updated every predetermined period, characterized in that the edge device for face recognition. According to claim 1,
A second photographing unit which photographs a photographing target and generates a depth image; And
Based on the depth image generated by the second photographing unit, it is determined whether the photographing target is a face or a photograph, and when it is determined that the photographing target is a face, the photographing image generated by the first photographing unit is inputted An edge device for face recognition, further comprising a authenticity determination unit to be generated as an image.

Images