KR102546327B1 - Edge device comparing face images using clustering technique and face authentication system having the same - Google Patents

Abstract

A face authentication system for comparing face images using a clustering technique according to an aspect of the present invention includes a user face recognition unit that extracts a plurality of user feature vectors by inputting a plurality of user images to a face authentication model, and each of K clusters. A feature vector clustering unit for clustering a plurality of user feature vectors based on the central value for , a target face recognition unit for extracting a target feature vector by inputting a target face image to a face authentication model, and K clusters for target feature vectors. and an authentication unit that authenticates whether or not the user is a legitimate user by comparing the feature vectors belonging to one of the user feature vectors.

Description

Edge device comparing face images using clustering technique and face authentication system including same

The present invention relates to an edge device and a face authentication system including the same.

Face authentication technology is one of the fields of biometrics, which means a technology that automatically identifies and authenticates a person by using the unique feature information contained in each person's face. It has excellent security compared to other authentication methods and is widely used in various fields.

A general face authentication system transmits a face image captured by a device installed at an entrance gate to a server, the server performs face recognition and user authentication according to the face recognition, and transmits the authentication result to the device to determine whether the gate is open. .

A typical face authentication system compares a pre-registered reference image with a target user image captured by a device to perform authentication for a corresponding target user. At this time, as the number of pre-registered reference images increases, the number of reference images to be compared with the target user image increases exponentially. Therefore, there is a problem in that authentication processing speed decreases.

SUMMARY OF THE INVENTION The present invention is to solve the above problems, and has as a technical task to provide an edge device capable of improving authentication processing speed and a face authentication system including the same.

A face authentication system for comparing face images using a clustering technique according to an aspect of the present invention includes a user face recognition unit that extracts a plurality of user feature vectors by inputting a plurality of user images to a face authentication model, and each of K clusters. A feature vector clustering unit for clustering a plurality of user feature vectors based on the central value for , a target face recognition unit for extracting a target feature vector by inputting a target face image to a face authentication model, and K clusters for target feature vectors. and an authentication unit that authenticates whether or not the user is a legitimate user by comparing the feature vectors belonging to one of the user feature vectors.

An edge device for comparing face images using a clustering technique according to another aspect of the present invention includes a photographing unit acquiring a photographed image of a target user to be authenticated, and inputting an input image generated based on the photographed image to a face authentication model. A target face recognition unit for extracting target feature vectors, and a plurality of target feature vectors having feature vectors extracted from face images of each of a plurality of users, cluster identification information of each feature vector, and identification information of each user. and an authentication unit for authenticating the target user by comparing the array file with an array file.

According to the present invention, user feature vectors may be clustered into a plurality of clusters, and target feature vectors may be compared only with user feature vectors belonging to clusters having a high degree of similarity. Accordingly, the present invention can greatly reduce the number of comparisons and improve the authentication processing speed.

1 is a block diagram schematically showing the configuration of a face authentication system according to an embodiment of the present invention.
2 is a block diagram schematically showing the configuration of a face recognition server according to an embodiment of the present invention.
FIG. 3 is a block diagram schematically showing the configuration of the feature vector clustering unit of FIG. 2 .
4 is a block diagram schematically showing the configuration of an edge device according to an embodiment of the present invention.
5 is a flowchart illustrating a method of clustering user feature vectors in a face recognition system according to an embodiment of the present invention.
6 is a flowchart illustrating a method of authenticating a target user in a face recognition system according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The meaning of terms described in this specification should be understood as follows.

Singular expressions should be understood to include plural expressions unless the context clearly defines otherwise, and terms such as “first” and “second” are used to distinguish one component from another, The scope of rights should not be limited by these terms.

It should be understood that terms such as "comprise" or "having" do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

The term “at least one” should be understood to include all possible combinations from one or more related items. For example, "at least one of the first item, the second item, and the third item" means not only the first item, the second item, or the third item, but also two of the first item, the second item, and the third item. It means a combination of all items that can be presented from one or more.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of a face authentication system according to an embodiment of the present invention, FIG. 2 is a block diagram schematically showing the configuration of a face recognition server according to an embodiment of the present invention, FIG. is a block diagram schematically showing the configuration of the feature vector clustering unit in FIG. 2 .

Referring to FIGS. 1 to 3 , a face authentication system 100 according to an embodiment of the present invention includes a face recognition server 110 and a plurality of edge devices 120 .

The face recognition server 110 creates a face recognition model, and using the generated face recognition model, an array file for authentication of a target user using a feature vector extracted from the user's face information input from the user terminal 130. (Array File) is created. Then, the face recognition server 110 transmits the generated array file to the edge device 120 so that the edge device 120 can authenticate the target user.

To this end, the face recognition server 110 includes an edge device management unit 210, a user registration unit 220, a user database 222, an input image generator 225, and a user face recognition unit (as shown in FIG. 2). 230), face recognition model 235, feature vector clustering unit 240, array file generation unit 245, access authority information management unit 250, interface unit 270, face recognition model training unit 280 included. can do.

The edge device management unit 210 registers information on the plurality of edge devices 120 installed in each place in the edge device database 219 . In an embodiment, the edge device management unit 210 may store identification information of each edge device 120 in the edge device database 219 by mapping the location where each edge device is installed. Here, the identification information of the edge device 120 may include a manufacturer and serial number.

Meanwhile, the edge device management unit 210 may receive authentication records from the edge device 120 at predetermined intervals through the interface unit 270 and store the received access records in the edge device information database 242.

The user registration unit 220 receives one or more user images from the user terminal 130 of a user who wishes to register. When the user image is received, the user registration unit 220 checks whether the corresponding user is the same person as the user image, and if it is determined that the user image is the same person, obtains information on access rights granted to the corresponding user, and stores the user database 222 together with the user image. ) to register.

The input image generation unit 225 generates an input image to be used for face recognition from the user image input by the user registration unit 220 . The input image generation unit 225 may generate a plurality of images obtained by moving a window having a predetermined pixel size on the user image as an input image for each user image. The input image generation unit 225 may input the generated plurality of input images to the face recognition unit 230 .

The user's face recognition unit 230 inputs a plurality of input images generated by the input image generation unit 225 to the face recognition model 235 trained by the face recognition model training unit 280 so that the face region is included. Acquire the user's face image. The user face recognition unit 230 extracts a user feature vector from the obtained user face image.

In an embodiment, the face recognition model 230 may include a face image extractor 237 that extracts a face image from an input image, and a feature vector extractor 239 that extracts a feature vector from the face image.

The face image extraction unit 237 according to the present invention is configured based on a convolutional neural network (CNN) and extracts a face image including a face region from an input image. The face image extraction unit 237 may include a first face detection unit, a second face detection unit, a third face detection unit, and a face image alignment unit.

The first face detection unit extracts features of each input image by applying a convolution operation to the input image input by the face recognition unit 230, and primarily extracts a face region from the corresponding input image based on the extracted feature. can To this end, the first face detection unit may include n convolution operation units, a sampling unit, first and second dimension reduction units, and a first probability value operation unit.

In one embodiment, the first face detection unit may include three convolution operation units. For convenience of explanation, it is described that the first face detection unit includes three convolution operation units, but this is only an example, and the first face detection unit may include four or more convolution operation units, or one or two convolution operation units. There will be.

Each of the n convolution operation units generates a feature map by applying a convolution filter to an input image, and reflects nonlinear characteristics to the feature map by applying an activation function to the generated feature map. In this case, the convolution filters applied to the first to third convolution operation units may be different from each other.

In one embodiment, the activation function used in the first to third convolution operation units may be an activation function that outputs positive values of pixel values of the feature map as they are and outputs negative values as values reduced by a predetermined size. there is. Here, the activation function means a function that outputs a completed result value by assigning a weight to a plurality of input information and combining them.

The sampling unit extracts feature values from the feature map by applying a sampling filter to the feature map output from the first convolution operation unit. In an embodiment, the sampling unit may extract, as a feature value of the feature map, a maximum value among pixel values included in a region corresponding to a sampling filter on the feature map. According to this embodiment, the sampling unit may be implemented as a max pooling layer, and the dimensionality of the feature map is reduced through the max pooling layer. The sampling unit inputs the dimension-reduced feature map to the second convolution operation unit.

The first dimension reduction unit reduces the dimensionality of the feature map output from the third convolution operation unit by applying a first dimension reduction filter to the feature map output from the third convolution operation unit. In one embodiment, the first dimensionality reduction filter may be set as a filter capable of reducing a feature map to two dimensions.

The first probability calculation unit calculates a first probability value as to whether or not the face region is included in the corresponding input image by applying a predetermined classification function to the 2-dimensional output data output by the first dimension reduction unit. In an embodiment, the first probability value calculation unit may determine that the face region is included in the input image when the calculated first probability value is greater than or equal to the first threshold value.

The second dimensionality reduction unit applies a second dimensionality reduction filter to the feature map output from the third convolution operation unit when the first probability value calculated by the first probability value operation unit is equal to or greater than the first threshold, and thereby the feature map output from the third convolution operation unit. reduces the dimension of In an embodiment, the second dimension reduction filter may be set as a filter capable of reducing a feature map to a 4-dimensional dimension, and the 2-dimensional reduction unit converts four values output in 4 dimensions to coordinates of the face region on a corresponding input image. to decide At this time, the coordinates of the face area are defined as the coordinates of the upper left vertex and the lower right vertex when the area including the face is displayed as a rectangular bounding box, or the coordinates of the upper right vertex and the lower left vertex It can be defined as the coordinates of

The second face detection unit receives input images determined to include the face area by the first face detection unit and the coordinates of the face area on the corresponding input images, and receives input images corresponding to the coordinates of the face area on the corresponding input images. A feature of the first sub-input images is extracted by applying a convolution operation to the first sub-input images, and a face region is secondarily extracted from the first sub-input images based on the extracted feature.

To this end, the second face detection unit includes n convolution operation units, second to third sampling units, a first dimension increasing unit, third and fourth dimension reducing units, and a second probability value calculating unit.

In one embodiment, the second face detection unit may include three convolution operation units. For convenience of explanation, it is described that the second face detection unit includes three convolution operation units, but this is only an example. The second face detection unit may include more than the number of convolution operation units included in the first face detection unit. there is.

Each of the fourth to sixth convolution operation units generates a feature map by applying a convolution filter to an input image, and reflects nonlinear characteristics to the feature map by applying an activation function to the generated feature map. In one embodiment, the activation function used by the fourth to sixth convolution operators may be an activation function that outputs positive values of the pixel values of the feature map as they are and outputs negative values as values reduced by a predetermined size. .

The second sampling unit applies a sampling filter to the feature map output from the fourth convolution operation unit to extract feature values from the corresponding feature map, and the third sampling unit applies the sampling filter to the feature map output from the fifth convolution operation unit to extract the corresponding feature. Extract feature values from the map. In an embodiment, the second and third sampling units may extract a maximum value among pixel values included in a region corresponding to a sampling filter on each feature map as a feature value of the feature map. According to this embodiment, the second and third sampling units may be implemented as a max pooling layer, and the dimension of each feature map is reduced through the max pooling layer.

The first dimension increasing unit increases the dimensionality of the feature map output from the sixth convolution operator by using a plurality of nodes so that the feature map has a dimension of a predetermined size. In one embodiment, the first dimension increasing unit may increase the dimension of the feature map output from the sixth convolution operation unit to have a size of 128*128 or 256*256.

The first dimension increasing unit applies an activation function to the dimensionally increased feature map to reflect nonlinear characteristics to the dimensionally increased feature map. In one embodiment, the first dimension increaser applies an activation function that outputs positive values of pixel values of the feature map as they are and outputs negative values as values reduced by a predetermined size, thereby providing nonlinear characteristics to the feature map. can reflect

The third dimension reduction unit reduces the dimensionality of the feature map output from the first dimension augmentation unit by applying a third dimension reduction filter to the feature map output from the first dimension augmentation unit. In one embodiment, the 3rd dimension reduction filter may be set as a filter capable of reducing a feature map to 2D.

The second probability calculation unit calculates a second probability value for whether or not the face region is included in the corresponding first sub-input image by applying a predetermined classification function to the 2-dimensional output data output by the 3-dimensional reduction unit. In an embodiment, the second probability calculator may determine that the face region is included in the first sub input image when the calculated second probability value is greater than or equal to a second threshold value greater than the first threshold value.

The fourth-dimensional reducer applies a fourth-dimensional reduction filter to the feature map output from the first-dimensional increaser when the second probability value calculated by the second probability value calculation unit is equal to or greater than the second threshold, thereby outputting the output from the first-dimensional increaser. Reduce the dimensionality of feature maps. In an embodiment, the 4th dimension reduction filter may be set as a filter capable of reducing the feature map to 4D, and the 4th dimension reduction unit converts the 4 values output in 4D to the corresponding first sub input image. It is determined by the coordinates of the face area. At this time, the coordinates of the face area are defined as the coordinates of the upper left vertex and the lower right vertex when the area including the face is displayed as a rectangular bounding box, or the coordinates of the upper right vertex and the lower left vertex. It can be.

The third face detection unit receives the coordinates of the face area on the first sub-input images determined to include the face area by the second face detection unit and the corresponding first sub-input images, and receives the corresponding first sub-input image A feature of the second sub-input images is extracted by applying a convolution operation to the second sub-input images corresponding to the coordinates of the face area on the , and the face area is formed on the second sub-input images in a third order based on the extracted feature. extract with

To this end, the third face detection unit includes n+1 convolution operation units, fourth to sixth sampling units, a second dimension increasing unit, a fifth to sixth dimension reducing unit, and a third probability value calculating unit.

In one embodiment, the third face detection unit may include four convolution operation units. For convenience of explanation, it is described that the third face detection unit includes four convolution operation units, but this is only an example if the third face detection unit includes more than the number of convolution operation units included in the second face detection unit. The number may be unlimited.

Each of the seventh to tenth convolution operation units generates a feature map by applying a convolution filter to an input image, and applies an activation function to the generated feature map to reflect nonlinear characteristics to the feature map. In one embodiment, the activation function used by the seventh to tenth convolution operators may be an activation function that outputs positive values of the pixel values of the feature map as they are and outputs negative values as values reduced by a predetermined size. .

The fourth sampling unit applies a sampling filter to the feature map output from the seventh convolution operation unit to extract feature values from the corresponding feature map, and the fifth sampling unit applies the sampling filter to the feature map output from the eighth convolution operation unit to extract the corresponding feature. Feature values are extracted from the map, and the sixth sampling unit extracts feature values from the corresponding feature map by applying a sampling filter to the feature map output from the ninth convolution operation unit. In one embodiment, the fourth to sixth sampling units may extract a maximum value among pixel values included in a region corresponding to a sampling filter on each feature map as a feature value of the feature map. According to this embodiment, the fourth to sixth sampling units may be implemented as a max pooling layer, and the dimension of each feature map is reduced through the max pooling layer.

The second dimension increaser increases the dimension of the feature map output from the tenth convolution operator by using a plurality of nodes so that the feature map has a dimension of a predetermined size. In one embodiment, the second dimension increaser may increase the dimensionality of the feature map output from the tenth convolution operator to have a size of 128*128 or 256*256.

The second dimension increasing unit applies an activation function to the dimensionally increased feature map to reflect nonlinear characteristics to the dimensionally increased feature map. In one embodiment, the second dimension increaser applies an activation function that outputs positive values of pixel values of the feature map as they are and outputs negative values as values reduced by a predetermined size, thereby providing nonlinear characteristics to the feature map. can reflect

The fifth dimension reducer reduces the dimensionality of the feature map output from the second dimension increaser by applying a fifth dimension reduction filter to the feature map output from the second dimension increaser. In one embodiment, the fifth dimension reduction filter may be set as a filter capable of reducing a feature map to two dimensions.

The third probability value calculation unit calculates a third probability value for whether or not the face region is included in the corresponding second sub-input image by applying a predetermined classification function to the 2-dimensional output data output by the 5-dimensional reduction unit. In an embodiment, the third probability calculation unit determines that the second sub input image includes the face region when the calculated third probability value is greater than or equal to a third threshold value greater than the second threshold value.

The sixth dimension reduction unit applies a sixth dimension reduction filter to the feature map output from the second dimension increase unit when the third probability value calculated by the third probability value calculation unit is equal to or greater than the third threshold value, thereby outputting the output from the second dimension increase unit. Reduce the dimensionality of feature maps. In an embodiment, the 6th dimension reduction filter may be set as a filter capable of reducing a feature map to 4D, and the 6th dimension reduction unit converts four values output in 4D to a corresponding second sub input image. It is determined by the coordinates of the face area. At this time, the coordinates of the face area are defined as the coordinates of the upper left vertex and the lower right vertex when the area including the face is displayed as a rectangular bounding box, or the coordinates of the upper right vertex and the lower left vertex. It can be.

The sixth dimension reducer extracts a face image from the second sub input image determined to include the face region using the calculated coordinates of the face region.

The 7th-dimensional reducer applies a 7th-dimensional reduction filter to the feature map output from the 2nd-dimensional increaser when the third probability value calculated by the 3rd probability value calculator is equal to or greater than the third threshold, thereby outputting the output from the 2nd-dimensional increaser. Reduce the dimensionality of feature maps. In an embodiment, the 7th dimension reduction filter may be set as a filter capable of reducing a feature map to 10 dimensions, and the 7th dimension reduction unit converts 10 values output in 10 dimensions to a corresponding second sub input image. Determined by landmark coordinates. At this time, the landmark coordinates mean the coordinates of the two eyes, the coordinates of the nose, and the coordinates of the two mouths on the second sub-input image, and the coordinates of the two mouths are the coordinates of the left tail of the mouth and the right tail of the mouth. means the coordinates for

As such, according to the present invention, the face image extraction unit 237 is implemented as first to third face detection units, and the first face detection unit is composed of a shallower depth network than the second face detection unit. The second face detection unit is composed of a network with a shallower depth than the third face detection unit, so that the face image extraction unit is formed in a stepwise form in a shallow-to-deep structure as a whole. Through this, it is possible to obtain a benefit in terms of face recognition speed by improving the accuracy of face image extraction and reducing complexity at the same time.

The face image alignment unit aligns the face images using the landmark coordinates output from the third face detection unit. Specifically, the face image aligning unit aligns the face image by performing at least one of rotation, translation, enlargement, and reduction of the face image using landmark coordinates of the extracted face image.

The reason for aligning the face images using the face image aligning unit in the present invention is to improve the face recognition performance by giving consistency to the facial images to be provided as input to the feature vector extracting unit 239 .

In an embodiment, the face image arranging unit may resize the face image extracted by the third face detection unit to have the same resolution as the face image used in the feature vector extraction unit 239 . Then, the face image alignment unit rotates and parallelizes the face image by moving the landmark coordinates calculated by the third face detection unit based on the reference landmark coordinates of the face image of the resolution used in the feature vector extraction unit 239. It can be moved, enlarged or reduced.

When a face image is input from the face image extractor 237, the feature vector extractor 239 extracts a feature vector from a face included in the corresponding face image. To this end, the feature vector extraction unit 239 includes a plurality of face image processing units and a feature vector generation unit.

A plurality of face image processors image-process input data to generate output data. At this time, the face image output from the face image extraction unit 237 is input as an input image to the first face image processing unit among the plurality of face image processing units, and the g+1th face image processing unit receives the input image from the g-th face image processing unit. Output data is input.

Since the functions and detailed configurations of the plurality of face image processing units are the same, one face image processing unit among the plurality of face image processing units will be exemplarily described below.

The face image processing unit first unit generates a feature map by performing a convolution operation on input data (a face image or output data of a previous face image processing unit), and a second unit assigns weights to the feature map generated by the first unit. It consists of 2 units, and an arithmetic unit.

The first unit includes a normalization unit, a first convolution operation unit and a non-linearization unit.

The normalization unit normalizes the face images input from the face image extraction unit in batch units. A batch means a unit of the number of face images to be processed at one time. The reason why the normalizer performs normalization in batches is that when normalization is performed in batches, the average and variance for each face image may be different from the average and variance for the entire batch, and these features act as a kind of noise, resulting in overall performance because it can be improved.

In addition, through batch normalization, the internal covariance shift phenomenon in which the distribution of inputs in each layer of the network changes inconsistently increases the complexity of learning and causes gradient vanishing or exploding. because it can be prevented.

The first convolution operation unit generates a first feature map by applying a first convolution filter to the face image normalized by the normalization unit. In one embodiment, the first convolution operation unit may generate a first feature map by applying a first convolution filter having a pixel size of 3*3 and having a stride value of 1 to the face image. As described above, since the first convolution operation unit according to the present invention applies a first convolution filter having a pixel size of 3*3 and a stride value of 1 to the face image, the resolution of the first feature map can be kept high.

The non-linearization unit imparts non-linear characteristics to the first feature map by applying an activation function to the first feature map. In an embodiment, the non-linearization unit applies an activation function to the first feature map that outputs positive pixel values equally among pixel values of the first feature map and outputs negative pixel values by reducing their size. 1 Nonlinear characteristics can be assigned to feature maps.

It has been described that the nonlinearization unit imparts nonlinear characteristics to the first feature map generated by the first convolution operation unit. However, in a modified embodiment, the normalizer may further normalize the first feature maps generated by the first convolution operator in batch units. According to this embodiment, the normalization unit may provide the normalized first feature map to the non-linearization unit, and the non-linearization unit may apply an activation function to the normalized first feature map to impart nonlinear characteristics to the normalized first feature map. there is.

Meanwhile, in the above-described embodiment, the first unit has been described as including only the first convolution operation unit. However, in a modified embodiment, the first unit may further include a second convolution operator.

Specifically, the second convolution operation unit generates a second feature map by applying a second convolution filter to the first feature map to which non-linear characteristics are assigned by the non-linearization unit. In one embodiment, the second convolution filter may be a filter different from the first convolution filter. The second convolution filter may be a filter having the same size as the first convolution filter but a different stride value. For example, the second convolution filter may be a filter having a pixel size of 3*3 and a stride value of 2.

According to this embodiment, the normalizer may further normalize the second feature maps generated by the second convolution operator in batch units.

Meanwhile, the first unit may further include a pre-normalization unit. The pre-normalization unit may normalize pixel values of pixels included in the face image input from the face image extractor 237 . For example, the pre-normalization unit may normalize the face image by subtracting 127.5 from each pixel value of the face image and then dividing the resultant value by 128. The pre-normalization unit may provide the pre-normalized input face image to the normalization unit.

The second unit weights the second feature map generated by the first unit. In the present invention, the reason why the second feature map is weighted through the second unit is that in the case of convolution operation, in the process of multiplying all channels of the input image by the convolution filter and summing them, both important and unimportant channels become entangled, so that data This is to assign a weight to the second feature map according to its importance for each channel, since the sensitivity of is lowered.

The second unit includes a sampling unit, a weight reflection unit, and an upscaling unit.

First, the sampling unit reduces the dimension by subsampling the second feature map input from the first unit. In an embodiment, the sampling unit may reduce the dimensionality of the second feature map by applying a global pooling filter to the second feature map. For example, when the dimension of the second feature map is H*W*C, the sampling unit may reduce the dimension of the second feature map to 1*1*C through subsampling of the second feature map.

The weight reflection unit assigns a weight to the second feature map whose dimension is reduced by the sampling unit. To this end, the weight reflection unit may include a dimension reduction unit, a first non-linearization unit, a dimension increase unit, and a second non-linearization unit.

The dimension reducer reduces the dimension of the subsampled second feature map by connecting the subsampled second feature map into one layer. For example, when the dimension of the second feature map output from the sampling unit is 1*1*C, the dimension reduction unit reduces the dimension of the second feature map to 1*1*C/r. Here, r denotes a reduction rate and may be determined according to the number of feature vectors to be extracted.

The first nonlinearizer applies a first activation function to the second feature map whose dimensionality is reduced by the dimensionality reducer, thereby imparting nonlinear characteristics to the second feature map whose dimensionality is reduced. In an embodiment, the first non-linearization unit applies a first activation function that outputs positive pixel values as they are and outputs negative pixel values as 0 among pixel values of the second feature map, thereby reducing the second dimension. Nonlinear characteristics can be assigned to feature maps.

The dimension increasing unit increases the dimension of the second feature map to which the nonlinear characteristics are imparted by the first nonlinearization unit. For example, when the dimension of the second feature map to which the nonlinear characteristics are assigned is 1*1*c/r, the dimension increasing unit increases the dimension of the second feature map to 1*1*C.

The second nonlinearization unit applies a second activation function to the second feature map whose dimension is increased by the dimension increasing unit, thereby imparting nonlinear characteristics to the second feature map whose dimension is increased. In one embodiment, the second activation function may be a function different from the first activation function. For example, the second nonlinearization unit applies a second activation function that causes positive pixel values to converge to a predetermined value and outputs negative pixel values as 0 among the pixel values of the second feature map with increased dimensionality. Nonlinear characteristics may be assigned to the increased second feature map.

As described above, the weight reflection unit according to the present invention assigns weights to the second feature map through the dimension reduction unit, the first nonlinearization unit, the dimension increase unit, and the second nonlinearization unit, and the bottleneck section through the dimensionality reduction unit and the dimension increase unit. By defining the gating mechanism by creating , it is possible to limit the model complexity and support generalization.

The upscaling unit upscales the second feature map, weighted by the weight reflection unit, to the same dimension as the face image input to the second unit. In one embodiment, when the dimension of the face image input to the second unit is H*W*C, the upscaling unit upscales the dimension of the weighted second feature map to H*W*C.

The calculation unit sums the upscaled second feature map output through the second unit with the face image input to the first unit. In the present invention, the reason for summing the upscaled second feature map output from the second unit through the operation unit with the face image input to the first unit is to prevent the vanish problem when the depth of the convolutional neural network is increased. It is to do.

The feature vector generator generates a predetermined number of feature vectors by merging feature maps output from the last face image processor among the plurality of face image processors into one layer and reducing the dimension. In one embodiment, the feature vector generation unit may output 128 or more feature vectors from the feature map output from the face image processing unit. For example, the feature vector generation unit may output 512 feature vectors from the feature map output from the face image processing unit.

The feature vector clustering unit 240 clusters the plurality of user feature vectors extracted by the user face recognition unit 230 into K clusters. To this end, the feature vector clustering unit 240 includes an initial central value selection unit 241, a clustering unit 242, a central value updating unit 243, and a re-clustering determination unit 244, as shown in FIG. do.

The initial center value selector 241 selects an initial center value for each of the K clusters. The number of clusters may be preset by an administrator. The initial center value may be randomly selected within a predetermined range. That is, the initial center value selector 241 may randomly select K center values within a predetermined range.

The clustering unit 242 clusters the plurality of user feature vectors extracted by the user face recognition unit 230 based on the center value of each of the K clusters. Specifically, the clustering unit 242 searches for a cluster having the closest or highest similarity to each of a plurality of user feature vectors. To this end, the clustering unit 242 calculates a similarity between each of the central values of the K clusters and the user feature vector.

In an embodiment, the clustering unit 242 may calculate a similarity by using a cosine distance between each of the central values of the K clusters and the user feature vector. The cosine distance may be calculated through Equation 1 below.

The d _cos denotes the cosine distance of x and y, x denotes the user feature vector, and y denotes the center value of the cluster.

The clustering unit 242 may determine a cluster having the highest similarity among the K clusters and allow the user feature vector to belong to the determined cluster. The clustering unit 242 may determine a cluster having a central value having the highest similarity as a cluster having a high similarity. For example, the clustering unit 242 may allocate cluster identification information corresponding to the determined cluster to the user feature vector.

The center value updating unit 243 updates the center value of each of the K clusters clustered by the clustering unit 242 . Specifically, the center value updater 243 may update the center value based on user feature vectors belonging to each of the clustered K clusters. In an embodiment, the center value updating unit 243 may update the center point of user feature vectors belonging to the corresponding cluster to the center value of the corresponding cluster.

The re-clustering determination unit 244 compares existing clusters with newly clustered clusters and determines whether to re-cluster. The re-clustering determiner 244 causes clustering to be performed again by the clustering unit 242 if the existing clusters are different from the newly clustered clusters. Accordingly, the clustering unit 242 re-clusters the plurality of user feature vectors based on the updated center value of each of the K clusters.

On the other hand, the re-clustering determiner 244 stops clustering if existing clusters and newly clustered clusters are the same.

The feature vector clustering unit 240 according to the present invention may repeatedly perform clustering until the mean vector and variance of user feature vectors belonging to each cluster are minimized. If the existing clusters and the newly clustered clusters are the same, the feature vector clustering unit 240 may determine that the average vector and variance of user feature vectors belonging to each cluster are minimum, and stop clustering.

Referring back to FIG. 2 , the array file generator 245 uses the user feature vectors extracted from the face images of each of the plurality of users by the user face recognition unit 230 and the cluster identification information of each user feature vector. Then, an array is created for each user, and an array file is created by merging the created arrays into one file. Here, the cluster identification information may be information on final clusters clustered by the feature vector clustering unit 240 . Specifically, the cluster identification information may include an identification value for a cluster composed of user feature vectors similar to the corresponding user feature vector among K clusters.

The array file generator 245 may store the generated array file in an array file database (not shown).

In one embodiment, the array generated by the array file generator 245 includes a plurality of user feature vectors obtained from each user's face image, cluster identification information of each user feature vector, and each user's key. It can consist of values. At this time, the user's key value includes each user's identification information and each user's access authority information. As described above, the identification information of each user can be defined as each user's ID, name, phone number, or employee number, and the access authority information of each user includes information on each floor that each user can access. can include

The access right information management unit 250 changes access right information assigned to each user or adds new access right information. In one embodiment, the access authority information management unit 250 may grant access authority information separately for each user or may grant access authority information in organizational units to which each user belongs.

Meanwhile, the face recognition server 110 according to an embodiment of the present invention may further include a scheduler 260. The scheduler 260 performs a function of registering new users collectively whenever a predetermined period arrives or a predetermined event occurs. For example, although it has been described that the user registration unit 210 performs a registration procedure for a new user when a registration request is generated from a user, when the face recognition server 110 includes the scheduler 260, it is performed in units of predetermined time or in advance. When a predetermined event occurs, the scheduler 260 initiates the operation of the user registration unit 220, the input image generation unit 225, and the user's face recognition unit 230 so that a new user registration procedure can be automatically performed.

The interface unit 270 encrypts the face recognition model, array file, and cluster information in a predetermined manner and transmits them to each edge device 120 . Here, the cluster information may include center value information for each of the K final clusters. In one embodiment, the interface unit 270 may encrypt the face recognition model, array file, and cluster information using a public key-based encryption algorithm and transmit them to each edge device 120 .

Meanwhile, the interface unit 270 may transmit the encrypted array file to the edge device 120 according to an agreed protocol with the edge device 120 . In addition, the interface unit 270 may receive an authentication record from each edge device 120 at predetermined intervals and provide it to the edge device 120 .

The face recognition model training unit 280 generates a face recognition model 235 based on a convolutional neural network and trains the face recognition model 235 . Specifically, the face recognition model training unit 280 generates an optimal face recognition model by continuously training the convolutional neural network constituting the face recognition model 235 .

To this end, the face recognition model training unit 280 includes a face image extraction training unit 282 that trains the face image extraction unit 237, a feature vector extraction training unit 284 that trains the feature vector extraction unit 239, and an error reduction unit 286 reducing an error of the feature vector extraction unit 239 .

The face image extraction training unit 282 trains the face image extraction unit 237 using a learning image. The feature vector extraction training unit 284 trains the feature vector extraction unit 239 using the learning image.

The face image extraction training unit 282 trains the first to third face detection units constituting the face image extraction unit 237 using the learning image. Specifically, the face image extraction training unit 282 inputs a plurality of training images having a predetermined size to the first to third face detection units, calculates a probability value to include a face region in the learning image and face region coordinates, and calculates Filter coefficients and weights of the convolution filters applied to the first to third face detection units are updated by feeding back the obtained probability values and face region coordinates to the first to third face detection units according to a back propagation algorithm.

The feature vector extraction training unit 284 trains the feature vector extraction unit 239 using the learning image. Specifically, the feature vector extraction training unit 505 extracts a feature vector from each training image by inputting a plurality of training images to the feature vector extraction unit 239 in a predetermined batch unit.

The feature vector extraction training unit 284 predicts a probability value that the training image will be included in a specific class by applying the extracted feature vectors to a predetermined classification function, and between the predicted probability value (hereinafter referred to as 'predicted value') and the actual value. The filter coefficients and weights of the convolution filters applied to the feature vector extractor 239 are updated by calculating the error and feeding back the result to the feature vector extractor 284 using the backpropagation algorithm.

The error reduction unit 286 determines the error between the predicted value based on the feature vectors extracted through the feature vector extraction unit 239 and the actual value in the process of the feature vector extraction training unit 284 training the feature vector extraction unit 239. reduces Specifically, the error reduction unit 286 calculates the error between the predicted value and the actual value based on the feature vectors extracted from the training image by the feature vector extraction unit 239, and converts each training image into a two-dimensional angle so that the error can be reduced. It is arranged on a plane, and the feature vector extraction training unit 284 trains the feature vector extraction unit 239 using a probability value according to the arrangement result.

The reason why the face recognition model training unit 280 according to the present invention trains the feature vector extraction unit 239 to reduce the error through the error reduction unit 286 is that the lighting or environment in which the face is photographed changes even though the face is the same person. In this case, an error such as not being able to distinguish the same person occurs, so that the feature vector extraction unit 239 is trained to extract a feature vector that can reduce the face recognition error through the error reduction unit 286 will be.

The error reduction unit 286 according to an embodiment of the present invention includes a face image arrangement unit 287 and a probability calculation unit 289.

The face image arrangement unit 287 arranges each training image on a two-dimensional angle plane based on the plurality of feature vectors extracted by the feature vector extraction unit 239 for the training image. Specifically, the face image arranging unit 287 calculates the cosine similarity between the learning images included in different classes, and calculates the reference angle, which is the separation angle between the respective learning images, according to the cosine similarity, thereby converting the learning images to a two-dimensional angle. be placed on a flat surface.

In the present invention, when the face image arranging unit 287 arranges the training images in a vector space according to the distance between each training images calculated based on the feature vector of each training image, an overlapping area may occur between the training images. However, there is a limitation in that it is difficult to clearly distinguish between the same person and another person during learning.

Therefore, in the present invention, the face image placement unit 287 calculates angles between training images included in different classes through cosine similarity, and arranges each training image on a two-dimensional angle plane based on the calculated angles. is to do

The probability calculating unit 289 varies the margin angle to be added to the reference angle calculated by the face image arranging unit 287 on the two-dimensional angular plane, and the probability that each training image is included in the corresponding class for each variable margin angle yield

Specifically, the probability calculation unit 525 varies the margin angles P1 and P2 added to the reference angles θ1 and θ2 between each training image so that training images having overlapping characteristics are spaced apart on a two-dimensional angular plane. do. In one embodiment, the margin angles P1 and P2 may be determined according to a learning rate within a range greater than 0 and less than 90 degrees.

For example, if the learning rate increases, the margin angle may increase accordingly, and if the learning rate decreases, the margin angle may decrease accordingly. At this time, the probability calculation unit 525 may vary the margin angle by a predetermined reference unit.

When the margin angle is added to the reference angle by the probability calculation unit 289, it can be seen that training images having overlapping characteristics are spaced apart from each other in the vector space.

The probability calculation unit 289 calculates the probability that each training image will be included in the corresponding class for each margin angle added to the reference angle, and provides the calculated probability value to the feature vector extraction training unit 284, so that the feature vector extraction training unit 284 ) enables the feature vector extraction unit 239 to learn based on the probability value calculated by the probability calculation unit 289. That is, the feature vector extraction training unit 284 updates at least one of the coefficients and weights of the convolution filters applied to the feature vector extraction unit 239 based on the probability value calculated by the probability calculation unit 289 to thereby update the feature vector extraction unit. (239) will be learned.

In an embodiment, the probability calculator 289 may calculate a probability that each training image is included in a corresponding class for each margin angle using Equation 2 below.

In Equation 2, x represents the reference angle, p represents the margin angle, and n represents the number of classes.

In one embodiment, the probability calculation unit 289 performs a predetermined test on the feature vector extraction unit 239 trained by the feature vector extraction training unit 284 based on the probability value calculated by the probability calculation unit 289. When the face image is applied, the margin angle can be continuously varied until the error between the predicted value and the actual value is less than the reference value.

That is, the probability calculation unit 289 sets the margin angle at the time when the error between the predicted value and the actual value calculated when a predetermined test face image is applied to the trained feature vector extraction unit 239 is equal to or less than the reference value as the final margin angle. Decide. In this case, the error between the predicted value and the actual value can be calculated using a cross entropy function.

If the error reduction is performed through the error reduction unit 286 as described above, it is possible to accurately classify the same person even when they are photographed in different environments or under different lighting.

In the above-described embodiment, the face recognition model training unit 280 may be implemented as software in the form of an algorithm and installed in the face recognition server 110 .

Referring back to FIG. 1 , the edge device 120 uses the face recognition model 235 distributed by the face recognition server 110 at each specific place to detect the face of a target user who wishes to enter or exit the place. and performs a function of authenticating the access of the target user based on the recognition result.

In the present invention, the reason why the face recognition server 110 does not perform face recognition and authentication of the target user and the edge device 120 performs face recognition and authentication of the target user is to perform face recognition and authentication of the target user. When performed in the recognition server 110, if a failure occurs in the face recognition server 110 or the network, face recognition and authentication cannot be performed, and as the number of users increases, the expansion of the expensive face recognition server 110 is required. because it becomes

Accordingly, the present invention applies an edge computing method to perform facial recognition and authentication of a target user in the edge device 120, thereby providing a face recognition service normally even if a failure occurs in the face recognition server 110 or the network It is possible to improve service provision reliability, and even if the number of users increases, it is not necessary to increase the expensive face recognition server 110, so the cost of building the face recognition system 100 can be reduced.

Hereinafter, the configuration of the edge device 120 according to the present invention will be described in more detail with reference to FIG. 4 .

4 is a block diagram schematically showing the configuration of an edge device according to an embodiment of the present invention.

Referring to FIG. 4 , the edge device 120 according to an embodiment of the present invention includes a photographing unit 1210, an input image generating unit 1250, a target face recognition unit 1300, an authentication unit 1310, and a face recognition unit. It includes a model 1320, an update unit 1330, a memory 1340, and an interface unit 1350.

When a target user to be authenticated approaches, the photographing unit 1210 photographs the target user and creates a photographed image. The capturing unit 1210 transmits the generated captured image to the input image generating unit 1250 .

The input image generating unit 1250 generates an input image to be used for face recognition from a photographed image of a target user transmitted from the photographing unit 1210 . In detail, the input image generation unit 1250 generates a plurality of target user images having different resolutions from the captured image of one target user by downsampling or upsampling the captured image of one target user to a predetermined level.

When a plurality of target user images having different resolutions are generated, the input image generating unit 1250 inputs a plurality of images obtained by moving a window having a predetermined pixel size on the target user image for each target user image. create an image The input image generator 1250 inputs the generated input image to the target face recognition unit 1300 .

When the target user's input image is received from the input image generator 1250, the target face recognition unit 1300 inputs the received input image of the target user into the face recognition model 1320 distributed from the face recognition server 110, A target face image is extracted, and a target feature vector is extracted from the extracted target face image.

Also, the face recognition model 1320 may be updated at predetermined intervals. For example, the edge device 120 receives a new face recognition model 1320 from the face recognition server 110 whenever the face recognition model 1320 is updated by the face recognition server 110, and thus the previously distributed face The recognition model 1320 may be updated to a new face recognition model 1320 .

The authentication unit 1310 compares the target feature vector obtained by the target face recognition unit 1300 with the array file received from the face recognition server 110 to authenticate the target user. Specifically, the authentication unit 1310 converts the target feature vector into a plurality of arrays having user feature vectors extracted from face images of each of a plurality of users, cluster identification information of each user feature vector, and identification information of each user. Compared to the configured array file, the target user can be authenticated.

The authentication unit 1310 searches for a cluster having the closest or high similarity to the target feature vector among the K clusters. To this end, the authenticator 1310 calculates a distance between each of the central values of the K clusters and the target feature vector.

In an embodiment, the authenticator 1310 may calculate a cosine distance between each of the central values of the K clusters and the target feature vector. The cosine distance may be calculated through Equation 1 above.

The authentication unit 1310 determines a cluster having the highest similarity among the K clusters. The authenticator 1310 may determine a cluster having a central value having the highest similarity as a cluster having a high similarity.

The authentication unit 1310 may authenticate a target user by comparing arrays including identification information corresponding to a cluster having a high similarity among a plurality of arrays with a target feature vector. That is, the authentication unit 1310 may authenticate the target user by comparing the target feature vector with user feature vectors belonging to a cluster closest to the target feature vector.

The authentication unit 1310 compares the target feature vector with the user feature vectors included in each of the arrays to calculate a similarity, and compares the calculated similarity with a reference threshold to determine whether the target user is a registered user who can enter and exit the corresponding place. authenticate In an embodiment, the authenticator 1310 may calculate a similarity by performing a dot product operation on a target feature vector and user feature vectors belonging to a cluster closest to the target feature vector. The authentication unit 1310 may authenticate the target user as a registered user who is allowed to enter the corresponding place if the calculated similarity is greater than or equal to a reference threshold.

Meanwhile, the authentication unit 1310 may determine a user mapped to a cluster having the greatest similarity as a user most similar to the target user. The authentication unit 1310 finally authenticates the target user as a user mapped to the corresponding cluster when the highest similarity is equal to or greater than the reference threshold.

The face recognition model 1320 is generated and distributed by the face recognition server 110, and the face recognition model 1320 is updated whenever the face recognition model 235 is trained and updated by the face recognition server 110. replaced with the face recognition model 235. At this time, the face recognition model 1320 may be received from the face recognition server 110 through the interface unit 1350 .

The update unit 1330 receives, from the face recognition server 110, a new array file composed of a plurality of arrays including feature vectors extracted from the optimal reference image at each predetermined update period, and changes the existing array file into a new array file. do.

When an array file including an optimal reference image or a reference threshold value is received from the face recognition server 110 through the interface unit 1350, the update unit 1330 uploads it to the first memory 1342, and the authentication unit 1310 uses this to authenticate the target user. In particular, the update unit 1330 according to the present invention may dynamically load an array file or a reference threshold value.

Specifically, when an array file is loaded in the first memory 1342, the array file update unit 1330, when a new array file is received from the face recognition server 110, transfers the new array file to the second memory 1344. and, when the loading of the new ray file into the second memory 1344 is completed, the array file loaded into the first memory 1342 is replaced with the new array file loaded into the second memory 1344.

The reason why the update unit 1330 dynamically loads the array file as described above is that the authentication unit 1310 performs the authentication process for the target user and at the same time allows the update unit 1330 to update a new array file. This is so that the device 120 can perform face recognition in real time based on the newly updated array file.

An array file used by the authentication unit 1310 is loaded into the first memory 1342, and a newly received new array file is loaded into the second memory 1344. When loading of the new array file into the second memory 1344 is completed, the array file recorded in the first memory 1342 is replaced by the new array file by the array file update unit 1330 .

The interface unit 1350 mediates data transmission and reception between the edge device 120 and the face recognition server 110 . Specifically, the interface unit 1350 receives the face recognition model 1320 from the face recognition server 110 . The interface unit 1350 receives the array file from the face recognition server 110 and loads it into the first memory 1342 or the second memory 1344 through the array file update unit 1330 . In addition, the interface unit 1350 periodically transmits the authentication record by the authentication unit 1330 to the face recognition server 110 .

In one embodiment, the array file, face recognition model 1320, and cluster information may be updated through the interface unit 1350 at predetermined intervals.

As described above, according to the present invention, the edge device 120 stores only the face recognition model 1320 for face recognition, an array file, and a reference threshold, and no user's face image or personal information is stored in the edge device 120 ( 120) is hacked, the user's personal information is not leaked, so security is strengthened.

Referring back to FIG. 1 , the user terminal 130 transmits a user image for newly registering a user to the face recognition server 110 together with user identification information. In one embodiment, the user terminal 130 is equipped with a face registration agent (not shown) capable of interoperating with the face recognition server 110, and the user executes the face registration agent on the user terminal 130 so that the user A photographed image of the face or a previously photographed image may be transmitted to the face recognition server 110 together with user identification information.

In one embodiment, the user terminal 130 may request to register a plurality of user images for each user. In this case, the plurality of images requested to be registered for each user may be photos taken in different environments or photos taken under different lighting.

As long as the user terminal 130 can request user registration by transmitting a user image to the face recognition server 110, any type can be used without limitation. For example, the user terminal 130 may be implemented as a smart phone, a laptop computer, a desktop computer, or a tablet PC.

The face recognition system 100 according to an embodiment of the present invention compares a target feature vector with user feature vectors belonging to one of a plurality of clusters to authenticate whether a user is legitimate. The face recognition system 100 according to an embodiment of the present invention clusters user feature vectors extracted from a plurality of user face images with similar user feature vectors. In this case, user feature vectors belonging to the same cluster may be similar to each other, and user feature vectors belonging to different clusters may be regarded as heterogeneous.

The face recognition system 100 according to an embodiment of the present invention performs clustering based on the central value of each of a plurality of clusters, until the average vector and variance of user feature vectors belonging to each cluster are minimized. Clustering can be repeated while updating values. When the face recognition system 100 according to an embodiment of the present invention assigns user feature vectors to one of a plurality of clusters, a cluster having a high similarity to each of the user feature vectors can be found using a cosine distance. . In this case, the degree of similarity may be calculated through a cosine distance. Since the cosine distance has a faster calculation speed than the Euclidean distance, user feature vectors can be quickly clustered.

Meanwhile, the face recognition system 100 according to an embodiment of the present invention may not compare a target feature vector with all user feature vectors, but only user feature vectors belonging to a cluster with a high degree of similarity. The existing face recognition system 100 compares a target feature vector with all user feature vectors to authenticate a target user. In the conventional face recognition system 100, when the number of target feature vectors and the number of user feature vectors increase, the number of comparisons increases rapidly, and authentication processing speed may drop. For example, if the number of target feature vectors is M and the number of user feature vectors is N, in the existing face recognition system 100, the number of comparisons is M*N, and the time complexity according to the authentication process is O(M*N) becomes

On the other hand, the face recognition system 100 according to an embodiment of the present invention may calculate a similarity of user feature vectors belonging to a cluster having a high similarity to a target feature vector through a single dot product operation. The face recognition system 100 according to an embodiment of the present invention first compares the target feature vector with the center value of each of the K clusters to determine a cluster having the highest similarity, and determines the user feature vectors belonging to the determined cluster and the target feature vector. The degree of similarity can be calculated by performing a dot product operation on the feature vector once. Accordingly, in the face recognition system 100 according to an embodiment of the present invention, the number of comparisons becomes K+M/K, and the time complexity according to the authentication process becomes O(K+M/K). Since the time complexity of the face recognition system 100 according to an embodiment of the present invention is greatly reduced compared to existing face recognition systems, the authentication processing speed can be greatly improved.

5 is a flowchart illustrating a method of clustering user feature vectors in a face recognition system according to an embodiment of the present invention.

Referring to FIG. 5 , the face recognition system 100 according to an embodiment of the present invention extracts a user feature vector from each of a plurality of user images (S501).

The face recognition system 100 according to an embodiment of the present invention generates an input image to be used for face recognition from a user image, and inputs a plurality of generated input images to a face recognition model. The face recognition system 100 according to an embodiment of the present invention may acquire a face image including a face region through a face recognition model, and extract a user feature vector from the acquired face image.

Next, the face recognition system 100 according to an embodiment of the present invention selects an initial center value for each of the K clusters (S502). In one embodiment, the initial center value selector 241 may randomly select K center values within a predetermined range.

Next, the face recognition system 100 according to an embodiment of the present invention clusters a plurality of user feature vectors based on the initial center value of each of the K clusters (S503).

The face recognition system 100 according to an embodiment of the present invention may search for a cluster having the closest or highest similarity to each of a plurality of user feature vectors. The face recognition system 100 according to an embodiment of the present invention may calculate a similarity between each of the initial central values of the K clusters and the user feature vector.

In an embodiment, the face recognition system 100 according to an embodiment of the present invention may calculate similarity using a cosine distance between each of the initial center values of K clusters and a user feature vector. .

The face recognition system 100 according to an embodiment of the present invention may determine a cluster having the highest similarity among K clusters, and allow a user feature vector to belong to the determined cluster. The face recognition system 100 according to an embodiment of the present invention may determine a cluster having a central value having the highest similarity as a cluster having a high similarity. The face recognition system 100 according to an embodiment of the present invention may allocate cluster identification information corresponding to the determined cluster to the user feature vector.

Next, the face recognition system 100 according to an embodiment of the present invention updates the initial center value of each of the K clusters (S504).

The face recognition system 100 according to an embodiment of the present invention may update an initial center value for each of the K clusters. The face recognition system 100 according to an embodiment of the present invention may update an initial center value based on user feature vectors belonging to each of the clustered K clusters. In an embodiment, the face recognition system 100 according to an embodiment of the present invention may update the central point of user feature vectors belonging to a corresponding cluster to a central value of the corresponding cluster.

Next, the face recognition system 100 according to an embodiment of the present invention clusters the plurality of user feature vectors again based on the updated center value of each of the K clusters (S505).

The face recognition system 100 according to an embodiment of the present invention may calculate a similarity between each of the updated central values of the K clusters and the user feature vector.

In an embodiment, the face recognition system 100 according to an embodiment of the present invention may calculate similarity using a cosine distance between each of the updated center values of K clusters and a user feature vector. there is.

The face recognition system 100 according to an embodiment of the present invention may determine a cluster having a central value having the highest similarity as a cluster having a high similarity. The face recognition system 100 according to an embodiment of the present invention may reassign cluster identification information corresponding to the determined cluster to the user feature vector.

Next, the face recognition system 100 according to an embodiment of the present invention compares existing clusters with newly clustered clusters (S506), and if they are different, performs S504 and S505 again. Meanwhile, the face recognition system 100 according to an embodiment of the present invention stops clustering when existing clusters and newly clustered clusters are the same.

6 is a flowchart illustrating a method of authenticating a target user in a face recognition system according to an embodiment of the present invention.

Referring to FIG. 6 , when a target face image is input (S601), the face recognition system 100 according to an embodiment of the present invention extracts a target feature vector from the target face image (S602).

The face recognition system 100 according to an embodiment of the present invention generates an input image from a target face image and inputs the generated input image to a face recognition model. The face recognition system 100 according to an embodiment of the present invention may acquire a face image including a face region through a face recognition model, and extract a target feature vector from the acquired face image.

Next, the face recognition system 100 according to an embodiment of the present invention determines a cluster similar to the target feature vector among the K clusters (S603).

The face recognition system 100 according to an embodiment of the present invention searches for a cluster that is closest to or has a high similarity to a target feature vector among K clusters. To this end, the face recognition system 100 according to an embodiment of the present invention calculates a distance between each of the central values of the K clusters and the target feature vector.

In an embodiment, the face recognition system 100 according to an embodiment of the present invention may calculate a cosine distance between each of the central values of the K clusters and the target feature vector.

The face recognition system 100 according to an embodiment of the present invention determines a cluster having the highest similarity among K clusters. The face recognition system 100 according to an embodiment of the present invention may determine a cluster having a central value having the highest similarity as a cluster having a high similarity.

Next, the face recognition system 100 according to an embodiment of the present invention compares the target feature vector with user feature vectors belonging to a cluster having a high degree of similarity (S604).

The face recognition system 100 according to an embodiment of the present invention compares a target feature vector with arrays including identification information corresponding to a cluster having a high degree of similarity among a plurality of arrays.

The face recognition system 100 according to an embodiment of the present invention compares user feature vectors belonging to a cluster having a high degree of similarity with a target feature vector to calculate a degree of similarity. In an embodiment, the face recognition system 100 according to an embodiment of the present invention may calculate a similarity through a Euclidean distance or a cosine distance between a target feature vector and a user feature vector. there is.

Next, the face recognition system 100 according to an embodiment of the present invention authenticates the target user as a legitimate user when the similarity between the target feature vector and the user feature vectors is equal to or greater than a reference threshold (S605).

The face recognition system 100 according to an embodiment of the present invention determines the user corresponding to the user feature vector having the greatest degree of similarity as the most similar user to the target user. The face recognition system 100 according to an embodiment of the present invention finally authenticates the target user as the corresponding user when the highest degree of similarity is equal to or greater than a reference threshold.

On the other hand, the face recognition system 100 according to an embodiment of the present invention rejects the authentication of the target user when the highest similarity is less than the reference threshold (S607).

Those skilled in the art to which the present invention pertains will be able to understand that the above-described present invention may be embodied in other specific forms without changing its technical spirit or essential features.

For example, the configuration of the face recognition server shown in FIG. 2 and the configuration of the edge device shown in FIG. 4 may be implemented in the form of a program. When the configuration of the face recognition server and the configuration of the edge device according to the present invention are implemented as a program, each configuration shown in FIGS. 2 and 4 is implemented as a code, and codes for implementing specific functions are implemented as a program Alternatively, a plurality of programs may be divided and implemented.

Those skilled in the art to which the present invention pertains will be able to understand that the above-described present invention may be embodied in other specific forms without changing its technical spirit or essential features.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

100: face recognition system 110: face recognition server
120: edge device 130: user terminal
210: Edge device management unit 220: User registration unit
225: input image generation unit 230: user face recognition unit
235: face recognition model 240: feature vector clustering unit
245: array file generation unit 280: face recognition model training unit

Claims (13)

a user face recognition unit inputting a plurality of user images to a face authentication model and extracting a plurality of user feature vectors;
a feature vector clustering unit clustering the plurality of user feature vectors based on a central value of each of the K clusters;
a target face recognition unit inputting a target face image to the face authentication model and extracting a target feature vector; and
An authentication unit that compares the target feature vector with user feature vectors belonging to one of the K clusters and authenticates whether or not the user is legitimate;
Each of the K clusters includes user feature vectors of different users,
The authenticator calculates the distance between the target feature vector and the central values of each of the K clusters, determines a cluster having the smallest distance as the similar cluster, and determines each of the user feature vectors belonging to the similar cluster. Clustering characterized by calculating a similarity between the target feature vector and the calculated similarity, authenticating as a legitimate user if the greatest similarity among the calculated similarities is greater than or equal to a threshold, and rejecting authentication if the greatest similarity is less than the threshold. A face recognition system that compares face images using a technique. The method of claim 1, wherein the feature vector clustering unit,
a clustering unit which clusters the plurality of user feature vectors based on the center value of each of the K clusters; and
and a center value updating unit for updating a center value using user feature vectors belonging to each of the clustered K clusters. According to claim 2,
The clustering unit re-clusters the plurality of user feature vectors based on the updated center value of each of the K clusters when the existing clusters and newly clustered clusters are different, using a clustering technique. A face recognition system that compares images. According to claim 2,
The face authentication system of claim 1 , wherein the clustering unit compares face images by using a clustering technique that transmits the clustered clusters to the authentication unit if existing clusters and newly clustered clusters are the same. According to claim 2,
The clustering unit calculates the distance between the user feature vector and the central values of each of the K clusters, and clusters the user feature vector so that it belongs to the cluster having the smallest distance. A face recognition system that compares face images by using According to claim 5,
The face authentication system for comparing face images using a clustering technique, characterized in that the clustering unit calculates a cosine distance between each of the central values of the K clusters and the user feature vector. delete According to claim 1,
The face authentication system for comparing face images using a clustering technique, characterized in that the authentication unit calculates a cosine distance between the target feature vector and center values of each of the K clusters. According to claim 1,
The authentication unit calculates a cosine distance between each of the user feature vectors belonging to the similar cluster and the target feature vector, and if the calculated value is greater than or equal to a threshold value, the face image is authenticated as a legitimate user. A face recognition system that compares a photographing unit acquiring a photographed image of a target user to be authenticated;
a target face recognition unit extracting target feature vectors by inputting an input image generated based on the captured image to a face authentication model;
The target feature vector is compared with an array file composed of a plurality of arrays having user feature vectors extracted from face images of each of a plurality of users, cluster identification information of each user feature vector, and identification information of each user, and the target feature vector is compared. Authentication unit for authenticating the user; and
An interface unit for receiving center value information of each of the array file, face recognition model, and K clusters from a face recognition server;
The authentication unit calculates the distance between the target feature vector and the central values of each of the K clusters, determines a cluster having the smallest central value as a similar cluster, and includes identification information corresponding to the similar cluster. Calculate a similarity between each of a plurality of arrays and the target feature vector, and if the highest similarity among the calculated similarities is greater than or equal to a threshold value, the target user is authenticated as a legitimate user, and if the highest similarity value is less than the threshold value , Edge device for comparing face images using a clustering technique, characterized in that for rejecting the authentication of the target user. According to claim 10,
The cluster identification information includes an identification value for a cluster consisting of user feature vectors similar to the corresponding user feature vector among the K clusters. delete delete

Images