CN111814603B - Face recognition method, medium and electronic equipment - Google Patents
Face recognition method, medium and electronic equipment Download PDFInfo
- Publication number
- CN111814603B CN111814603B CN202010582289.8A CN202010582289A CN111814603B CN 111814603 B CN111814603 B CN 111814603B CN 202010582289 A CN202010582289 A CN 202010582289A CN 111814603 B CN111814603 B CN 111814603B
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- feature vector
- face recognition
- recognition method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention provides a face recognition method, a medium and electronic equipment; the face recognition method comprises the following steps: acquiring a first image and a second image; acquiring a public non-occlusion area of the first image and the second image; acquiring a first feature vector according to a public non-shielding area in the first image; acquiring a second feature vector according to the public non-shielding area in the second image; and obtaining the similarity of the first image and the second image according to the first feature vector and the second feature vector, and obtaining a face recognition result according to the similarity. The face recognition method can improve the accuracy of face recognition.
Description
Technical Field
The invention belongs to the field of image analysis, relates to an image recognition method, and in particular relates to a face recognition method, a medium and electronic equipment.
Background
Face recognition is a hot research and application direction in the field of artificial intelligence vision, and the technology is widely applied to commercial passenger flow analysis, security monitoring, mobile phone application and organization information verification comparison and peer-to-peer scenes. Generally, face recognition can be divided into two types, namely active fit type and passive noninductive type, according to the face acquisition and comparison mode. The method comprises the steps that an active matching type user is required to perform recognition comparison after face picture collection is completed, and common application scenes comprise face unlocking, face payment, face gate machine bayonet security check, airport railway station bank personnel verification and the like; the active fit face collection typically requires the user to uncap, remove glasses, sunglasses, etc., to minimize interference with facial masks. The passive noninductive type image recognition and comparison method has the advantages that the high-quality image of the face of the user is acquired in the picture as much as possible by means of a camera to carry out recognition and comparison, and the user is difficult to perceive without matching in the whole process. Therefore, in passive non-inductive face recognition, it is not possible nor practical to require the user to remove face masks that would reduce the accuracy of the passive non-inductive face recognition.
Disclosure of Invention
In view of the above drawbacks of the prior art, an object of the present invention is to provide a face recognition method, medium and electronic device, which are used for solving the problem that face shielding in the prior art can reduce the accuracy of passive non-inductive face recognition.
To achieve the above and other related objects, a first aspect of the present invention provides a face recognition method, medium, and electronic device. The face recognition method comprises the following steps: acquiring a first image and a second image; acquiring a public non-occlusion area of the first image and the second image; acquiring a first feature vector according to a public non-shielding area in the first image; acquiring a second feature vector according to the public non-shielding area in the second image; and obtaining the similarity of the first image and the second image according to the first feature vector and the second feature vector, and obtaining a face recognition result according to the similarity.
In an embodiment of the first aspect, the method for obtaining a common unobstructed area of the first image and the second image includes: segmenting the first image to obtain a plurality of first sub-images; segmenting the second image to obtain a plurality of second sub-images; processing the first image by using a first convolution neural network to acquire the shielding condition of the first sub-image; processing the second image by using the first convolution neural network to acquire the shielding condition of the second sub-image; and acquiring the public non-shielding area according to the shielding condition of the first sub-image and the shielding condition of the second sub-image.
In an embodiment of the first aspect, the method for obtaining the occlusion situation of the first sub-image by processing the first image using a first convolutional neural network includes: scaling and preprocessing the first image to obtain a scaled preprocessed first image; and taking the first image subjected to scaling pretreatment as the input of the first convolution neural network, wherein the output of the first convolution neural network is the shielding condition of the first sub-image.
In an embodiment of the first aspect, the implementation method for obtaining the first feature vector according to the common unobstructed area in the first image includes: processing the first image and the first sub-image by using a second convolutional neural network to obtain a first global feature vector and a first local feature vector; the first global feature vector corresponds to the first image, and the first local feature vector corresponds to the first sub-image; and acquiring a first sub-image corresponding to the public non-occlusion region, and acquiring a first feature vector according to the first local feature vector and the first global feature vector corresponding to the first sub-image.
In an embodiment of the first aspect, a height of each of the first sub-images is the same; the heights of the second sub-images are the same.
In an embodiment of the first aspect, the occlusion situation of the first sub-image includes a probability that each of the first sub-images is occluded.
In an embodiment of the first aspect, a loss function is used in training the first convolutional neural networkThe number isWherein N is the number of training sub-images, < +.>Predicted occlusion for the ith training sub-image, y i The actual occlusion situation of the ith training sub-image.
In an embodiment of the first aspect, the implementation method for obtaining the similarity between the first image and the second image according to the first feature vector and the second feature vector includes: acquiring a distance between the first feature vector and the second feature vector; and acquiring the similarity of the first image and the second image according to the distance.
A second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon; the computer program, when executed by a processor, implements the face recognition method according to the first aspect.
A third aspect of the present invention provides an electronic apparatus comprising: a memory storing a computer program; the processor is in communication connection with the memory and executes the face recognition method according to the first aspect when the computer program is called; and the display is in communication connection with the processor and the memory and is used for displaying a related GUI interactive interface of the face recognition method.
As described above, one technical scheme of the face recognition method, the medium and the electronic equipment provided by the invention has the following beneficial effects:
the face recognition method can acquire the public non-shielding area of the first image and the second image, and acquire the similarity of the first image and the second image based on the public non-shielding area, so that face recognition is realized. Because the public non-occlusion area does not contain the face occlusion object, the face identification method can reduce or even eliminate the influence of the face occlusion object on face identification, and improves the accuracy of passive non-inductive face identification.
Drawings
Fig. 1 is a flowchart of a face recognition method according to an embodiment of the invention.
Fig. 2 is a flowchart of step S12 in an embodiment of the face recognition method according to the present invention.
Fig. 3A is a diagram illustrating an exemplary image segmentation according to an embodiment of the face recognition method of the present invention.
Fig. 3B is a diagram illustrating an exemplary mask for capturing an image according to an embodiment of the face recognition method of the present invention.
Fig. 4 is a flowchart of step S123 in an embodiment of the face recognition method according to the present invention.
Fig. 5A is a schematic diagram illustrating a path diagram of step S13 in an embodiment of the face recognition method according to the present invention.
Fig. 5B is a diagram illustrating feature extraction of an image according to an embodiment of the face recognition method of the present invention.
Fig. 5C is a diagram illustrating an exemplary image feature vector acquisition method according to an embodiment of the present invention.
Fig. 6 is a flowchart illustrating training of the first convolutional neural network according to an embodiment of the face recognition method of the present invention.
Fig. 7 is a flowchart illustrating the image similarity obtaining method according to an embodiment of the present invention.
Fig. 8A is a flowchart of a face recognition method according to an embodiment of the invention.
Fig. 8B is a flowchart of step S81 in an embodiment of the face recognition method according to the present invention.
Fig. 8C is a flowchart illustrating step S83 of the face recognition method according to an embodiment of the present invention.
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Description of element reference numerals
1. First image
111-118 first sub-image
12. First global feature vector
121-128 first local feature vector
900. Electronic equipment
910. Memory device
920. Processor and method for controlling the same
930. Display device
S11 to S15 steps
S121 to S125 steps
S1231 to S1232 steps
S131 to S132 steps
S61 to S64 steps
S71 to S72 steps
S81 to S86 steps
S811-S814 steps
Steps S831 to S832
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the illustrations, not according to the number, shape and size of the components in actual implementation, and the form, number and proportion of each component in actual implementation may be arbitrarily changed, and the layout of the components may be more complex.
The essence of face recognition is to compare and analyze two or more images, so as to determine whether each image is an image of the same user. In passive non-inductive face recognition, the following two situations are often encountered: for 2 or more face images of the same user, the occlusion status in each face image is different, for example, image 1 is a near-sighted mask, and image 2 is a sunglasses mask, which can lead to a problem of low recognition rate, namely: 2 or more images of the same user are identified as different people. For 2 or more face images of different users, if the shielding conditions of the different users are the same, for example, the faces of the user 1 and the user 2 are both shielded by the same sunglasses, the problem of high recognition error rate is easily caused by the same shielding conditions, namely: and taking face images of different users as face images of the same user. It is known that both of the above cases result in poor accuracy of face recognition.
In order to solve the problem, the invention provides a face recognition method. The face recognition method obtains corresponding first feature vectors and second feature vectors based on a public non-shielding area of the first image and the second image, obtains the similarity of the first image and the second image according to the first feature vectors and the second feature vectors, and further obtains a face recognition result. Because the public non-occlusion area does not contain the occlusion object in the first image and does not contain the occlusion object in the second image, the identification result of the face identification method is irrelevant to the occlusion object in the images, and therefore the accuracy of face identification can be improved.
Referring to fig. 1, in an embodiment of the present invention, the face recognition method includes:
s11, acquiring a first image and a second image; wherein the first image and the second image both contain a facial region of a user; the first image and the second image may include a face region of the same user, or may include face regions of different users. Preferably, the face part of the person contained in the first image is identical to the face part of the person contained in the second image, for example: both include all parts between chin and forehead; alternatively, both include all the parts between the mouth and the eyebrows. Further, at least one of the first image and the second image includes a face mask.
S12, a public non-occlusion area of the first image and the second image is acquired. Wherein the common unobstructed area is within the area, the first image does not include an obstruction, and the second image does not include an obstruction; in specific application, the public non-shielding area can be obtained through pixel value comparison, image segmentation and other technologies.
Considering that common obscurations typically result in lateral obstruction of the user's face, e.g., eyeglasses will obstruct one lateral region including both eyes and mask will obstruct another lateral region including the mouth, it is preferred that the common unobstructed region consists of one or more lateral regions.
S13, acquiring a first feature vector according to the public non-occlusion area in the first image. The first feature vector is a feature vector of the first image, and includes characteristics of color, texture, shape, gray scale and the like of the first image. Preferably, the first feature vector is related only to a common unobstructed area in the first image, and is independent of images outside the common unobstructed area.
S14, acquiring a second feature vector according to the public non-occlusion area in the second image. The second feature vector is a feature vector of the second image, and includes characteristics of color, texture, shape, gray scale and the like of the second image. Preferably, the second feature vector is related only to the common unobstructed area in the second image, and not to images outside the common unobstructed area.
S15, obtaining the similarity of the first image and the second image according to the first feature vector and the second feature vector, and obtaining a face recognition result according to the similarity. The higher the similarity between the first image and the second image is, the greater the probability that the user in the first image and the user in the second image are the same user is. In a specific application, a similarity threshold may be set, and if the similarity between the first image and the second image is greater than or equal to the similarity threshold, the first image and the user in the second image are considered to be the same; and if the similarity of the first image and the second image is smaller than the similarity threshold value, the first image is considered to be different from the user in the second image.
As can be seen from the above description, the face recognition method according to the present embodiment is performed based on the common non-occlusion region, where the common non-occlusion region in the first image does not include an occlusion object, and the common non-occlusion region in the second image does not include an occlusion object, so that the face recognition method can overcome the problems of low recognition rate or high recognition error rate caused by the presence of an occlusion object, that is: the face recognition method can improve the accuracy of face recognition.
It should be noted that, although only how to perform face recognition according to the first image and the second image is described in the present embodiment, the face recognition method described in the present embodiment is equally applicable to a scenario in which face recognition is performed by a plurality of images. For example, if there are a first image, a second image and a third image, the face recognition method may obtain the similarity of the three images by obtaining a common non-occlusion region of 3 face images, and according to the first feature vector, the second feature vector and the third feature vector, thereby obtaining a face recognition result; at this time, the face recognition result includes a judgment as to whether the users in any two images are identical. In the above process, the third feature vector is obtained according to the common non-occlusion region in the third image, and the obtaining manner is the same as that in step S13 or S14.
In addition, the face recognition method of the present embodiment is also applicable to scenes in which the first image and the second image are acquired at different times, in addition to scenes in which the first image and the second image are acquired at the same time.
Referring to fig. 2, in an embodiment of the invention, a method for obtaining a common unobstructed area of the first image and the second image includes:
s121, dividing the first imageCutting to obtain a plurality of first sub-images P 1,1 ，P 1,2 ，P 1,3 ……P 1,N . Wherein N is the number of the first sub-images.
S122, dividing the second image to obtain a plurality of second sub-images P 2,1 ，P 2,2 ，P 2,3 ……P 2,N . Preferably, the second image is segmented in the same way as the first image, namely: for any i < N, the first sub-image P 1,i And a second sub-image P 2,i Is the same size.
S123, processing the first image by using a first convolution neural network to acquire the shielding condition of the first sub-image. The occlusion condition of the first sub-image may be occlusion or non-occlusion, or may be a probability that the first sub-image is occluded.
Specifically, the first image is used as an input of the first convolutional neural network, and an output of the first convolutional neural network is an occlusion condition of the first sub-image. The first convolutional neural network is a pre-trained convolutional neural network model. Preferably, the occlusion condition of each first sub-image can be acquired through step S123.
S124, processing the second image by using the first convolution neural network to acquire the shielding condition of the second sub-image. The occlusion condition of the second sub-image may be occlusion or non-occlusion, or may be a probability that the second sub-image is occluded. Specifically, the second image is used as the input of the first convolutional neural network, and the output of the first convolutional neural network is the shielding condition of the second sub-image. Preferably, the occlusion condition of each second sub-image can be acquired through step S124.
S125, according to the shielding condition of the first sub-image and the shielding condition of the second sub-image, the public non-shielding area is obtained. Specifically, the first sub-image is traversed, wherein, for any i < N, if the first sub-image P 1,i And a second sub-image P 2,i Neither is nor is the same asIf there is an occlusion, the first sub-image P 1,i The corresponding region belongs to the common non-occlusion region of the first image, and the second sub-image P 2,i The corresponding region belongs to a common unobstructed region of the second image.
In an embodiment of the present invention, the occlusion situation of the first sub-image may be represented by using a first Mask1. The first Mask1 is an N-dimensional vector, each element corresponds to one sub-image, an element of 1 indicates that the corresponding sub-image is not blocked, and an element of 0 indicates that the corresponding sub-image is blocked. For example, referring to fig. 3A, the first image 1 is divided into 8 first sub-images 111-118, wherein the first Mask1 is [1,1,1,0,0,1,1,1] when the first sub-image 114 and the first sub-image 115 are blocked.
Referring to fig. 3B, in this embodiment, the first Mask1 may be obtained by inputting the first image into the first convolutional neural network. Specifically, when the output result of the first convolutional neural network is blocked or not, using 1 to indicate that the first convolutional neural network is not blocked and 0 to indicate that the first convolutional neural network is blocked, and combining the output of the first convolutional neural network to obtain the first Mask1; when the output result of the first convolutional neural network is the blocked probability, the first Mask1 may be obtained according to a probability threshold and the blocked probability: when a certain output result is greater than or equal to the probability threshold, the corresponding element in the first Mask1 is 0; when a certain output result is smaller than the probability threshold, the corresponding element in the first Mask1 is 1.
Similarly, the occlusion situation of the second sub-image may be represented by a second Mask 2; and inputting the second image into the first convolutional neural network can obtain the second Mask2.
In this embodiment, the common unobstructed area may be obtained through the first Mask1 and the second Mask2. Specifically, a public mask=mask 1 n Mask2 is defined, and the area corresponding to the public Mask is the public non-shielding area.
Referring to fig. 4, in an embodiment of the present invention, a method for processing the first image by using a first convolutional neural network to obtain an occlusion situation of the first sub-image includes:
and S1231, scaling and preprocessing the first image to obtain a scaled preprocessed first image. Wherein the order of the scaling and the preprocessing may be interchanged. Specifically, the scaling refers to scaling the first image to an input size of the first convolutional neural network, for example, the input image may be scaled to 112×112 pixels, so that the first convolutional neural network processes the first image. Furthermore, the first image involved in face recognition often contains irrelevant information, and to eliminate the influence of the irrelevant information, step S1231 further includes preprocessing the first image, where the preprocessing, for example, subtracts the pixel mean value of the first image from each pixel point of the first image and normalizes to the range of [ -1,1]. The preprocessing can restore useful real information in the first image, enhance the detectability of the information and simplify the data to the maximum extent, thereby improving the reliability of the face recognition.
S1232, taking the first image subjected to scaling pretreatment as the input of the first convolutional neural network, wherein the output of the first convolutional neural network is the shielding condition of the first sub-image. The occlusion condition of the first sub-image may be qualitative data such as occlusion or non-occlusion, or quantitative data such as occlusion probability. In addition, the output of the first convolutional neural network may be a first Mask1 corresponding to the first image, where the first Mask1 is used to represent the occlusion condition of the first sub-image.
Similarly, in a specific application, the second image may be scaled and preprocessed to obtain a scaled preprocessed second image, and the first convolutional neural network may be used to process the second image to obtain an occlusion condition of the second sub-image. The procedure is similar to the above steps S1231 to S1232, and will not be repeated here.
Referring to fig. 5A and 5B, in an embodiment of the invention, a method for obtaining a first feature vector according to a common unobstructed area in the first image includes:
s131, the first image 1 and the first sub-images 111-118 are processed by using a second convolution neural network to obtain a first global feature vector 12 and a first local feature vector 122. Wherein the first global feature vector 12 corresponds to the first image 1 and has a number of 1; the first local feature vectors 121 to 128 correspond to the first sub-images 111 to 118, respectively, and each first sub-image corresponds to one first local feature vector, for example, the first sub-image 111 corresponds to the first local feature vector 121, and the first sub-image 112 corresponds to the first local feature vector 122. Preferably, the dimensions of the first global feature vector 12 and each of the first local feature vectors 122 are the same, and the first global feature vector 12 is the same as the type of elements in each of the first local feature vectors 122, except that the elements in the first global feature vector 12 correspond to the first image 1 and the elements in each of the first local feature vectors 122 correspond to each of the first sub-images.
The second convolutional neural network is a pre-trained convolutional neural network model; the input of the first image is mainly used for acquiring the first global feature vector, and the input of the first sub-image is mainly used for acquiring first local feature information. Specifically, by adopting different training modes, the first image and each first sub-image can be respectively processed by using the second convolutional neural network in the step, so as to respectively obtain a first global feature vector and a first local feature vector which are respectively corresponding to each first image and each first sub-image; the second convolutional neural network may also be used to process the first image and all the first sub-images simultaneously to obtain the first global feature vector and all the first local feature vectors simultaneously. The specific training manner of the second convolutional neural network may be implemented by using the prior art, which is not described herein.
Preferably, the first globalThe feature vector and the first local feature vector may be stitched by a contact function into a first vector, in particular the first vector f 1 ＝contact(f G ,f L_1 ,f L_2 ,......,f L_N ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein f G For the global feature vector, f L_i Is the i-th local feature vector. The contact function is used to compare f G And f L_i Splicing without any other operation, when f G And f L_i The first vector f is the vector of M rows and 1 columns 1 Is a matrix of M rows and N columns.
S132, acquiring a first sub-image corresponding to the public non-occlusion region, and acquiring a first feature vector according to the first local feature vector and the first global feature vector corresponding to the first sub-image. The first sub-image corresponding to the public non-occlusion region refers to that the region corresponding to the first sub-image belongs to the public non-occlusion region. For example, the first sub-image corresponding to the common unobstructed area in fig. 5B includes a first sub-image 111, a first sub-image 112, a first sub-image 113, a first sub-image 116, a first sub-image 117, and a first sub-image 118, and the first local feature vectors corresponding to the first sub-image are a first local feature vector 121, a first local feature vector 122, a first local feature vector 123, a first local feature vector 126, a first local feature vector 127, and a first local feature vector 128.
One implementation method for obtaining the first feature vector according to the first local feature vector corresponding to the first sub-image and the first global feature vector includes: the first local feature vector and the first global feature vector in the step S132 are spliced through a contact function, so that the first feature vector f can be obtained 1 'A'; for example, for fig. 5B, a first feature vector f obtained according to the method described above 1 ' is: f (f) 1 '＝contact(f G ,f L_1 ,f L_2 ,f L_3 ,f L_6 ,f L_7 ,f L_8 )。
According to the first local characteristic direction corresponding to the first sub-imageAnother implementation method for obtaining the first feature vector by the quantity and the first global feature vector includes: for the first vector f 1 ＝contact(f G ,f L_1 ,f L_2 ,......,f L_N ) If a certain first local feature vector f L_i Does not correspond to the common unobstructed area, then the first local feature vector f L_i From the first vector f 1 Delete in the middle; traversing the first vector f 1 All the first local feature vectors in the list are deleted, and the first feature vector f is obtained by deleting all the first local feature vectors which do not correspond to the common non-shielding area 1 '. For example, for the first sub-image and the first local feature vector in fig. 5B, the first local feature vectors that do not correspond to the common non-occlusion region are first local feature vectors 124 and 125, and the first feature vector f is obtained by deleting the two first local feature vectors from the first vector 1 '＝contact(f G ,f L_1 ,f L_2 ,f L_3 ,f L_6 ,f L_7 ,f L_8 )。
A further implementation method for obtaining the first feature vector according to the first local feature vector and the first global feature vector corresponding to the first sub-image includes: acquiring the public Mask; for an element of 0 in the common Mask, the first vector f 1 Corresponding first local feature vector deletion. For example, referring to fig. 5C, the common Mask mask= [1,1,0,0,0,0,1,1 ]]Wherein the element of 0 is 3 rd, 4 th, 5 th and 6 th; thus, from the first vector f 1 Corresponding first local feature vectors 123, 124, 125 and 126 are deleted to obtain the first feature vector f 1 '。
Preferably, before step S131, the face recognition method of the present embodiment further includes a step of scaling and/or preprocessing the first image and the first sub-image. Wherein the scaling step is for scaling the first image and each first sub-image to an input size of the second convolutional neural network, e.g. 112 x 112 pixels. The preprocessing step is used for eliminating irrelevant information in the first image and the first sub-image, and the implementation method includes: for any one of the first image and the first sub-image, subtracting the pixel average value of the image from each pixel point in the image and normalizing to [ -1,1].
In this embodiment, the second feature vector f may be obtained by a similar method 2 ' is limited to the description and will not be described in detail herein.
In an embodiment of the invention, the heights of the first sub-images are the same; the heights of the second sub-images are the same. That is, in this embodiment, step S121 performs N equal division on the first image to obtain N first sub-images with the same height, and step S122 performs N equal division on the second image to obtain N second sub-images with the same height. In this embodiment, by equally dividing the first image and the second image, the sizes of the first sub-image and the second sub-image are the same, which is beneficial to improving the processing speed and the processing accuracy of the first convolutional neural network.
Referring to fig. 6, in an embodiment of the invention, a training method of the first convolutional neural network is as follows:
s61, acquiring a plurality of training images; each training image includes at least a partial facial region.
S62, dividing each training image to obtain N Zhang Xunlian sub-images; the segmentation method of the training image is the same as that in step S121 or step S122.
S63, obtaining shielding conditions of the training sub-images; the shielding condition can be marked manually or obtained by an AI mode, and is not limited herein.
S64, training a convolutional neural network by using the shielding conditions of the training image and the training sub-image to obtain the first convolutional neural network. Training the convolutional neural network may be achieved by an existing training method, and will not be described in detail herein.
In one embodiment of the present invention, the loss function employed in training the first convolutional neural network isWherein N is the number of training sub-images, < +.>Predicted occlusion for the ith training sub-image, y i The actual occlusion situation of the ith training sub-image.
Wherein, the liquid crystal display device comprises a liquid crystal display device,there may be two values, i.e. the ith sub-image is blocked or unblocked, and accordingly y i The ith sub-image is blocked or not blocked to take two values; for example, it is possible to indicate blocked by 0 and unblocked by 1, at which time, the +.>Has a value of 0 or 1, y i The value of (2) is also 0 or 1.
A predicted probability value, which may be that the ith sub-picture is occluded or not occluded, at which time +.>The value range of (2) is [0, 1]]The method comprises the steps of carrying out a first treatment on the surface of the Accordingly, y i For the actual probability value, y, that the ith sub-image is occluded or not occluded i The value range of (2) is also [0, 1]]。
Because convolutional neural networks have lower accuracy when outputting qualitative results, and relatively higher accuracy when outputting quantitative results. In view of this, in an embodiment of the present invention, the occlusion situation of the first sub-images includes a probability that each of the first sub-images is occluded, which is a certain amount of results, and thus the accuracy is relatively high. To achieve this, when training the first convolutional neural network, the occlusion situation of the training sub-image acquired in step S63 is also represented by a corresponding probability.
After the first convolutional neural network outputs the probability that each first sub-image is blocked, the first sub-image can be determined to belong to blocked or non-blocked according to a probability threshold. Specifically, when the probability of a certain first sub-image being blocked is greater than or equal to a probability threshold value, the first sub-image is considered to be blocked; otherwise, the first sub-image is considered to be unobstructed. The probability threshold may be set according to actual requirements, for example, 0.5, 0.75, etc.
Referring to fig. 7, in an embodiment of the invention, a method for obtaining a similarity between the first image and the second image according to the first feature vector and the second feature vector includes:
s71, obtaining the distance between the first characteristic vector and the second characteristic vector. The distance may be Euclidean distance, chebyshev distance, minkowski distance, manhattan distance, etc., without limitation.
S72, obtaining the similarity of the first image and the second image according to the distance. Specifically, the larger the distance between the first feature vector and the second feature vector, the lower the similarity between the first image and the second image, and further the probability that the user in the first image and the user in the second image are different users is known to be larger. Likewise, the smaller the distance between the first feature vector and the second feature vector, the greater the probability that the user in the first image is the same user as the user in the second image.
The method for calculating the similarity comprises the following steps: similary=1-dist (f 1 ',f 2 ' s); wherein Similarity is a Similarity between the first image and the second image, dist (f 1 ',f 2 ') is the first feature vector f 1 ' and the second eigenvector f 2 ' distance between.
Referring to fig. 8A, in an embodiment of the invention, the face recognition method includes:
s81, carrying out shielding condition recognition on two or more images to be recognized so as to obtain shielding conditions of the images; wherein, the shielding condition is represented by a mask corresponding to each image.
S82, intersection sets are taken for masks corresponding to the images, so that a public Mask is obtained.
S83, extracting features of each image to be identified to obtain a feature extraction result vector; the feature extraction result vector comprises a global feature vector and a local feature vector corresponding to each image.
S84, processing the feature extraction result vector according to the public Mask to obtain a feature vector f' corresponding to each image. Specifically, according to the common Mask, the feature vector f' may be generated by removing a local feature vector of the blocked portion from the result vector of the feature extraction, and a section of the program for implementing the process includes:
and S85, performing similarity calculation by using the feature vectors corresponding to the images to obtain the similarity between any two images. Specifically, for the ith image and the jth image, the Similarity between the two images i,j ＝1-dist(f i ',f j ' s); wherein f i ' is the feature vector corresponding to the ith image, f j ' is the feature vector corresponding to the jth image, dist (f i ',f j ') is f i ' and f j ' normalized distance between.
S86, obtaining a face recognition result according to the similarity between any two images. When the similarity between any two images is greater than or equal to a similarity threshold, the faces in the two images are considered to belong to the same user; otherwise, the faces in the two images are considered to belong to different users.
Specifically, referring to fig. 8B, for any image to be identified, the implementation method for identifying the shielding condition of the image to obtain the shielding condition of the image includes:
s811, dividing the image by N equally to obtain N sub-images.
And S812, scaling the size of the image to the input size of the first convolutional neural network, and preprocessing the image.
S813, processing the image by using the first convolution neural network to obtain shielding probability of each sub-image.
S814, obtaining a mask of each sub-image according to the shielding probability corresponding to the image. Wherein the mask of the image is an N-dimensional vector, and each element corresponds to a sub-image: when the shielding probability of the sub-image is more than or equal to a probability threshold value, the element in the corresponding mask is 1; when the shielding probability of the sub-image is smaller than the probability threshold, the element in the corresponding mask is 0.
Referring to fig. 8C, for any image to be identified, the implementation method for extracting features of the image includes:
and S831, inputting the image and the sub-image obtained after N equal division of the image into a second convolution neural network, wherein the output of the second convolution neural network is the global feature vector and N local feature vectors. The input of the image full graph mainly extracts global features, and the input of N sub-images is mainly used for extracting local feature information; the dimensions of the global feature vector and each local feature vector are fixed and do not change due to changes in the input image, and the dimensions of the global feature vector and each local feature vector are equal.
In the training process of the second convolutional neural network, when the loss function is calculated, a common Mask is obtained by taking an intersection of masks of all training images in the whole batch, and image output characteristics of all images in the batch are readjusted according to the common Mask, namely, local characteristics of a shielding part are removed. Therefore, the present embodiment uses the feature vector f of the removed occlusion part j ' calculating the loss function, the implementation process can be realized by using face recognition loss functions of some main streams.
S832, splicing the global feature vector and the N local feature vectors by using a contact function to obtain a result vector f of the feature extraction i ＝contact(f G ,f L_1 ,f L_2 ,......,f L_N ) Wherein f i For the result vector of feature extraction, f G For the global feature vector, f L_i Is the i-th local feature vector.
Based on the above description of the face recognition method, the present invention also provides a computer-readable storage medium having a computer program stored thereon; the computer program when executed by a processor implements the face recognition method of the present invention.
Based on the above description of the face recognition method, the invention also provides electronic equipment. Referring to fig. 9, in an embodiment of the invention, the electronic device 900 includes: a memory 910 storing a computer program; a processor 920, communicatively connected to the memory 910, for executing the face recognition method according to the present invention when the computer program is called; and a display 930 communicatively coupled to the processor 920 and the memory 910 for displaying a GUI interactive interface associated with the face recognition method.
The protection scope of the face recognition method of the present invention is not limited to the execution sequence of the steps listed in the present embodiment, and all the schemes implemented by the steps of increasing or decreasing and step replacing in the prior art according to the principles of the present invention are included in the protection scope of the present invention.
The invention provides a multi-granularity face recognition method based on a convolutional neural network, which can simultaneously acquire global features and local features by dividing N equal parts of an input image on the premise of not obviously increasing the calculated amount, and reduce interference caused by a shielding object on recognition by removing the local features of a shielding part, so that the matching recognition of the shielded and non-shielded faces can be effectively improved.
The face recognition method can acquire the public non-shielding area of the first image and the second image, and acquire the similarity of the first image and the second image based on the public non-shielding area, so that face recognition is realized. Because the public non-occlusion area does not contain the face occlusion object, the face identification method can reduce or even eliminate the influence of the face occlusion object on face identification, and improves the accuracy of passive non-inductive face identification.
In summary, the present invention effectively overcomes the disadvantages of the prior art and has high industrial utility value.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (9)
1. A face recognition method, characterized in that the face recognition method comprises:
acquiring a first image and a second image;
acquiring a public non-occlusion area of the first image and the second image;
acquiring a first feature vector according to a public non-shielding area in the first image;
acquiring a second feature vector according to the public non-shielding area in the second image;
obtaining the similarity of the first image and the second image according to the first feature vector and the second feature vector, and obtaining a face recognition result according to the similarity;
the implementation method for acquiring the public non-occlusion area of the first image and the second image comprises the following steps:
segmenting the first image to obtain a plurality of first sub-images;
segmenting the second image to obtain a plurality of second sub-images;
processing the first image by using a first convolution neural network to acquire the shielding condition of the first sub-image;
processing the second image by using the first convolution neural network to acquire the shielding condition of the second sub-image;
and acquiring the public non-shielding area according to the shielding condition of the first sub-image and the shielding condition of the second sub-image.
2. The face recognition method according to claim 1, wherein the implementation method for processing the first image by using a first convolutional neural network to obtain the occlusion situation of the first sub-image includes:
scaling and preprocessing the first image to obtain a scaled preprocessed first image;
and taking the first image subjected to scaling pretreatment as the input of the first convolution neural network, wherein the output of the first convolution neural network is the shielding condition of the first sub-image.
3. The face recognition method according to claim 1, wherein the implementation method for acquiring the first feature vector according to the common unobstructed area in the first image includes:
processing the first image and the first sub-image by using a second convolutional neural network to obtain a first global feature vector and a first local feature vector; the first global feature vector corresponds to the first image, and the first local feature vector corresponds to the first sub-image;
and acquiring a first sub-image corresponding to the public non-occlusion region, and acquiring a first feature vector according to the first local feature vector and the first global feature vector corresponding to the first sub-image.
4. The face recognition method according to claim 1, wherein: the heights of the first sub-images are the same; the heights of the second sub-images are the same.
5. The face recognition method according to claim 1, wherein: the occlusion situation of the first sub-images includes a probability that each of the first sub-images is occluded.
6. The face recognition method according to claim 1, wherein: the loss function adopted in training the first convolutional neural network is thatWherein N is the number of training sub-images, < +.>Predicted occlusion for the ith training sub-image, y i The actual occlusion situation of the ith training sub-image.
7. The face recognition method according to claim 1, wherein the implementation method for obtaining the similarity of the first image and the second image according to the first feature vector and the second feature vector includes:
acquiring a distance between the first feature vector and the second feature vector;
and acquiring the similarity of the first image and the second image according to the distance.
8. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, implements the face recognition method of any one of claims 1-7.
9. An electronic device, the electronic device comprising:
a memory storing a computer program;
a processor communicatively coupled to the memory, the processor executing the face recognition method of any one of claims 1-7 when the computer program is invoked;
and the display is in communication connection with the processor and the memory and is used for displaying a related GUI interactive interface of the face recognition method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010582289.8A CN111814603B (en) | 2020-06-23 | 2020-06-23 | Face recognition method, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010582289.8A CN111814603B (en) | 2020-06-23 | 2020-06-23 | Face recognition method, medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111814603A CN111814603A (en) | 2020-10-23 |
| CN111814603B true CN111814603B (en) | 2023-09-05 |
Family
ID=72846027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010582289.8A Active CN111814603B (en) | 2020-06-23 | 2020-06-23 | Face recognition method, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111814603B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114511893A (en) * | 2020-10-26 | 2022-05-17 | 京东方科技集团股份有限公司 | Convolutional neural network training method, face recognition method and face recognition device |
| CN112613474B (en) * | 2020-12-30 | 2022-01-18 | 珠海大横琴科技发展有限公司 | Pedestrian re-identification method and device |
| CN113011386B (en) * | 2021-04-13 | 2023-04-07 | 重庆大学 | Expression recognition method and system based on equally divided characteristic graphs |
| CN116563926B (en) * | 2023-05-17 | 2024-03-01 | 智慧眼科技股份有限公司 | Face recognition method, system, equipment and computer readable storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855496A (en) * | 2012-08-24 | 2013-01-02 | 苏州大学 | Method and system for authenticating shielded face |
| KR20150046718A (en) * | 2013-10-22 | 2015-04-30 | 삼성전자주식회사 | Apparatus and method of face tracking |
| CN104751108A (en) * | 2013-12-31 | 2015-07-01 | 汉王科技股份有限公司 | Face image recognition device and face image recognition method |
| CN106372603A (en) * | 2016-08-31 | 2017-02-01 | 重庆大学 | Shielding face identification method and shielding face identification device |
| CN108090433A (en) * | 2017-12-12 | 2018-05-29 | 厦门集微科技有限公司 | Face identification method and device, storage medium, processor |
| CN109002799A (en) * | 2018-07-19 | 2018-12-14 | 苏州市职业大学 | Face identification method |
| CN109145745A (en) * | 2018-07-20 | 2019-01-04 | 上海工程技术大学 | A kind of face identification method under circumstance of occlusion |
| CN109325412A (en) * | 2018-08-17 | 2019-02-12 | 平安科技（深圳）有限公司 | Pedestrian recognition method, device, computer equipment and storage medium |
| CN110532979A (en) * | 2019-09-03 | 2019-12-03 | 深圳市华芯技研科技有限公司 | A kind of 3-D image face identification method and system |
| CN110569731A (en) * | 2019-08-07 | 2019-12-13 | 北京旷视科技有限公司 | face recognition method and device and electronic equipment |
| CN110569756A (en) * | 2019-08-26 | 2019-12-13 | 长沙理工大学 | face recognition model construction method, recognition method, device and storage medium |
| CN110598587A (en) * | 2019-08-27 | 2019-12-20 | 汇纳科技股份有限公司 | Expression recognition network training method, system, medium and terminal combined with weak supervision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407912B (en) * | 2016-08-31 | 2019-04-02 | 腾讯科技（深圳）有限公司 | A kind of method and device of face verification |
-
2020
- 2020-06-23 CN CN202010582289.8A patent/CN111814603B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855496A (en) * | 2012-08-24 | 2013-01-02 | 苏州大学 | Method and system for authenticating shielded face |
| KR20150046718A (en) * | 2013-10-22 | 2015-04-30 | 삼성전자주식회사 | Apparatus and method of face tracking |
| CN104751108A (en) * | 2013-12-31 | 2015-07-01 | 汉王科技股份有限公司 | Face image recognition device and face image recognition method |
| CN106372603A (en) * | 2016-08-31 | 2017-02-01 | 重庆大学 | Shielding face identification method and shielding face identification device |
| CN108090433A (en) * | 2017-12-12 | 2018-05-29 | 厦门集微科技有限公司 | Face identification method and device, storage medium, processor |
| CN109002799A (en) * | 2018-07-19 | 2018-12-14 | 苏州市职业大学 | Face identification method |
| CN109145745A (en) * | 2018-07-20 | 2019-01-04 | 上海工程技术大学 | A kind of face identification method under circumstance of occlusion |
| CN109325412A (en) * | 2018-08-17 | 2019-02-12 | 平安科技（深圳）有限公司 | Pedestrian recognition method, device, computer equipment and storage medium |
| CN110569731A (en) * | 2019-08-07 | 2019-12-13 | 北京旷视科技有限公司 | face recognition method and device and electronic equipment |
| CN110569756A (en) * | 2019-08-26 | 2019-12-13 | 长沙理工大学 | face recognition model construction method, recognition method, device and storage medium |
| CN110598587A (en) * | 2019-08-27 | 2019-12-20 | 汇纳科技股份有限公司 | Expression recognition network training method, system, medium and terminal combined with weak supervision |
| CN110532979A (en) * | 2019-09-03 | 2019-12-03 | 深圳市华芯技研科技有限公司 | A kind of 3-D image face identification method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111814603A (en) | 2020-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111814603B (en) | Face recognition method, medium and electronic equipment | |
| CN110826519B (en) | Face shielding detection method and device, computer equipment and storage medium | |
| CN111191616A (en) | Face shielding detection method, device, equipment and storage medium | |
| CN106485215B (en) | Face shielding detection method based on deep convolutional neural network | |
| CN110287846A (en) | A kind of face critical point detection method based on attention mechanism | |
| CN106056079B (en) | A kind of occlusion detection method of image capture device and human face five-sense-organ | |
| KR20180109665A (en) | A method and apparatus of image processing for object detection | |
| CN110348331B (en) | Face recognition method and electronic equipment | |
| CN113420731B (en) | Model training method, electronic device and computer-readable storage medium | |
| CN111914748B (en) | Face recognition method, device, electronic equipment and computer readable storage medium | |
| CN111898413A (en) | Face recognition method, face recognition device, electronic equipment and medium | |
| CN107844742A (en) | Facial image glasses minimizing technology, device and storage medium | |
| CN111611849A (en) | Face recognition system for access control equipment | |
| CN110059634B (en) | Large-scene face snapshot method | |
| CN110287791A (en) | A kind of screening technique and system for face picture | |
| CN111814571A (en) | Mask face recognition method and system based on background filtering | |
| CN112560584A (en) | Face detection method and device, storage medium and terminal | |
| CN112766028A (en) | Face fuzzy processing method and device, electronic equipment and storage medium | |
| CN113963426A (en) | Model training method, mask wearing face recognition method, electronic device and storage medium | |
| Chandak et al. | Semantic image completion and enhancement using deep learning | |
| CN116309590B (en) | Visual computing method, system, electronic equipment and medium based on artificial intelligence | |
| Liu et al. | Implementation System of Human Eye Tracking Algorithm Based on FPGA. | |
| CN113239739A (en) | Method and device for identifying wearing article | |
| CN110866500A (en) | Face detection alignment system, method, device, platform, mobile terminal and storage medium | |
| CN115205943A (en) | Image processing method, image processing device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 201203 No. 6, Lane 55, Chuanhe Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai Applicant after: Winner Technology Co.,Ltd. Address before: 201505 Room 216, 333 Tingfeng Highway, Tinglin Town, Jinshan District, Shanghai Applicant before: Winner Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |