CN112329666A - Face recognition method and device, electronic equipment and storage medium - Google Patents

Face recognition method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112329666A
CN112329666A CN202011251504.2A CN202011251504A CN112329666A CN 112329666 A CN112329666 A CN 112329666A CN 202011251504 A CN202011251504 A CN 202011251504A CN 112329666 A CN112329666 A CN 112329666A
Authority
CN
China
Prior art keywords
face
shallow
similarity
image
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011251504.2A
Other languages
Chinese (zh)
Inventor
罗胜寅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Life Insurance Company of China Ltd
Original Assignee
Ping An Life Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Life Insurance Company of China Ltd filed Critical Ping An Life Insurance Company of China Ltd
Priority to CN202011251504.2A priority Critical patent/CN112329666A/en
Publication of CN112329666A publication Critical patent/CN112329666A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

The invention relates to an artificial intelligence technology, and discloses a face recognition method, which comprises the following steps: the client identifies and obtains a face image from an original video stream, shallow face features of the face image are extracted and transmitted to the server, the server searches and obtains a first identified face according to the shallow face features and calculates corresponding first face similarity, when the client judges that the first face similarity is not within a preset threshold value, the face image is transmitted to the server, the server extracts deep face features of the face image, and searches and obtains a second identified face according to the deep face features, and a final identification result is obtained. In addition, the invention also relates to a block chain technology, and the final identification result can be stored in a node of the block chain. The invention also provides a face recognition device, electronic equipment and a storage medium. The invention can solve the problems of low speed and low precision of face recognition. The invention also relates to the field of digital medical treatment, and the final recognition result can be applied to an intelligent medical treatment system.

Description

Face recognition method and device, electronic equipment and storage medium
Technical Field
The present invention relates to artificial intelligence technology, and in particular, to a method and an apparatus for face recognition, an electronic device, and a computer-readable storage medium.
Background
The face recognition system is widely used in practical application, and in practical application, the design scheme is divided into a client side deployed face recognition system or a service end deployed face recognition system. When the client deploys the face recognition system, the face image is compared with a face image feature library stored in a local database of the client through the client, so that face recognition is realized. When the face recognition system is deployed at the server side, the user side is required to transmit the face image to the server side, and the server side compares the face image with a face image feature library stored in a server side database, so that face recognition is realized. For both methods, the following drawbacks exist: 1. when the client is deployed, the computing capacity of the client is limited, and meanwhile, the risk that the local feature library of the client is tampered exists; 2. when the server is deployed, the bandwidth pressure of the server is large, the concurrency capability of the server faces higher pressure, and the computing capability of the terminal cannot be effectively utilized. 3. When the client side deployment or the server side deployment is only carried out, the overall recognition speed is low, and the recognition accuracy is low.
Disclosure of Invention
The invention provides a face recognition method, a face recognition device and a computer readable storage medium, and mainly aims to solve the problems of low face recognition speed and low accuracy.
In order to achieve the above object, the present invention provides a face recognition method applied to a client, including:
acquiring an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image;
extracting shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server;
and when the first face similarity is not within a preset first threshold range, sending the preprocessed face image to a server, receiving a second recognized face returned by the server according to the face image and a corresponding second face similarity, and obtaining a final recognition result according to the second face similarity.
Optionally, the performing face detection on each frame of image in the original video stream, and performing alignment and clipping operations on the detected face image to obtain a preprocessed face image includes:
carrying out face detection on each frame of image of the original video stream, and carrying out screenshot on the detected face image to obtain an initial face image;
detecting the positions of the face feature points in the initial face image, and carrying out position-driven deformation on the positions of the face feature points to obtain an aligned face image;
and cutting the aligned face image into a picture with a preset size to obtain the preprocessed face image.
Optionally, the extracting shallow face features in the face image includes:
carrying out graying and noise reduction operation on the face image to obtain a processed picture;
and extracting the shallow face features in the processed picture by using a preset shallow extraction model.
Optionally, after extracting the shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face and a corresponding first face similarity returned by the server, the method further includes:
and when the first face similarity falls into any sub-threshold range of the first threshold range, outputting a corresponding recognition result.
In order to achieve the above object, the present invention provides a face recognition method applied to a server, including:
receiving shallow face features sent by a client, performing face recognition by using the shallow face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client;
when the first face similarity is not within a preset first threshold range, receiving a preprocessed face image sent by a client, and performing deep feature extraction on the face image to obtain deep face features;
and performing face recognition by using the deep face features to obtain a second recognized face, and returning the second recognized face and the corresponding second face similarity to the client.
Optionally, the receiving a shallow face feature sent by a client, performing face recognition by using the shallow face feature to obtain a first recognized face, and returning the first recognized face and a corresponding first face similarity to the client, includes:
acquiring a face feature database in the server, and searching the face features in the face feature database according to the shallow face features to obtain the first identified face;
performing feature matching on the shallow face features and the first recognized face by using a preset approximate nearest neighbor algorithm to obtain a shallow feature binary tree;
calculating the similarity of adjacent nodes in the shallow feature binary tree to obtain the first similarity;
and returning the first similarity and the first recognized face to the client.
In order to solve the above problem, the present invention further provides a face recognition apparatus applied to a client, the apparatus comprising:
the video processing module is used for acquiring an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image;
the shallow layer extraction module is used for extracting shallow layer face features in the face image, transmitting the shallow layer face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server;
and the face recognition module is used for sending the preprocessed face image to a server side when the first face similarity is not within a preset first threshold range, receiving a second recognized face returned by the server side according to the face image and a corresponding second face similarity, and obtaining a final recognition result according to the second face similarity.
In order to solve the above problem, the present invention further provides a face recognition device applied to a server, where the face recognition device includes:
the shallow layer recognition module is used for receiving shallow layer face features sent by the client, performing face recognition by using the shallow layer face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client;
the deep extraction module is used for receiving the preprocessed face image sent by the client and carrying out deep feature extraction on the face image to obtain deep face features when the first face similarity is not within a preset first threshold range;
and the deep layer recognition module is used for carrying out face recognition by utilizing the deep layer face characteristics to obtain a second recognized face and returning the second recognized face and the corresponding second face similarity to the client.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one instruction; and
and the processor executes the instructions stored in the memory to realize the face recognition method.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned face recognition method.
According to the embodiment of the invention, the client and the server are deployed simultaneously, and the shallow extraction model is transferred to the client, so that the computing power of the client is fully utilized. And the superficial layer human face feature file has small volume, the pressure of the bandwidth is small when the file is transmitted, the corresponding speed is high, and the cost of the bandwidth is greatly reduced. The shallow feature extraction is carried out by utilizing the client, some calculations of the server are shared, and the concurrent calculation pressure of the server is reduced. Meanwhile, the shallow extraction model identifies the face which is easy to identify, and the face which is difficult to identify is delivered to the deep extraction model in the server, so that the face identification rate is improved, and the overall identification precision is high. Therefore, the face recognition method, the face recognition device and the computer readable storage medium provided by the invention can solve the problems of low face recognition rate and low precision.
Drawings
Fig. 1 is a schematic flow chart of a face recognition method according to a first embodiment of the present invention;
FIG. 2 is a schematic flow chart showing a detailed implementation of one of the steps in FIG. 1;
FIG. 3 is a schematic flow chart showing another step of FIG. 1;
fig. 4 is a schematic flow chart of a face recognition method according to a second embodiment of the present invention;
FIG. 5 is a schematic flow chart showing a detailed implementation of one of the steps in FIG. 4;
FIG. 6 is a functional block diagram of a face recognition apparatus according to a third embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device for implementing the face recognition method according to a fourth embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the present application provides a face recognition method, and an execution subject of the face recognition method includes but is not limited to at least one of electronic devices that can be configured to execute the method provided by the embodiment of the present application, such as a server and a terminal. In other words, the face recognition method may be executed by software or hardware installed in the terminal device or the server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
The method is characterized in that a client is used for extracting shallow face features in a face image, and a server firstly carries out face recognition according to the shallow face features; and when the result of face recognition according to the shallow face features is inaccurate, if the highest similarity of face recognition is smaller than a preset threshold, extracting the deep face features in the face image by the server side, and performing face recognition according to the deep face features. In the embodiment of the invention, the server side is used for comparing the face recognition, namely the face image feature library is stored in the server side, so that the risk of tampering is reduced.
The detailed implementation process of the present invention can be referred to the following description related to fig. 1 to 5.
Fig. 1 is a schematic flow chart illustrating a face recognition method according to a first embodiment of the present invention. In this embodiment, the face recognition method is applied to a client, and includes:
s11, obtaining an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image.
In the embodiment of the present invention, the original video stream may be acquired by using a camera device of a client. The client can be mobile equipment such as a mobile phone and a notebook computer.
Preferably, referring to fig. 2, the performing face detection on each frame of image in the original video stream, and performing alignment and clipping operations on the detected face image to obtain a preprocessed face image includes:
s110, carrying out face detection on each frame of image of the original video stream, and carrying out screenshot on the detected face image to obtain an initial face image;
s111, detecting the positions of the face feature points in the initial face image, and carrying out position-driven deformation on the positions of the face feature points to obtain an aligned face image;
the embodiment of the invention can detect the positions of the human face feature points in the initial human face image by a preset 3D correction method and carry out position-driven deformation on the positions of the human face feature points. The positions of the human face feature points comprise the positions of the left side of the nose, the lower side of the nostrils, the positions of the pupils, the lower side of the upper lip and the like.
S112, cutting the aligned face image into a picture with a preset size to obtain the preprocessed face image.
The preset size may be in an RGB format of 112 × 112. The embodiment of the invention cuts the face image into the picture with the preset size to obtain the preprocessed face image, thereby reducing the occupation of the storage space of the client and increasing the transmission speed of the face image.
S12, extracting shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server.
In the embodiment of the present invention, the shallow face features refer to Local features extracted from the face image, such as Local Binary Pattern (LBP) features, and the shallow face features are not sensitive to illumination and can be used for quickly identifying face features.
Preferably, referring to fig. 3, the extracting the shallow face features in the face image includes:
s120, graying and denoising the face image to obtain a processed picture;
and S121, extracting shallow face features in the processed picture by using a preset shallow extraction model.
The graying is to convert the face image from an RGB format to a gray scale format, and the conversion method comprises the following steps:
r, G, B pixel values in the face image are obtained;
converting the R, G, B pixel values to grayscale values using the following equation:
gray scale formula 0.30R + 0.59G + 0.11B
Further, the denoising operation is to perform denoising processing on the gray scale value based on an adaptive image denoising filtering method, where the adaptive image denoising filtering method is:
Figure BDA0002769791960000061
Figure BDA0002769791960000062
wherein, (x, y) represents the coordinates of pixel points of the face image, f (x, y) is output data after denoising processing is carried out on the gray scale value based on an adaptive image denoising filtering method, eta (x, y) is noise, g (x, y) is the gray scale value data,
Figure BDA0002769791960000063
is the total variance of the noise of the gray-scale value data,
Figure BDA0002769791960000064
is the pixel gray level mean value of (x, y),
Figure BDA0002769791960000065
and L represents the coordinate of the current pixel point, wherein the pixel gray variance of (x, y) is shown.
Preferably, in the embodiment of the present invention, the preset shallow extraction model may be a mobile device face recognition network (MobileFaceNets) model that is currently disclosed. The MobileFaceNet model comprises an input layer, a convolution layer, an average pooling layer and an output layer, wherein the input layer receives the processed picture, the convolution layer and the average pooling layer obtain the shallow face features, and the shallow face feature set is obtained through output of the output layer.
According to the embodiment of the invention, the shallow face features can be rapidly extracted through the preset shallow extraction model, and the face recognition rate is improved.
And S13, judging whether the first face similarity is within a preset first threshold range.
In this embodiment of the present invention, the first threshold range may include a plurality of sub-threshold ranges. The embodiment of the invention judges whether the first face similarity is in any sub-threshold range.
When the first face similarity falls into any sub-threshold range of the first threshold range, S14 is executed, and a corresponding recognition result is output.
In an embodiment of the present invention, the sub-threshold ranges of the first threshold range may include [ 0, 0.8 ] and [ 0.9,1 ]. When the first similarity is within the range of (0, 0.8), the face recognition is successful, and the first recognized face is the face obtained through the recognition. And when the first similarity is within the range of (0.9, 1), the obtained recognition result is that face recognition fails, and the corresponding face cannot be searched in the database.
Further, when the first face similarity is not within the preset first threshold range, S15 is executed, the preprocessed face image is sent to a server, and a second recognized face and a corresponding second face similarity returned by the server according to the face image are received.
And S16, judging whether the second face similarity is within a preset second threshold range.
When the second face similarity is within the second threshold range, S17 is executed and the output recognition is successful, and when the second face similarity is not within the second threshold range, S18 is executed and the output recognition is failed.
In the embodiment of the present invention, the second threshold range may be [ 0, 0.85 ].
When the second similarity is within the range of (0, 0.85), the face recognition is successful, and the second recognized face is the face obtained through the recognition. And when the second similarity is not within the range of (0, 0.85), the obtained final recognition result is that the face recognition fails, and the corresponding face cannot be searched in the database.
The embodiment of the invention transfers the shallow feature extraction model to the client, so that the client has certain face recognition capability, the computing capability of the client is effectively utilized, and the face recognition rate is greatly improved. Meanwhile, the face image with the first similarity not meeting the preset threshold range is sent to the server side to further accurately detect the deep face features, and a final recognition result is obtained according to the returned second face similarity, so that the face recognition precision is improved.
Fig. 4 is a flow chart illustrating a face recognition method according to a second embodiment of the present invention. In this embodiment, the face recognition method is applied to a server, and includes:
s21, receiving the shallow face features sent by the client, carrying out face recognition by using the shallow face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client.
Preferably, as shown in fig. 5, the performing face recognition by using the shallow face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client includes:
s210, acquiring a face feature database in the server, and searching the face features in the face feature database according to the shallow face features;
s211, performing feature matching on the shallow face features and the face features in the face feature database by using a preset approximate nearest neighbor algorithm to obtain a shallow feature binary tree;
preferably, in the embodiment of the present invention, the preset approximate nearest neighbor algorithm may use a currently disclosed anyyte algorithm, where the anyyte algorithm uses the shallow face features and the face features in the face feature database to construct a binary tree, and the more similar the face features in the shallow face features and the face features in the face feature database are, the more likely the face features are classified as adjacent nodes in the binary tree.
S212, calculating the similarity of adjacent nodes in the shallow feature binary tree to obtain a plurality of similarity values;
in the embodiment of the invention, the similarity is Euclidean distance, the Euclidean distance of the same face features is as small as possible, and the Euclidean distances of different face features are as large as possible.
In the embodiment of the present invention, the first similarity is calculated by the following formula:
Figure BDA0002769791960000081
d1 is the first similarity, n is the node pair number in the shallow feature binary tree, xi,yjFeatures in adjacent nodes of the shallow feature binary tree.
And S213, selecting one similarity value from the similarity values and the face corresponding to the similarity value as the first similarity and the first identified face according to a preset rule, and returning the first identified face to the client.
In an embodiment of the present invention, the preset rule is to select a minimum value from the plurality of similarity values. Because the volume of the shallow face feature file is smaller than that of the face image, the bandwidth pressure is small when the shallow face feature file is transmitted, and the corresponding transmission speed is high. Meanwhile, the similarity can be directly calculated according to a preset approximate nearest neighbor algorithm, so that the embodiment of the invention greatly improves the speed of face recognition.
When the first face similarity is not within the preset first threshold range, the embodiment of the present invention may execute S22, receive the preprocessed face image sent by the client, and perform deep feature extraction on the face image to obtain deep face features.
Preferably, in the embodiment of the present invention, a preset deep extraction model is used to perform deep feature extraction on the face image, the deep extraction model may be a presently disclosed ResNet101 model, and the ResNet101 includes 101 convolutional layers, so that the face feature extraction can be performed more accurately. The deep human face features are more complete and accurate human face features extracted based on methods such as deep learning and neural networks.
Furthermore, the embodiment of the invention can accurately extract the deep human face features through the preset deep extraction model, thereby improving the precision of human face recognition.
And S23, performing face recognition by using the deep face features to obtain a second recognized face, and returning the second recognized face and the corresponding second face similarity to the client.
In the embodiment of the present invention, the detailed execution process of S23 is similar to that described in fig. 5, except that the shallow facial features are replaced by the deep facial features, which is not described herein again.
According to the embodiment of the invention, the shallow extraction model is transferred to the client, so that the computing power of the client is fully utilized. And the superficial layer human face feature file has small volume, the pressure of the bandwidth is small when the file is transmitted, the corresponding speed is high, and the cost of the bandwidth is greatly reduced. The shallow feature extraction is carried out by utilizing the client, some calculations of the server are shared, and the concurrent calculation pressure of the server is reduced. Meanwhile, the shallow extraction model identifies the face which is easy to identify, and the face which is difficult to identify is delivered to the deep extraction model in the server, so that the face identification rate is improved, and the overall identification precision is high. Therefore, the face recognition method, the face recognition device and the computer readable storage medium provided by the invention can solve the problems of low face recognition rate and low precision.
Fig. 6 is a functional block diagram of a face recognition apparatus according to a third embodiment of the present invention.
The face recognition apparatus of the present invention may be divided into a first face recognition apparatus 100 and a second face recognition apparatus 200. Wherein the first face recognition apparatus 100 may be installed in a client and the second face recognition apparatus 200 may be installed in a server.
According to the implemented functions, the first face recognition device 100 may include a video processing module 101, a shallow layer extraction module 102, and a face recognition module 103; and the second face recognition device 200 may include a shallow recognition module 201, a deep extraction module 202, and a deep recognition module 203.
The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In this embodiment, the functions of the modules in the first face recognition apparatus 100 are as follows:
the video processing module 101 is configured to obtain an original video stream, perform face detection on each frame of image in the original video stream, and perform alignment and clipping operations on the detected face image to obtain a preprocessed face image.
In the embodiment of the present invention, the original video stream may be acquired by using a camera device of a client. The client can be mobile equipment such as a mobile phone and a notebook computer.
Preferably, the video processing module 101 obtains the preprocessed face image by:
carrying out face detection on each frame of image of the original video stream, and carrying out screenshot on the detected face image to obtain an initial face image;
detecting the positions of the face feature points in the initial face image, and carrying out position-driven deformation on the positions of the face feature points to obtain an aligned face image;
the embodiment of the invention can detect the positions of the human face feature points in the initial human face image by a preset 3D correction method and carry out position-driven deformation on the positions of the human face feature points. The positions of the human face feature points comprise the positions of the left side of the nose, the lower side of the nostrils, the positions of the pupils, the lower side of the upper lip and the like.
And cutting the aligned face image into a picture with a preset size to obtain the preprocessed face image.
The preset size may be in an RGB format of 112 × 112. The embodiment of the invention cuts the face image into the picture with the preset size to obtain the preprocessed face image, thereby reducing the occupation of the storage space of the client and increasing the transmission speed of the face image.
The shallow layer extraction module 102 is configured to extract a shallow layer face feature in the face image, transmit the shallow layer face feature to a server, and receive a first recognized face and a corresponding first face similarity returned by the server.
In the embodiment of the present invention, the shallow face features refer to Local features extracted from the face image, such as Local Binary Pattern (LBP) features, and the shallow face features are not sensitive to illumination and can be used for quickly identifying face features.
Preferably, the shallow layer extraction module 102 extracts a shallow layer face feature in the face image by:
carrying out graying and noise reduction operation on the face image to obtain a processed picture;
and extracting the shallow face features in the processed picture by using a preset shallow extraction model.
The graying is to convert the face image from an RGB format to a gray scale format, and the conversion method comprises the following steps:
r, G, B pixel values in the face image are obtained;
converting the R, G, B pixel values to grayscale values using the following equation:
gray scale formula 0.30R + 0.59G + 0.11B
Further, the denoising operation is to perform denoising processing on the gray scale value based on an adaptive image denoising filtering method, where the adaptive image denoising filtering method is:
Figure BDA0002769791960000111
Figure BDA0002769791960000112
wherein, (x, y) represents the coordinates of pixel points of the face image, f (x, y) is output data after denoising processing is carried out on the gray scale value based on an adaptive image denoising filtering method, eta (x, y) is noise, g (x, y) is the gray scale value data,
Figure BDA0002769791960000113
is the total variance of the noise of the gray-scale value data,
Figure BDA0002769791960000114
is the pixel gray level mean value of (x, y),
Figure BDA0002769791960000115
and L represents the coordinate of the current pixel point, wherein the pixel gray variance of (x, y) is shown.
Preferably, in the embodiment of the present invention, the preset shallow extraction model may be a mobile device face recognition network (MobileFaceNets) model that is currently disclosed. The MobileFaceNet model comprises an input layer, a convolution layer, an average pooling layer and an output layer, wherein the input layer receives the processed picture, the convolution layer and the average pooling layer obtain the shallow face features, and the shallow face feature set is obtained through output of the output layer.
According to the embodiment of the invention, the shallow face features can be rapidly extracted through the preset shallow extraction model, and the face recognition rate is improved.
The face recognition module 103 is configured to determine whether the first face similarity is within a preset first threshold range.
In this embodiment of the present invention, the first threshold range may include a plurality of sub-threshold ranges. The embodiment of the invention judges whether the first face similarity is in any sub-threshold range.
And when the first face similarity falls into any sub-threshold range of the first threshold range, outputting a corresponding recognition result.
In an embodiment of the present invention, the sub-threshold ranges of the first threshold range may include [ 0, 0.8 ] and [ 0.9,1 ]. When the first similarity is within the range of (0, 0.8), the face recognition is successful, and the first recognized face is the face obtained through the recognition. And when the first similarity is within the range of (0.9, 1), the obtained recognition result is that face recognition fails, and the corresponding face cannot be searched in the database.
Further, when the first face similarity is not within a preset first threshold range, the preprocessed face image is sent to a server, and a second recognized face and a corresponding second face similarity returned by the server according to the face image are received.
And judging whether the second face similarity is within a preset second threshold value range.
Outputting a recognition success when the second face similarity is within the second threshold range, and outputting a recognition failure when the second face similarity is not within the second threshold range.
In the embodiment of the present invention, the second threshold range may be [ 0, 0.85 ]
When the second similarity is within the range of (0, 0.85), the face recognition is successful, and the second recognized face is the face obtained through the recognition. And when the second similarity is not within the range of (0, 0.85), the obtained final recognition result is that the face recognition fails, and the corresponding face cannot be searched in the database.
The embodiment of the invention transfers the shallow feature extraction model to the client, so that the client has certain face recognition capability, the computing capability of the client is effectively utilized, and the face recognition rate is greatly improved. Meanwhile, the face image with the first similarity not meeting the preset threshold range is sent to the server side to further accurately detect the deep face features, and a final recognition result is obtained according to the returned second face similarity, so that the face recognition precision is improved.
In this embodiment, the functions of the modules in the second face recognition apparatus 200 are as follows:
the shallow layer recognition module 201 is configured to receive a shallow layer face feature sent by a client, perform face recognition by using the shallow layer face feature to obtain a first recognized face, and return the first recognized face and a corresponding first face similarity to the client.
Preferably, the shallow layer recognition module 201 performs face recognition by using the shallow layer face features to obtain a first recognized face, and returns the first recognized face and the corresponding first face similarity to the client:
acquiring a face feature database in the server, and searching the face features in the face feature database according to the shallow face features;
performing feature matching on the shallow face features and the face features in the face feature database by using a preset approximate nearest neighbor algorithm to obtain a shallow feature binary tree;
preferably, in the embodiment of the present invention, the preset approximate nearest neighbor algorithm may use a currently disclosed anyyte algorithm, where the anyyte algorithm uses the shallow face features and the face features in the face feature database to construct a binary tree, and the more similar the face features in the shallow face features and the face features in the face feature database are, the more likely the face features are classified as adjacent nodes in the binary tree.
Calculating the similarity of adjacent nodes in the shallow feature binary tree to obtain a plurality of similarity values;
in the embodiment of the invention, the similarity is Euclidean distance, the Euclidean distance of the same face features is as small as possible, and the Euclidean distances of different face features are as large as possible.
In the embodiment of the present invention, the first similarity is calculated by the following formula:
Figure BDA0002769791960000131
d1 is the first similarity, n is the node pair number in the shallow feature binary tree, xi,yjFor bits in adjacent nodes of the shallow feature binary treeAnd (5) carrying out characterization.
And selecting one similarity value from the similarity values and the face corresponding to the similarity value as the first similarity and the first identified face according to a preset rule, and returning the first identified face to the client.
In an embodiment of the present invention, the preset rule is to select a minimum value from the plurality of similarity values. Because the volume of the shallow face feature file is smaller than that of the face image, the bandwidth pressure is small when the shallow face feature file is transmitted, and the corresponding transmission speed is high. Meanwhile, the similarity can be directly calculated according to a preset approximate nearest neighbor algorithm, so that the embodiment of the invention greatly improves the speed of face recognition.
When the first face similarity is not within the preset first threshold range, the embodiment of the present invention may execute the deep extraction module 202 to perform deep feature extraction on the face image by receiving the preprocessed face image sent by the client, so as to obtain deep face features.
Preferably, in the embodiment of the present invention, a preset deep extraction model is used to perform deep feature extraction on the face image, the deep extraction model may be a presently disclosed ResNet101 model, and the ResNet101 includes 101 convolutional layers, so that the face feature extraction can be performed more accurately. The deep human face features are more complete and accurate human face features extracted based on methods such as deep learning and neural networks.
Furthermore, the embodiment of the invention can accurately extract the deep human face features through the preset deep extraction model, thereby improving the precision of human face recognition.
The deep recognition module 203 performs face recognition by using the deep face features to obtain a second recognized face, and returns the second recognized face and the corresponding second face similarity to the client.
In the embodiment of the present invention, the deep layer recognition module 203 is similar to the shallow layer recognition module 201, except that the shallow layer face features are replaced by the deep layer face features, which is not described herein again.
According to the embodiment of the invention, the shallow extraction model is transferred to the client, so that the computing power of the client is fully utilized. And the superficial layer human face feature file has small volume, the pressure of the bandwidth is small when the file is transmitted, the corresponding speed is high, and the cost of the bandwidth is greatly reduced. The shallow feature extraction is carried out by utilizing the client, some calculations of the server are shared, and the concurrent calculation pressure of the server is reduced. Meanwhile, the shallow extraction model identifies the face which is easy to identify, and the face which is difficult to identify is delivered to the deep extraction model in the server, so that the face identification rate is improved, and the overall identification precision is high. Therefore, the face recognition method, the face recognition device and the computer readable storage medium provided by the invention can solve the problems of low face recognition rate and low precision.
Fig. 7 is a schematic structural diagram of an electronic device for implementing the face recognition method according to a fourth embodiment of the present invention.
In this embodiment of the present invention, the electronic device 1 may be a server-side electronic device or a client-side electronic device. In detail, the electronic device 1 may include a processor 10, a memory 11 and a bus, and may further include a computer program, such as a face recognition program 12, stored in the memory 11 and executable on the processor 10.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of the face recognition program 12, but also to temporarily store data that has been output or is to be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., a face recognition program, etc.) stored in the memory 11 and calling data stored in the memory 11.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 7 only shows an electronic device with components, and it will be understood by a person skilled in the art that the structure shown in fig. 7 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The memory 11 of the electronic device 1 stores a face recognition program 12 that is a combination of instructions that, when executed in the processor 10, may be a face recognition method.
In detail, when the electronic device is a client electronic device, the face recognition method includes:
acquiring an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image;
extracting shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server;
and when the first face similarity is not within a preset first threshold range, sending the preprocessed face image to a server, receiving a second recognized face returned by the server according to the face image and a corresponding second face similarity, and obtaining a final recognition result according to the second face similarity.
Further, when the electronic device is a server-side electronic device, the face recognition method includes:
receiving shallow face features sent by a client, performing face recognition by using the shallow face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client;
when the first face similarity is not within a preset first threshold range, receiving a preprocessed face image sent by a client, and performing deep feature extraction on the face image to obtain deep face features;
and performing face recognition by using the deep face features to obtain a second recognized face, and returning the second recognized face and the corresponding second face similarity to the client.
Specifically, the specific implementation method of the processor 10 for the instruction may refer to the description of the relevant steps in the embodiments corresponding to fig. 1 to fig. 5, which is not repeated herein.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The storage medium may be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The invention also relates to the field of digital medical treatment, and the final recognition result of the face recognition can be applied to an intelligent medical treatment system.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims (10)

1. A face recognition method is applied to a client and comprises the following steps:
acquiring an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image;
extracting shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server;
and when the first face similarity is not within a preset first threshold range, sending the preprocessed face image to a server, receiving a second recognized face returned by the server according to the face image and a corresponding second face similarity, and obtaining a final recognition result according to the second face similarity.
2. The method of claim 1, wherein the performing face detection on each frame of image in the original video stream, aligning and cropping the detected face image to obtain a preprocessed face image comprises:
carrying out face detection on each frame of image of the original video stream, and carrying out screenshot on the detected face image to obtain an initial face image;
detecting the positions of the face feature points in the initial face image, and carrying out position-driven deformation on the positions of the face feature points to obtain an aligned face image;
and cutting the aligned face image into a picture with a preset size to obtain the preprocessed face image.
3. The method for recognizing human face according to claim 1, wherein the extracting the shallow human face features in the human face image comprises:
carrying out graying and noise reduction operation on the face image to obtain a processed picture;
and extracting the shallow face features in the processed picture by using a preset shallow extraction model.
4. The method of claim 1, wherein after extracting the shallow face features in the face image, transmitting the shallow face features to a server, and receiving a first recognized face returned by the server and a corresponding first face similarity, the method further comprises:
and when the first face similarity falls into any sub-threshold range of the first threshold range, outputting a corresponding recognition result.
5. A face recognition method is applied to a server and comprises the following steps:
receiving shallow face features sent by a client, performing face recognition by using the shallow face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client;
when the first face similarity is not within a preset first threshold range, receiving a preprocessed face image sent by a client, and performing deep feature extraction on the face image to obtain deep face features;
and performing face recognition by using the deep face features to obtain a second recognized face, and returning the second recognized face and the corresponding second face similarity to the client.
6. The method of claim 5, wherein the receiving a shallow face feature sent by a client, performing face recognition by using the shallow face feature to obtain a first recognized face, and returning the first recognized face and a corresponding first face similarity to the client comprises:
acquiring a face feature database in the server, and searching the face features in the face feature database according to the shallow face features to obtain the first identified face;
performing feature matching on the shallow face features and the first recognized face by using a preset approximate nearest neighbor algorithm to obtain a shallow feature binary tree;
calculating the similarity of adjacent nodes in the shallow feature binary tree to obtain the first similarity;
and returning the first similarity and the first recognized face to the client.
7. A face recognition apparatus, applied to a client, the apparatus comprising:
the video processing module is used for acquiring an original video stream, carrying out face detection on each frame of image in the original video stream, and carrying out alignment and cutting operation on the detected face image to obtain a preprocessed face image;
the shallow layer extraction module is used for extracting shallow layer face features in the face image, transmitting the shallow layer face features to a server, and receiving a first recognized face and corresponding first face similarity returned by the server;
and the face recognition module is used for sending the preprocessed face image to a server side when the first face similarity is not within a preset first threshold range, receiving a second recognized face returned by the server side according to the face image and a corresponding second face similarity, and obtaining a final recognition result according to the second face similarity.
8. A face recognition device, which is applied to a server, the face recognition device comprising:
the shallow layer recognition module is used for receiving shallow layer face features sent by the client, performing face recognition by using the shallow layer face features to obtain a first recognized face, and returning the first recognized face and the corresponding first face similarity to the client;
the deep extraction module is used for receiving the preprocessed face image sent by the client and carrying out deep feature extraction on the face image to obtain deep face features when the first face similarity is not within a preset first threshold range;
and the deep layer recognition module is used for carrying out face recognition by utilizing the deep layer face characteristics to obtain a second recognized face and returning the second recognized face and the corresponding second face similarity to the client.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a face recognition method as claimed in any one of claims 1 to 6.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a face recognition method according to any one of claims 1 to 6.
CN202011251504.2A 2020-11-10 2020-11-10 Face recognition method and device, electronic equipment and storage medium Pending CN112329666A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011251504.2A CN112329666A (en) 2020-11-10 2020-11-10 Face recognition method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011251504.2A CN112329666A (en) 2020-11-10 2020-11-10 Face recognition method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112329666A true CN112329666A (en) 2021-02-05

Family

ID=74318897

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011251504.2A Pending CN112329666A (en) 2020-11-10 2020-11-10 Face recognition method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112329666A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113705469A (en) * 2021-08-30 2021-11-26 平安科技(深圳)有限公司 Face recognition method and device, electronic equipment and computer readable storage medium
US20220180089A1 (en) * 2020-12-04 2022-06-09 Qnap Systems, Inc. Human face identification apparatus, distributed human face identification system and method thereof

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108614970A (en) * 2018-04-03 2018-10-02 腾讯科技(深圳)有限公司 Detection method, model training method, device and the equipment of Virus
CN110705451A (en) * 2019-09-27 2020-01-17 支付宝(杭州)信息技术有限公司 Face recognition method, face recognition device, terminal and server
CN111223163A (en) * 2020-01-07 2020-06-02 苏州瑞派宁科技有限公司 Image reconstruction method, device, equipment, system and computer readable storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108614970A (en) * 2018-04-03 2018-10-02 腾讯科技(深圳)有限公司 Detection method, model training method, device and the equipment of Virus
CN110705451A (en) * 2019-09-27 2020-01-17 支付宝(杭州)信息技术有限公司 Face recognition method, face recognition device, terminal and server
CN111223163A (en) * 2020-01-07 2020-06-02 苏州瑞派宁科技有限公司 Image reconstruction method, device, equipment, system and computer readable storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
邓畅等: "深度学习在云和终端上的混合分布式部署研究", 智能物联技术, vol. 51, no. 1, 31 January 2019 (2019-01-31), pages 12 - 17 *
韦智勇等: "基于人脸识别的移动终端身份认证系统的设计与实现", 企业科技与发展, no. 6, 31 December 2018 (2018-12-31), pages 42 - 45 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220180089A1 (en) * 2020-12-04 2022-06-09 Qnap Systems, Inc. Human face identification apparatus, distributed human face identification system and method thereof
US11514718B2 (en) * 2020-12-04 2022-11-29 Qnap Systems, Inc. Human face identification apparatus, distributed human face identification system and method thereof
CN113705469A (en) * 2021-08-30 2021-11-26 平安科技(深圳)有限公司 Face recognition method and device, electronic equipment and computer readable storage medium

Similar Documents

Publication Publication Date Title
CN112395978B (en) Behavior detection method, behavior detection device and computer readable storage medium
CN113283446B (en) Method and device for identifying object in image, electronic equipment and storage medium
CN112036292B (en) Word recognition method and device based on neural network and readable storage medium
CN112396005A (en) Biological characteristic image recognition method and device, electronic equipment and readable storage medium
CN113705462B (en) Face recognition method, device, electronic equipment and computer readable storage medium
CN111476225B (en) In-vehicle human face identification method, device, equipment and medium based on artificial intelligence
CN112581227A (en) Product recommendation method and device, electronic equipment and storage medium
CN111639704A (en) Target identification method, device and computer readable storage medium
CN113033543A (en) Curved text recognition method, device, equipment and medium
CN111931729B (en) Pedestrian detection method, device, equipment and medium based on artificial intelligence
CN112329666A (en) Face recognition method and device, electronic equipment and storage medium
CN112528903B (en) Face image acquisition method and device, electronic equipment and medium
CN113705469A (en) Face recognition method and device, electronic equipment and computer readable storage medium
CN113704474A (en) Bank outlet equipment operation guide generation method, device, equipment and storage medium
CN115409041B (en) Unstructured data extraction method, device, equipment and storage medium
CN115798004A (en) Face card punching method and device based on local area, electronic equipment and medium
CN113515591B (en) Text defect information identification method and device, electronic equipment and storage medium
CN115114992A (en) Method, device and equipment for training classification model and storage medium
CN114677526A (en) Image classification method, device, equipment and medium
CN112434601A (en) Vehicle law violation detection method, device, equipment and medium based on driving video
CN114187476A (en) Vehicle insurance information checking method, device, equipment and medium based on image analysis
CN113505698A (en) Face recognition method, device, equipment and medium based on counterstudy
CN112561893A (en) Picture matching method and device, electronic equipment and storage medium
CN112580505A (en) Method and device for identifying opening and closing states of network points, electronic equipment and storage medium
CN112861649A (en) Fingerprint signature generation method and device, electronic equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination