CN111160202A - AR equipment-based identity verification method, AR equipment-based identity verification device, AR equipment-based identity verification equipment and storage medium - Google Patents
AR equipment-based identity verification method, AR equipment-based identity verification device, AR equipment-based identity verification equipment and storage medium Download PDFInfo
- Publication number
- CN111160202A CN111160202A CN201911341990.4A CN201911341990A CN111160202A CN 111160202 A CN111160202 A CN 111160202A CN 201911341990 A CN201911341990 A CN 201911341990A CN 111160202 A CN111160202 A CN 111160202A
- Authority
- CN
- China
- Prior art keywords
- face
- verified
- image
- face image
- verification result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Collating Specific Patterns (AREA)
- Image Analysis (AREA)
Abstract
The application provides an identity verification method, an identity verification device, identity verification equipment and a storage medium based on AR equipment, wherein the method comprises the following steps: receiving a field real-time video acquired by preset image acquisition equipment; carrying out face detection on the live real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified; sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result; and receiving the verification result returned by the server, and displaying the verification result in an information display window. The embodiment of the application can meet the real-time performance of security workers in personnel identity verification.
Description
Technical Field
The present application relates to the field of computer graphics processing technologies, and in particular, to an identity verification method, apparatus, device, and storage medium based on an AR device.
Background
Wisdom garden aims at realizing the interconnection between each plate in traditional garden through internet of things to effectively integrate with the help of technologies such as cloud computing, big data, make garden infrastructure operation more intelligent, the management is more high-efficient, the security is more guaranteed. At present some gardens have realized the intellectuality of certain degree, but still do not reduce the dependence to the manpower, on the garden security problem, still need the staff to check the identity of different personnel in the region, for example: security personnel need to prevent irrelevant suspicious people from entering areas which are not allowed to enter, foreground personnel need to confirm and guide information of visitors coming to the park, and the like. The staff usually checks the identity of the person who goes in or out by means of a mobile phone or other terminal equipment, but the real-time requirement cannot be met.
Disclosure of Invention
In view of the above technical problems, the present application provides an identity verification method, apparatus, device and storage medium based on AR device, which can meet the real-time performance of security personnel in performing personnel identity verification.
In order to achieve the above object, a first aspect of the embodiments of the present application provides an identity verification method based on an AR device, where the method includes:
receiving a field real-time video acquired by preset image acquisition equipment;
carrying out face detection on the live real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified;
sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and receiving the verification result returned by the server, and displaying the verification result in an information display window.
With reference to the first aspect, in a possible implementation manner, the performing, by a face detection algorithm, face detection on the live real-time video includes:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
With reference to the first aspect, in a possible implementation manner, the tracking the detected face to acquire a face image to be verified includes:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
With reference to the first aspect, in a possible implementation manner, after the receiving the live real-time video captured by the preset image capturing device, the method further includes:
and storing the field real-time video to a preset storage space, and setting the storage duration of the field real-time video.
A second aspect of the present application provides a method for uploading a face image, where the method includes:
receiving a face image to be verified sent by AR equipment;
matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and returning the verification result to the AR equipment, so that the AR equipment displays the verification result on an information display window.
With reference to the second aspect, in a possible implementation manner, before the matching the facial image to be verified with the standard facial image stored in the facial database to obtain the verification result, the method further includes:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
With reference to the second aspect, in a possible implementation manner, the matching the facial image to be verified with a standard facial image stored in a facial database to obtain a verification result includes:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
With reference to the second aspect, in a possible implementation manner, before the receiving the live real-time video captured by the preset image capturing device, the method further includes:
collecting standard face images of all users, and extracting the second feature vector of the standard face image of each user;
and correspondingly storing the standard face image of each user, the second feature vector of the standard face image of each user and the identity information of each user to the face database.
A third aspect of the embodiments of the present application provides an identity verification apparatus based on an AR device, where the apparatus includes:
the video acquisition module is used for receiving a field real-time video acquired by preset image acquisition equipment;
the face snapshot module is used for carrying out face detection on the on-site real-time video through a face detection algorithm and tracking the detected face to acquire a face image to be verified;
the image sending module is used for sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the information display module is used for receiving the verification result returned by the server and displaying the verification result on an information display window.
The fourth aspect of the embodiments of the present application further provides an identity verification apparatus based on an AR device, where the apparatus includes:
the image receiving module is used for receiving the face image to be verified sent by the AR equipment;
the face comparison module is used for matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the result sending module is used for returning the verification result to the AR equipment so that the AR equipment displays the verification result in an information display window.
A fifth aspect of embodiments of the present application provides an electronic device, which includes an input device, an output device,
a processor adapted to implement one or more instructions; and the number of the first and second groups,
a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to perform the above-described AR device-based identity verification method.
A sixth aspect of embodiments of the present application provides a computer storage medium storing one or more instructions adapted to be loaded by a processor and execute the above method for verifying an identity based on an AR device.
The above scheme of the present application includes at least the following beneficial effects: compared with the prior art, the method and the device have the advantages that the on-site real-time video collected by the preset image collecting device is received; then, carrying out face detection on the on-site real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified; sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result; and receiving the verification result returned by the server, and displaying the verification result in an information display window. Therefore, the AR equipment is adopted for video acquisition, meanwhile, the face detection and the face tracking are carried out through an integrated algorithm in the AR equipment, and the interaction between the AR equipment and the server is realized based on a fifth-generation mobile communication technology, so that the real-time performance of security staff in the process of identity verification can be met.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a diagram of a network system architecture according to an embodiment of the present application;
fig. 2 is a schematic flowchart of an identity verification method based on an AR device according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a multitask convolutional neural network provided in an embodiment of the present application;
FIG. 4 is an exemplary diagram of an image pyramid provided in an embodiment of the present application;
fig. 5 is a schematic flowchart of another identity verification method based on an AR device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an identity verification apparatus based on an AR device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of another identity verification apparatus based on an AR device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "comprising" and "having," and any variations thereof, as appearing in the specification, claims and drawings of this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. Furthermore, the terms "first," "second," and "third," etc. are used to distinguish between different objects and are not used to describe a particular order.
First, a network system architecture to which the solution of the embodiments of the present application may be applied will be described by way of example with reference to the accompanying drawings. Referring to fig. 1, fig. 1 is a network system architecture diagram provided in an embodiment of the present application, and as shown in fig. 1, the system architecture at least includes AR glasses, a server, and an information transmission module based on a 5G (5th generation mobile communication technology) network architecture. The AR glasses comprise a video acquisition module, a face snapshot module, an information display module and a video storage module, the video acquisition module mainly refers to a camera arranged in the AR glasses, the camera is used for acquiring real-time videos of security workers in an identity verification site, the face snapshot module is mainly used for intercepting face images from videos acquired by the video acquisition module, the face images needing to be used are sent to a server through an information transmission module, the video storage module is mainly used for storing videos acquired by the video acquisition module into a storage card arranged in the AR glasses, and the information display module is mainly used for displaying verification results of the server on the AR glasses. The server comprises but is not limited to an image preprocessing module, an information input module, a face comparison module and a face library module, wherein the image preprocessing module is used for carrying out preprocessing such as denoising and enhancing on a face image sent by the AR glasses, the information input module is used for providing a worker or a user needing to register information for carrying out standard face image and identity information input, the face comparison module is mainly used for face recognition, the face image sent by the AR glasses is compared with the standard face image in the face library module to obtain a verification result, the verification result is returned to the AR glasses through the information transmission module, the face library module stores the standard face image of the user, the identity information and a face feature vector, and the server can be selected as a far-end server. The information transmission module is established on a 5G network, has the advantages of large bandwidth, low delay and low power consumption, and can meet the requirements of high concurrency, low delay and power saving of application. Based on above-mentioned network system framework, AR glasses and the mutual time delay greatly reduced of server have effectively fallen and have shortened the staff to the personnel that get into the garden and carry out the consuming time of identity verification, and the real-time is guaranteed.
Based on the above description, please refer to fig. 2, where fig. 2 is a schematic flowchart of an identity verification method based on an AR device according to an embodiment of the present application, where the identity verification method based on an AR device is applied to the AR device, and the AR device is an execution subject, as shown in fig. 2, including steps S21-S24:
s21, receiving a field real-time video collected by preset image collecting equipment;
in the embodiment of the application, the AR equipment is AR glasses, a chip is arranged in the AR equipment, the image acquisition equipment is a camera arranged in the AR equipment, the resolution of the camera is selected according to an actual using scene, the resolution of an outdoor scene is generally larger than that of an indoor scene, meanwhile, the resolution of the camera meets the requirement of a server for face comparison, the light measuring mode of the camera is set as central point light measurement, and the face of a person to be checked is ensured to be positioned in the central area of a picture when the light measuring mode of the camera is used; in order to ensure that the pictures do not generate dynamic blurring during verification, the shutter parameter of the camera is generally set within 1/200 seconds; the aperture can be dynamically adjusted according to the shutter speed, so long as the exposure of the picture is ensured within a reasonable range. When a person enters a certain specific area, the worker in charge of duty can aim the AR equipment at the face area of the person as much as possible to collect the real-time video on site.
S22, performing face detection on the live real-time video through a face detection algorithm, and tracking the detected face to acquire a face image to be checked;
in a specific embodiment of the present application, the AR device integrates a face detection algorithm, for example: an AdaBoost frame-based target detection algorithm, a deformable component model algorithm, a Multi-task convolutional neural network algorithm (MTCNN), a yolo target detection algorithm and the like, wherein in view of the requirements of the scheme on real-time performance and power consumption, a pre-trained Multi-task convolutional neural network can be adopted for face detection.
In a possible implementation manner, the performing face detection on the live real-time video through the face detection algorithm includes:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
Specifically, as shown in fig. 3, the structure of the multitask convolutional neural network includes P-NET, R-NET and O-NET, where the output of the first sub-network, i.e., P-NET, the output of the second sub-network, i.e., R-NET, and the output of the third sub-network, i.e., O-NET, include the classification of whether a face is present, the bounding box regression, and the face key point. Target image frames, namely image frames of a target object appearing in a live real-time video for the first time, inputting the target image frames into P-NET for primary face recognition and key point positioning to obtain a plurality of initial candidate frames, calibrating the initial candidate frames by using bounding box regression, and then performing non-maximum value inhibition on the initial candidate frames to obtain a certain number of first candidate frames. And taking the output of the P-NET as the input of the R-NET to optimize the detection result of the P-NET, specifically, filtering out the candidate frames which do not meet the requirements in the first candidate frame to obtain a small number of second candidate frames, taking the output of the R-NET as the input of the O-NET to perform final face recognition and key point detection, continuously filtering the second candidate frames to obtain the center point coordinates, the width and the length of the final face detection frame, and simultaneously displaying the key points of the face in the target image frame.
In a possible implementation manner, the tracking the detected face to acquire the image of the face to be verified includes:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
Specifically, after a face detection frame is obtained by adopting a face detection algorithm, a certain number of points can be selected in the face detection frame for tracking, preferably, a face centroid, positioned face key points and the like can be selected, then a target image frame is converted and scaled to construct an image pyramid shown in fig. 4, then the position of the face centroid in each layer of image is positioned, an optical flow in each layer of image is calculated, an optical flow of the face centroid in the bottom layer image (namely, the target image frame) of the image pyramid is obtained through layer-by-layer iteration, a corresponding point in the next frame of the face centroid is obtained according to the position of the face centroid in the target image frame and the optical flow, then the corresponding point of the face centroid in the next frame is matched with the face centroid by adopting a Hungarian algorithm to obtain an optimal corresponding point, and the optimal corresponding point is used for predicting the position of a human face in the next frame in the target image frame, to enable tracking. In the tracking process, a plurality of face images can be intercepted, then at least one face image with better quality is determined from the plurality of face images by detecting the definition, the size and the like of each face image, and the face image is used as a face image to be verified.
S23, sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
in the embodiment of the application, after the face image to be checked is selected by the AR device, the face image to be checked is sent to the server through the information transmission module, after the face image to be checked is received by the server, a series of preprocessing operations can be carried out on the face image to be checked, then the face image to be checked is matched with the standard face image stored in the face database, a preset number of users with high matching degree are selected, and the identity information of the preset number of users with high matching degree is used as a checking result.
And S24, receiving the verification result returned by the server, and displaying the verification result on an information display window.
In the specific embodiment of the application, the information display window is the display screen of the AR device, the server returns the verification result to the AR device, the AR device analyzes the verification result, the structured verification result is displayed on the information display window, the actual picture and the structured information are overlapped, and the identity of the target person can be quickly verified only by looking at the target person by a worker.
According to the embodiment of the application, the on-site real-time video collected by the preset image collecting device is received; then, carrying out face detection on the on-site real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified; sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result; and receiving the verification result returned by the server, and displaying the verification result in an information display window. Therefore, the AR equipment is adopted for video acquisition, meanwhile, the face detection and the face tracking are carried out through an integrated algorithm in the AR equipment, and the interaction between the AR equipment and the server is realized based on a fifth-generation mobile communication technology, so that the real-time performance of security staff in the process of identity verification can be met.
Referring to fig. 5, fig. 5 is a schematic flowchart of another AR device-based identity verification method according to an embodiment of the present application, where the AR device-based identity verification method is applied to a server, as shown in fig. 5, and includes steps S51-S53:
s51, receiving a face image to be verified sent by the AR equipment;
s52, matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
s53, returning the verification result to the AR device, and enabling the AR device to display the verification result in an information display window.
In the embodiment of the application, the standard face images stored in the face database are all face images of registered users, and the registered users have the authority of going in and out of a park or a specific area. When the face matching is performed, a method based on geometric features, a method of local feature analysis, a eigenface method, a neural network method, and the like may be specifically used, which is not limited here, for example: feature points can be selected by adopting size-invariant feature transformation to carry out face matching, or feature extraction can be carried out by adopting a convolutional neural network to carry out face matching, and finally, identity information of a preset number of users with high matching degree is used as a verification result and returned to the AR equipment through an information transmission module.
In a possible implementation manner, before the matching the facial image to be verified with the standard facial image stored in the facial database to obtain the verification result, the method further includes:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
In the embodiment of the application, in order to improve the matching accuracy, the face image to be checked needs to be preprocessed before face matching. The Gaussian filter scans each pixel in a face image to be checked through a template, replaces the value of a pixel point in the center of the template with a weighted average gray value of the pixel in the neighborhood determined by the template, is suitable for eliminating Gaussian noise points, enhances the face image to be checked with the noise points eliminated through a histogram equalization algorithm so as to improve the definition, finally inputs the enhanced face image to be checked into a trained multitask convolution neural network to position five face key points of two eyes, a nose and left and right mouth angles, and corrects the face based on the coordinates of the five face key points. In some examples, a super-resolution algorithm may be added to the preprocessing to further increase the resolution of the face image to be verified.
In a possible implementation manner, the matching the facial image to be verified with the standard facial image stored in the facial database to obtain the verification result includes:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
In the embodiment of the application, the convolutional neural network is adopted to extract the features of the face image to be verified after face correction to obtain the multidimensional first feature vector, meanwhile, in order to improve the matching speed, the scheme can extract the second feature vector of the standard face image in the face database in advance and then correspondingly store the second feature vector, and the time consumption for extracting the second feature vector during matching is reduced. And calculating cosine similarity between the first eigenvector and the second eigenvector by adopting a cosine calculation formula, sequencing the first eigenvector and the second eigenvector from high to low according to the cosine similarity, and selecting the identity information of the users corresponding to a preset number of standard face images as a verification result, wherein the preset number can be set according to the actual condition.
It can be seen that, in the embodiment of the application, the face image to be verified sent by the AR equipment is received; matching the face image to be verified with a standard face image stored in a face database to obtain a verification result; and returning the verification result to the AR equipment, so that the AR equipment displays the verification result on an information display window, and the real-time performance of security staff in personnel identity verification can be met.
Based on the description of the embodiment of the method in fig. 2, please refer to fig. 6, and fig. 6 is a schematic structural diagram of an identity verification apparatus based on AR devices according to an embodiment of the present application, as shown in fig. 6, the apparatus includes:
the video acquisition module 61 is used for receiving a field real-time video acquired by preset image acquisition equipment;
the face snapshot module 62 is configured to perform face detection on the live real-time video through a face detection algorithm, and track a detected face to obtain a face image to be verified;
the image sending module 63 is configured to send the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the information display module 64 is configured to receive the verification result returned by the server, and display the verification result in an information display window.
In a possible example, the face snapshot module 62 performs face detection on the live real-time video through a face detection algorithm, specifically to:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
In a possible example, the face snapshot module 62 is specifically configured to, in terms of tracking the detected face to acquire the face image to be verified:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
Based on the description of the embodiment of the method in fig. 5, please refer to fig. 7, fig. 7 is a schematic structural diagram of another identity verification apparatus based on AR devices according to the embodiment of the present application, and as shown in fig. 7, the apparatus includes:
the image receiving module 71 is configured to receive a face image to be verified sent by the AR device;
the face comparison module 72 is configured to match the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the result sending module 73 is configured to return the verification result to the AR device, so that the AR device displays the verification result in an information display window.
In a possible example, before the face comparison module 72 matches the face image to be verified with the standard face image stored in the face database to obtain the verification result, the face comparison module is further configured to:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
In a possible example, the face comparison module 72 is specifically configured to, in terms of matching the face image to be verified with a standard face image stored in a face database to obtain a verification result:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
According to an embodiment of the present application, each unit in the AR device-based identity verification apparatus shown in fig. 6 and fig. 7 may be respectively or entirely combined into one or several other units to form the unit, or some unit(s) therein may be further split into multiple units with smaller functions to form the unit(s), which may implement the same operation without affecting implementation of technical effects of embodiments of the present invention. The units are divided based on logic functions, and in practical application, the functions of one unit can be realized by a plurality of units, or the functions of a plurality of units can be realized by one unit. In other embodiments of the present invention, the AR device-based identity verification apparatus may also include other units, and in practical applications, these functions may also be implemented by being assisted by other units, and may be implemented by cooperation of multiple units.
According to another embodiment of the present application, the AR device-based identity verification apparatus device as shown in fig. 6 or fig. 7 may be constructed by running a computer program (including program codes) capable of executing the steps involved in the corresponding method as shown in fig. 2 or fig. 5 on a general-purpose computing device, such as a computer, including a processing element and a storage element, such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and the like, and the face image uploading method of the embodiment of the present invention may be implemented. The computer program may be recorded on a computer-readable recording medium, for example, and loaded and executed in the above-described computing apparatus via the computer-readable recording medium.
Based on the description of the method embodiment and the device embodiment, the embodiment of the invention also provides electronic equipment. Referring to fig. 8, the electronic device includes at least a processor 81, an input device 82, an output device 83, and a computer storage medium 84. The processor 81, input device 82, output device 83, and computer storage medium 84 within the electronic device may be connected by a bus or other means.
A computer storage medium 84 may be stored in the memory of the electronic device, the computer storage medium 84 being for storing a computer program comprising program instructions, the processor 81 being for executing the program instructions stored by the computer storage medium 84. The processor 81 (or CPU) is a computing core and a control core of the electronic device, and is adapted to implement one or more instructions, and in particular, is adapted to load and execute the one or more instructions so as to implement a corresponding method flow or a corresponding function.
In one embodiment, the processor 81 of the electronic device provided in this embodiment of the present application may be configured to perform a series of identity verification processes based on the AR device, including:
receiving a field real-time video acquired by preset image acquisition equipment;
carrying out face detection on the live real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified;
sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and receiving the verification result returned by the server, and displaying the verification result in an information display window.
In one possible example, the processor 81 executes the face detection on the live real-time video through the face detection algorithm, including:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
In one possible example, the processor 81 performs the tracking on the detected face to acquire the face image to be verified, including:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
In an embodiment, the processor 81 of the electronic device provided in this embodiment of the present application may be further configured to perform another series of identity verification processes based on the AR device, including:
receiving a face image to be verified sent by AR equipment;
matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and returning the verification result to the AR equipment, so that the AR equipment displays the verification result on an information display window.
In one possible example, the processor 81 is further configured to, before the matching of the face image to be verified with the standard face image stored in the face database to obtain the verification result, perform:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
In one possible example, the matching of the face image to be verified and the standard face image stored in the face database by the processor 81 to obtain the verification result includes:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
For example, the electronic device may be an AR device or a server, a cloud server, an edge server, or the like. The electronic devices may include, but are not limited to, a processor 81, an input device 82, an output device 83, and a computer storage medium 84. It will be appreciated by those skilled in the art that the schematic diagrams are merely examples of an electronic device and are not limiting of an electronic device and may include more or fewer components than those shown, or some components in combination, or different components.
It should be noted that, since the steps in the above-mentioned method for verifying the identity based on the AR device are implemented when the processor 81 of the electronic device executes the computer program, the embodiments or implementations of the method for verifying the identity based on the AR device are all applicable to the electronic device, and all can achieve the same or similar beneficial effects.
An embodiment of the present application further provides a computer storage medium (Memory), which is a Memory device in an electronic device and is used to store programs and data. It is understood that the computer storage medium herein may include a built-in storage medium in the terminal, and may also include an extended storage medium supported by the terminal. The computer storage medium provides a storage space that stores an operating system of the terminal. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), suitable for loading and execution by processor 81. The computer storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one disk memory; alternatively, it may be at least one computer storage medium located remotely from the processor 81. In one embodiment, one or more instructions stored in a computer storage medium may be loaded and executed by processor 81 to perform the corresponding steps described above with respect to the AR device-based identity verification method; in particular implementations, one or more instructions in the computer storage medium are loaded by processor 81 and perform the following steps:
receiving a field real-time video acquired by preset image acquisition equipment;
carrying out face detection on the live real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified;
sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and receiving the verification result returned by the server, and displaying the verification result in an information display window.
In one possible example, one or more instructions in the computer storage medium, when loaded by processor 81, are further operable to perform the steps of:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
In one possible example, one or more instructions in the computer storage medium, when loaded by processor 81, are further operable to perform the steps of:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
In another specific implementation of the embodiment, one or more instructions in the computer storage medium are loaded by the processor 81 and perform the following steps:
receiving a face image to be verified sent by AR equipment;
matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and returning the verification result to the AR equipment, so that the AR equipment displays the verification result on an information display window.
In one possible example, one or more instructions in the computer storage medium, when loaded by processor 81, are further operable to perform the steps of:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
In one possible example, one or more instructions in the computer storage medium, when loaded by processor 81, are further operable to perform the steps of:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
Illustratively, the computer storage medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
It should be noted that, since the computer program of the computer storage medium is executed by the processor to implement the steps in the above-mentioned identity verification method based on the AR device, all embodiments or implementations of the above-mentioned identity verification method based on the AR device are applicable to the computer storage medium, and can achieve the same or similar beneficial effects.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. An identity verification method based on an AR device, the method comprising:
receiving a field real-time video acquired by preset image acquisition equipment;
carrying out face detection on the live real-time video through a face detection algorithm, and tracking the detected face to obtain a face image to be verified;
sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and receiving the verification result returned by the server, and displaying the verification result in an information display window.
2. The method of claim 1, wherein the performing face detection on the live real-time video through a face detection algorithm comprises:
inputting the target image frame of the live real-time video into a first sub-network of a pre-trained multitask convolutional neural network for face recognition to obtain a first candidate frame;
filtering the first candidate frame by using a second sub-network of the multitask convolutional neural network to obtain a second candidate frame;
and filtering the second candidate frame and positioning the key points of the human face again by utilizing a second sub-network of the multitask convolutional neural network to obtain a human face detection frame of the human face in the target image frame.
3. The method according to claim 2, wherein the tracking the detected face to obtain the image of the face to be verified comprises:
calculating a corresponding point of the face centroid in the next frame of the target image frame by taking the center of the face detection frame as the face centroid and adopting a pyramid optical flow algorithm;
matching the face centroid with the corresponding point by adopting a Hungarian algorithm to track the detected face;
and intercepting a plurality of face images in the process of tracking the detected face, and determining the face image to be verified from the plurality of face images.
4. An identity verification method based on an AR device, the method comprising:
receiving a face image to be verified sent by AR equipment;
matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and returning the verification result to the AR equipment, so that the AR equipment displays the verification result on an information display window.
5. The method according to claim 4, wherein before matching the face image to be verified with the standard face image stored in the face database to obtain a verification result, the method further comprises:
eliminating noise points on the face image to be checked by adopting a Gaussian filter;
enhancing the denoised face image to be verified to improve the definition of the face image to be verified;
and correcting the human face in the human face image to be checked with enhanced definition by utilizing a pre-trained multitask convolution neural network.
6. The method according to claim 5, wherein the matching the face image to be verified with the standard face image stored in the face database to obtain the verification result comprises:
extracting a first feature vector of the face image to be verified after face correction;
acquiring a second feature vector of a stored standard face image from the face database, and calculating cosine similarity between the first feature vector and the second feature vector;
and taking the identity information of the user corresponding to the first N standard face images as the verification result according to the sequence of the cosine similarity from large to small.
7. An identity verification apparatus based on an AR device, the apparatus comprising:
the video acquisition module is used for receiving a field real-time video acquired by preset image acquisition equipment;
the face snapshot module is used for carrying out face detection on the on-site real-time video through a face detection algorithm and tracking the detected face to acquire a face image to be verified;
the image sending module is used for sending the face image to be verified to a server, so that the server matches the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the information display module is used for receiving the verification result returned by the server and displaying the verification result on an information display window.
8. An identity verification apparatus based on an AR device, the apparatus comprising:
the image receiving module is used for receiving the face image to be verified sent by the AR equipment;
the face comparison module is used for matching the face image to be verified with a standard face image stored in a face database to obtain a verification result;
and the result sending module is used for returning the verification result to the AR equipment so that the AR equipment displays the verification result in an information display window.
9. An electronic device comprising an input device and an output device, further comprising:
a processor adapted to implement one or more instructions; and the number of the first and second groups,
a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to perform the method of AR device based identity verification according to any of claims 1-6.
10. A computer storage medium having stored thereon one or more instructions adapted to be loaded by a processor and to perform the AR device based identity verification method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911341990.4A CN111160202B (en) | 2019-12-20 | 2019-12-20 | Identity verification method, device, equipment and storage medium based on AR equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911341990.4A CN111160202B (en) | 2019-12-20 | 2019-12-20 | Identity verification method, device, equipment and storage medium based on AR equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111160202A true CN111160202A (en) | 2020-05-15 |
| CN111160202B CN111160202B (en) | 2023-09-05 |
Family
ID=70558233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911341990.4A Active CN111160202B (en) | 2019-12-20 | 2019-12-20 | Identity verification method, device, equipment and storage medium based on AR equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111160202B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036262A (en) * | 2020-08-11 | 2020-12-04 | 海尔优家智能科技（北京）有限公司 | Face recognition processing method and device |
| CN112233343A (en) * | 2020-10-19 | 2021-01-15 | 中国工商银行股份有限公司 | Self-service terminal equipment service data processing method and device |
| CN112507798A (en) * | 2020-11-12 | 2021-03-16 | 上海优扬新媒信息技术有限公司 | Living body detection method, electronic device, and storage medium |
| CN112818833A (en) * | 2021-01-29 | 2021-05-18 | 中能国际建筑投资集团有限公司 | Face multitask detection method, system, device and medium based on deep learning |
| CN112861710A (en) * | 2021-02-05 | 2021-05-28 | 建信金融科技有限责任公司 | Management method and system for financial equipment, financial equipment and storage medium |
| CN113343862A (en) * | 2021-06-11 | 2021-09-03 | 上海中通吉网络技术有限公司 | Face acquisition method |
| CN113361456A (en) * | 2021-06-28 | 2021-09-07 | 北京影谱科技股份有限公司 | Face recognition method and system |
| CN115170993A (en) * | 2022-09-08 | 2022-10-11 | 浙江百诺数智环境科技股份有限公司 | AR acquisition and analysis-based on-site inspection method and system for waste gas treatment equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229260A (en) * | 2016-12-21 | 2018-06-29 | 杭州海康威视系统技术有限公司 | A kind of identity information checking method and system |
| CN109063593A (en) * | 2018-07-13 | 2018-12-21 | 北京智芯原动科技有限公司 | A kind of face tracking method and device |
| CN109359548A (en) * | 2018-09-19 | 2019-02-19 | 深圳市商汤科技有限公司 | Plurality of human faces identifies monitoring method and device, electronic equipment and storage medium |
| CN109558815A (en) * | 2018-11-16 | 2019-04-02 | 恒安嘉新（北京）科技股份公司 | A kind of detection of real time multi-human face and tracking |
| CN109829436A (en) * | 2019-02-02 | 2019-05-31 | 福州大学 | Multi-face tracking method based on depth appearance characteristics and self-adaptive aggregation network |
-
2019
- 2019-12-20 CN CN201911341990.4A patent/CN111160202B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229260A (en) * | 2016-12-21 | 2018-06-29 | 杭州海康威视系统技术有限公司 | A kind of identity information checking method and system |
| CN109063593A (en) * | 2018-07-13 | 2018-12-21 | 北京智芯原动科技有限公司 | A kind of face tracking method and device |
| CN109359548A (en) * | 2018-09-19 | 2019-02-19 | 深圳市商汤科技有限公司 | Plurality of human faces identifies monitoring method and device, electronic equipment and storage medium |
| CN109558815A (en) * | 2018-11-16 | 2019-04-02 | 恒安嘉新（北京）科技股份公司 | A kind of detection of real time multi-human face and tracking |
| CN109829436A (en) * | 2019-02-02 | 2019-05-31 | 福州大学 | Multi-face tracking method based on depth appearance characteristics and self-adaptive aggregation network |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036262A (en) * | 2020-08-11 | 2020-12-04 | 海尔优家智能科技（北京）有限公司 | Face recognition processing method and device |
| CN112233343A (en) * | 2020-10-19 | 2021-01-15 | 中国工商银行股份有限公司 | Self-service terminal equipment service data processing method and device |
| CN112507798A (en) * | 2020-11-12 | 2021-03-16 | 上海优扬新媒信息技术有限公司 | Living body detection method, electronic device, and storage medium |
| CN112507798B (en) * | 2020-11-12 | 2024-02-23 | 度小满科技(北京)有限公司 | Living body detection method, electronic device and storage medium |
| CN112818833A (en) * | 2021-01-29 | 2021-05-18 | 中能国际建筑投资集团有限公司 | Face multitask detection method, system, device and medium based on deep learning |
| CN112818833B (en) * | 2021-01-29 | 2024-04-12 | 中能国际建筑投资集团有限公司 | Face multitasking detection method, system, device and medium based on deep learning |
| CN112861710A (en) * | 2021-02-05 | 2021-05-28 | 建信金融科技有限责任公司 | Management method and system for financial equipment, financial equipment and storage medium |
| CN113343862A (en) * | 2021-06-11 | 2021-09-03 | 上海中通吉网络技术有限公司 | Face acquisition method |
| CN113361456A (en) * | 2021-06-28 | 2021-09-07 | 北京影谱科技股份有限公司 | Face recognition method and system |
| CN113361456B (en) * | 2021-06-28 | 2024-05-07 | 北京影谱科技股份有限公司 | Face recognition method and system |
| CN115170993A (en) * | 2022-09-08 | 2022-10-11 | 浙江百诺数智环境科技股份有限公司 | AR acquisition and analysis-based on-site inspection method and system for waste gas treatment equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111160202B (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111160202B (en) | Identity verification method, device, equipment and storage medium based on AR equipment | |
| CN106446873B (en) | Face detection method and device | |
| CN108875732B (en) | Model training and instance segmentation method, device and system and storage medium | |
| US8345921B1 (en) | Object detection with false positive filtering | |
| CN110232369B (en) | Face recognition method and electronic equipment | |
| CN105518709A (en) | Method, system and computer program product for identifying human face | |
| CN111797653A (en) | Image annotation method and device based on high-dimensional image | |
| WO2022041830A1 (en) | Pedestrian re-identification method and device | |
| TW202026948A (en) | Methods and devices for biological testing and storage medium thereof | |
| CN109116129B (en) | Terminal detection method, detection device, system and storage medium | |
| CN111462155B (en) | Motion detection method, device, computer equipment and storage medium | |
| CN112381075A (en) | Method and system for carrying out face recognition under specific scene of machine room | |
| CN112364843A (en) | Plug-in aerial image target positioning detection method, system and equipment | |
| CN112052730B (en) | 3D dynamic portrait identification monitoring equipment and method | |
| CN113158773B (en) | Training method and training device for living body detection model | |
| CN114387548A (en) | Video and liveness detection method, system, device, storage medium and program product | |
| CN112183431A (en) | Real-time pedestrian number statistical method and device, camera and server | |
| CN112926491A (en) | User identification method and device, electronic equipment and storage medium | |
| CN112766065A (en) | Mobile terminal examinee identity authentication method, device, terminal and storage medium | |
| CN109389367B (en) | Personnel attendance checking method, device and storage medium | |
| CN108875472B (en) | Image acquisition device and face identity verification method based on image acquisition device | |
| CN113642428B (en) | Face living body detection method and device, electronic equipment and storage medium | |
| CN115601712A (en) | Image data processing method and system suitable for field safety measures | |
| CN112819859B (en) | Multi-target tracking method and device applied to intelligent security | |
| CN114173158A (en) | Face recognition method, cloud device, client device, electronic device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |