CN114842543A - Three-dimensional face recognition method and device, electronic equipment and storage medium - Google Patents
Three-dimensional face recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114842543A CN114842543A CN202210615124.5A CN202210615124A CN114842543A CN 114842543 A CN114842543 A CN 114842543A CN 202210615124 A CN202210615124 A CN 202210615124A CN 114842543 A CN114842543 A CN 114842543A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- cloud data
- layer
- face
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims description 17
- 238000000605 extraction Methods 0.000 claims abstract description 31
- 238000005259 measurement Methods 0.000 claims abstract description 20
- 238000001514 detection method Methods 0.000 claims description 27
- 230000009466 transformation Effects 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 17
- 238000011176 pooling Methods 0.000 claims description 15
- 238000009826 distribution Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000000379 polymerizing effect Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 6
- 238000009499 grossing Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 210000000887 face Anatomy 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 210000000720 eyelash Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 210000003786 sclera Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention relates to a three-dimensional face recognition method, which comprises the following steps: acquiring three-dimensional face point cloud data to be recognized; inputting the three-dimensional face point cloud data to be recognized into an RP-net network model to obtain local-global face characteristics; and carrying out similarity measurement on the local-global face features and the face features of a candidate face to obtain a similarity measurement value, and judging whether the three-dimensional face point cloud data to be identified is the candidate face according to the similarity measurement value. Compared with the prior art, the invention provides the three-dimensional face recognition method which carries out feature extraction on the three-dimensional face point cloud data through the RP-net network model, and simultaneously captures the local features and the global features of the face in the feature extraction of the RP-net network model, wherein the local features can describe the face features in more detail, and the recognition accuracy of the three-dimensional face can be improved.
Description
Technical Field
The present invention relates to the field of three-dimensional face recognition technologies, and in particular, to a three-dimensional face recognition method, an apparatus, an electronic device, and a storage medium.
Background
With the continuous improvement of the information degree of the current society, the related information security problem is more and more emphasized, and all the information security can not leave the authentication of the personal identity finally. Whether personal privacy information, property security, or government confidential documents and administrative authorities, the identity of the relevant personnel needs to be authenticated to ensure security. The traditional identity authentication modes such as certificates, passwords, seals, cards and the like have the defects and hidden dangers, authentication tools such as certificates, cards and the like are easy to damage or lose, passwords are easy to be confused and forgotten and the like, and due to the advantages of reliability and convenience, the emerging biometric identification technology has the advantages which are incomparable with the traditional identity identification and authentication technology, and is widely concerned and used by society.
The three-dimensional face recognition technology has unique advantages, and is mainly embodied in that:
(1) the three-dimensional face recognition has the characteristics of constant illumination and posture, the three-dimensional shape data of the face can be regarded as not changing along with the change of light rays and sight lines, accessories such as makeup have great influence on the two-dimensional image data, but have no obvious influence on the three-dimensional data.
(2) Three-dimensional data has a definite spatial shape representation, so its information is richer than two-dimensional images.
In the existing three-dimensional face recognition method, the position information and the depth information of a three-dimensional image are represented by three-dimensional face point cloud data, and the three-dimensional face point cloud data is classified and recognized through a deep learning model, but the existing deep learning model has insufficient details for feature extraction of the three-dimensional face point cloud data, and the recognition rate of the three-dimensional face recognition is low.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a three-dimensional face recognition method which can simultaneously extract the local features and the global features of three-dimensional face point cloud data and improve the recognition rate of three-dimensional face recognition.
The invention is realized by the following technical scheme: a three-dimensional face recognition method comprises the following steps:
acquiring three-dimensional face point cloud data to be recognized;
inputting the three-dimensional face point cloud data to be recognized into an RP-net network model to obtain local-global face features, wherein the RP-net network model comprises a sampling module, a grouping module and a feature extraction module, and the sampling module is used for sampling the three-dimensional face point cloud data to be recognized to obtain a plurality of key points;
the grouping module is used for taking each key point as a circle center and taking the characteristic extraction radius value r as a radius to obtain a plurality of circular areas; grouping the three-dimensional face point cloud data to be recognized according to the circular area to obtain a plurality of key point cloud data sets;
the feature extraction module comprises a first transformation layer, a local feature marking layer, a first MLP layer, a second transformation layer, a second MLP layer and a pooling layer, wherein the first transformation layer is used for carrying out alignment operation on each key point cloud data set; the local feature marking layer is used for rotating the key point cloud data set output by the first conversion layer around a three-dimensional coordinate axis by T rotation angles { theta [ theta ]) respectively k 1,2, T, obtaining a rotating point cloud data set Q 'corresponding to three-dimensional coordinate axes' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) (ii) a Respectively collecting the rotation point cloud data set Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Projecting on three planes where three-dimensional coordinate axes are located to obtain corresponding projection point cloud data sets; separately capturing the projection point cloud data setsIs divided into N on average b ×N b Each grid is calculated, and the point cloud data quantity in each grid is calculated to obtain a corresponding distribution matrix D x ，D y ，D z (ii) a According to the distribution matrix D respectively x ，D y ，D z Obtaining the corresponding central moment mu mn And shannon entropy e; respectively according to the central moment mu mn Obtaining the sub-feature descriptors f corresponding to three-dimensional coordinate axes according to the Shannon entropy e x (θ k )，f y (θ k )，f z (θ k ) (ii) a Sub-feature descriptor f x (θ k )，f y (θ k )，f z (θ k ) Polymerizing to obtain a local feature descriptor f; the first MLP layer is used for performing dimension raising operation on the key point cloud data set output by the first transformation layer according to the local feature descriptor f; the second transformation layer is used for carrying out alignment operation on the key point cloud data set output by the first MLP layer; the second MLP layer is used for performing dimension increasing operation on the key point cloud data set output by the second conversion layer; the pooling layer is used for performing maximum pooling operation on the key point cloud data set output by the second MLP layer to obtain the local-global face features;
and carrying out similarity measurement on the local-global face features and the face features of a candidate face to obtain a similarity measurement value, and judging whether the three-dimensional face point cloud data to be identified is the candidate face according to the similarity measurement value.
Compared with the prior art, the invention provides the three-dimensional face recognition method which carries out feature extraction on the three-dimensional face point cloud data through the RP-net network model, and simultaneously captures the local features and the global features of the face in the feature extraction of the RP-net network model, wherein the local features can describe the face features in more detail, and the recognition accuracy of the three-dimensional face can be improved.
Further, after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the following steps:
horizontally slicing the three-dimensional face point cloud data to be recognized to obtain a plurality of horizontal contour maps;
aiming at each horizontal contour map, placing a plurality of detection points on a horizontal contour edge line, setting a detection circle by taking each detection point as a circle center, obtaining detection distances from two intersection points of the detection circle and the horizontal contour edge line to corresponding detection points, and determining the detection point corresponding to the largest detection distance as a nose tip candidate point;
determining the nose tip candidate point with the maximum detection distance as a nose tip point;
and calculating the distance between each data point in the three-dimensional face point cloud data to be recognized and the nose tip point, and removing the data points with the distance larger than a preset distance.
Further, after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the following steps: and acquiring a median coordinate of the data points in the same field aiming at each data point in the three-dimensional face point cloud data to be recognized, and replacing the coordinate of the data point with the median coordinate.
Further, for each data point in the three-dimensional face point cloud data to be recognized, obtaining a median coordinate of the data point in the same field, and replacing the coordinate of the data point with the median coordinate, the method further comprises the following steps: and filling the missing holes in the three-dimensional human face point cloud data to be recognized through cubic interpolation.
Further, after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the following steps: and acquiring a normal vector of each data point in the three-dimensional face point cloud data to be recognized, and adding the normal vector into the three-dimensional face point cloud data to be recognized.
Further, the central moment μ mn Is expressed as
Wherein the content of the first and second substances,
d (i, j) denotes the ith row and jth column of the distribution matrix D, D ═ D x ,D y ,D z ]，μ mn ＝[μ 11 ,μ 12 ,μ 21 ,μ 22 ]。
Further, the similarity metric is a nearest neighbor distance ratio.
Based on the same inventive concept, the invention also provides a three-dimensional face recognition device, comprising:
the data acquisition module is used for acquiring three-dimensional face point cloud data to be recognized;
the system comprises a feature acquisition module, a feature extraction module and a feature extraction module, wherein the feature acquisition module is used for inputting the three-dimensional face point cloud data to be recognized into an RP-net network model to obtain local-global face features, the RP-net network model comprises a sampling module, a grouping module and a feature extraction module, and the sampling module is used for sampling the three-dimensional face point cloud data to be recognized to obtain a plurality of key points;
the grouping module is used for taking each key point as a circle center and taking the characteristic extraction radius value r as a radius to obtain a plurality of circular areas; grouping the three-dimensional face point cloud data to be recognized according to the circular area to obtain a plurality of key point cloud data sets;
the feature extraction module comprises a first transformation layer, a local feature marking layer, a first MLP layer, a second transformation layer, a second MLP layer and a pooling layer, wherein the first transformation layer is used for carrying out alignment operation on each key point cloud data set; the local feature marking layer is used for rotating the key point cloud data set output by the first conversion layer around a three-dimensional coordinate axis by T rotation angles { theta } k 1,2, T, obtaining a rotating point cloud data set Q 'corresponding to three-dimensional coordinate axes' x (θ k )，Q′ y (θ k )，Q z ′(θ k ) (ii) a Respectively collecting the rotation point cloud data set Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Projecting on three planes where three-dimensional coordinate axes are located to obtain corresponding projection point cloud data sets; respectively averagely dividing the projection point cloud data set into N b ×N b Each grid is obtained, and the point cloud data volume in each grid is calculated to obtain pairsCorresponding distribution matrix D x ，D y ，D z (ii) a According to the distribution matrix D respectively x ，D y ，D z Obtaining the corresponding central moment mu mn And shannon entropy e; respectively according to the central moment mu mn Obtaining the sub-feature descriptors f corresponding to three-dimensional coordinate axes according to the Shannon entropy e x (θ k )，f y (θ k )，f z (θ k ) (ii) a Sub-feature descriptor f x (θ k )，f y (θ k )，f z (θ k ) Polymerizing to obtain a local feature descriptor f; the first MLP layer is used for performing dimension raising operation on the key point cloud data set output by the first transformation layer according to the local feature descriptor f; the second transformation layer is used for carrying out alignment operation on the key point cloud data set output by the first MLP layer; the second MLP layer is used for performing dimension increasing operation on the key point cloud data set output by the second conversion layer; the pooling layer is used for performing maximum pooling operation on the key point cloud data set output by the second MLP layer to obtain the local-global face features;
and the matching module is used for carrying out similarity measurement on the local-global face features and the face features of a candidate face to obtain a similarity measurement value, and judging whether the three-dimensional face point cloud data to be identified is the candidate face according to the similarity measurement value.
Based on the same inventive concept, the present invention also provides an electronic device, comprising:
a processor;
a memory for storing a computer program for execution by the processor;
wherein the processor implements the steps of the above method when executing the computer program.
Based on the same inventive concept, the present invention also provides a computer-readable storage medium, characterized in that a computer program is stored thereon, which, when executed, implements the steps of the above-described method.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
Fig. 1 is a schematic flow chart of a three-dimensional face recognition method according to an embodiment;
FIG. 2 is a schematic diagram of a network structure of an embodiment of an RP-net network model;
fig. 3 is a schematic structural diagram of the three-dimensional face recognition apparatus according to the embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims.
In the description of the present application, it is to be understood that the terms "first," "second," "third," and the like are used solely for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order, nor should be construed to indicate or imply relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate. Further, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
Please refer to fig. 1, which is a schematic flow chart of the three-dimensional face recognition method of the present embodiment, the method includes steps S1-S3:
and step S1, acquiring the three-dimensional face point cloud data to be recognized, and preprocessing the three-dimensional face point cloud data.
The three-dimensional face point cloud data is a representation mode of three-dimensional face information, the three-dimensional face point cloud data comprises a plurality of data points, each data point corresponds to a three-dimensional coordinate in a three-dimensional coordinate system, and the data points with the three-dimensional coordinates jointly form a three-dimensional face. The three-dimensional axes in the three-dimensional coordinate system are respectively an x axis, a y axis and a z axis, the plane where the x axis and the y axis are located is an xy plane, the plane where the x axis and the z axis are located is an xz plane, and the plane where the y axis and the z axis are located is a yz plane.
The three-dimensional face point cloud data can be acquired by scanning the face through special scanning equipment, and can be stored in any computer storage medium such as a memory.
The method specifically comprises the steps of preprocessing a three-dimensional face point cloud image, wherein the face clipping operation is used for clipping redundant data points when the three-dimensional face point cloud data to be recognized contains the redundant data points except for a face, such as a body below a shoulder, and the like, so as to obtain three-dimensional face point cloud data mainly comprising the face, and the method specifically comprises the following steps of S111-S114:
s111: horizontally slicing the point cloud data of the three-dimensional human faces to be recognized to obtain a plurality of horizontal contour maps of the three-dimensional human faces;
in a preferred embodiment, step S111 may also perform uniform interpolation on the obtained horizontal contour map to fill the holes in the horizontal contour map.
S112: in each horizontal contour map, a plurality of detection points are placed on a horizontal contour edge line at a certain density, a detection circle with a fixed radius is arranged by taking each detection point as a circle center, the detection distance h from two intersection points of the detection circle and the horizontal contour edge line to the corresponding detection point is obtained, and the detection point corresponding to the maximum detection distance h is determined as a nasal tip candidate point;
s113: and carrying out thinning screening on the nasal tip candidate points to determine the nasal tip points.
In an alternative implementation, the nose tip candidate point can be regarded as a set of data points on the bridge of the nose, and the nose tip candidate point corresponding to the maximum detection distance h is determined as the nose tip point.
In an alternative implementation, the nose tip candidate points can be subjected to a refined screening by a random sample consensus (RANSAC) algorithm.
S114: calculating the distance between each point and the nose tip point in the three-dimensional human face point cloud data to be recognized, cutting out the points of which the distance is greater than the preset distance, and cutting out the three-dimensional human face point cloud data mainly comprising human faces.
In one implementation, the predetermined distance in step S114 may be set to 90 mm.
The face smoothing operation is used for eliminating noise spikes in the three-dimensional face point cloud data to be recognized, especially the noise spikes are easy to appear in the area of eyes, nose spikes, teeth and the like, and the subsequent recognition accuracy can be improved by eliminating the noise spikes. The method specifically comprises the following steps of S121: acquiring the median coordinate of each data point of the three-dimensional face point cloud data to be recognized in one field, and replacing the data point coordinate with the median coordinate.
Since a hole defect may also occur on the three-dimensional surface of the three-dimensional face point cloud data in the process of eliminating the noise spike, and a hole defect caused by specular reflection of a dark region of the face, an open mouth, a blocking object, a sclera, a pupil, eyelashes and the like when the three-dimensional face point cloud data is collected, in a preferred embodiment, the facial smoothing operation may further include step S122: filling of the hole is performed by cubic interpolation. Thereby, the holes can be eliminated to improve the identification accuracy.
And the data enhancement operation is used for acquiring the normal vector of each data point in the three-dimensional face point cloud data to be recognized and increasing the normal vector of each data point in the three-dimensional face point cloud data. The normal vector of the data point can display more local features, and extraction and identification of the face features are facilitated. Specifically, the normal vector of each data point is the plane of the data pointNormal vectors, which can be obtained by fitting local neighbors of each data point by minimizing a cost function. A three-dimensional face point cloud data containing m data points is expressed as P ═ P 1 ,p 2 ,...,p s ] T Wherein the three-dimensional coordinate of the ith data point is represented as p i ＝[p ix ,p iy ,p iz ] T The normal vector of the ith data point is represented as n i ＝[n ix ,n iy ,n iz ] T Wherein n is ix ,n iy ,n iz Is a data point p i Normal component in the x, y, z channels of the three-dimensional coordinate system, data point p i The surrounding set of l neighboring points is denoted Q i ＝[p i1 ,p i2 ,...,p il ] T The data point p is minimized by minimizing a cost function minA i Normal vector and neighbor set Q i Form a plane vector point multiplied by 0, i.e. data point p i Normal vector and neighbor set Q i The formed plane is vertical, and the expression of the minimum cost function minA is as follows:
in specific implementation, a matrix with the field of 5 × 5 can be selected for data enhancement operation, and then m × n × 6 three-dimensional face point cloud data is obtained, wherein m and n are the data range size of the three-dimensional face point cloud data.
And step S2, inputting the three-dimensional human face point cloud data preprocessed in the step S1 into an RP-net network model to obtain local-global human face characteristics.
Please refer to fig. 2, which is a schematic network structure diagram of an RP-net network model of this embodiment, where the RP-net network model includes a sampling module, a grouping module and a feature extraction module, where the sampling module is configured to perform downsampling on three-dimensional face point cloud data, and the method specifically includes the following steps: sampling is performed through Farthest Point Sampling (FPS) to obtain a plurality of key points, where the initial key points may be random data points or nasal tip points obtained in step S113.
The grouping module is used for grouping the three-dimensional face point cloud data according to the key points acquired by the sampling module, and specifically, each key point is taken as a circle center, and the characteristic extraction radius value r is taken as a radius to obtain a plurality of circular areas; dividing the three-dimensional face point cloud data into a plurality of key point cloud data sets according to a circular area, and successively inputting each key point cloud data set into a feature extraction layer. When the RP-net network model is trained, the characteristic extraction radius value r in the grouping module can be set to be a plurality of different step values, and the final characteristic extraction radius value r is determined according to the recognition rate of the model.
The feature extraction module is used for extracting features of each key point cloud data set output by the grouping module. Specifically, the feature extraction layer comprises a first transformation layer, a local feature marker layer, a first MLP layer, a second transformation layer, a second MLP layer and a pooling layer, wherein the first transformation layer is used for performing alignment operation on each key point cloud data set through T-net and matrix multiplication (matrix multiplication).
The local feature marking layer is used for acquiring a local feature descriptor of the key point cloud data set output by the first conversion layer and marking the local feature of the key point cloud data set through the local feature descriptor. The method specifically comprises the following steps: rotating the key point cloud data set by T rotation angles { theta ] around the x-axis, the y-axis and the z-axis of the three-dimensional coordinate system respectively k 1, 2., T, resulting in a corresponding three sets of rotated point cloud datasets Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) (ii) a Each rotation point cloud data set Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Respectively projecting on xy, xz and yz planes to obtain projection point cloud data sets corresponding to the three planes
Projecting a point cloud data set
Are respectively averagely divided into N b ×N b Personal netGrid, and calculating the point cloud data quantity in each grid to obtain three corresponding groups of rotating point cloud data sets Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Has a size of N b ×N b Distribution matrix D of x ，D y ，D z (ii) a To distribution matrix D x ，D y ，D z Carrying out normalization operation to realize invariance to grid resolution change; to distribution matrix D x ，D y ，D z Performing compression operation to obtain central moment mu mn And shannon entropy e, where central moment μ mn The expression of (a) is:
wherein the content of the first and second substances,
d (i, j) denotes the ith row and jth column of the distribution matrix D, D ═ D x ,D y ,D z ]. In this embodiment, μ mn ＝[μ 11 ,μ 12 ,μ 21 ,μ 22 ]。
The expression of the Shannon entropy e is
According to the central moment mu mn And the Shannon entropy e to obtain a sub-feature descriptor f rotating around the x-axis x (θ k ) Sub-feature descriptor f rotating around the y-axis y (θ k ) And a sub-feature descriptor f rotated about the z-axis z (θ k )；
Sub-feature descriptor f x (θ k ) Sub-feature descriptor f y (θ k ) And a sub-feature descriptor f z (θ k ) Carrying out aggregation to obtain a local feature descriptor f, wherein the expression of the local feature descriptor f is
f＝{f x (θ k ),f y (θ k ),f z (θ k )}
The first MLP layer is a multilayer perceptron and is used for performing dimension-increasing operation on each key point cloud data set according to the local feature descriptor f.
And the second conversion layer is used for carrying out alignment operation on the key point cloud data set output by the first MLP layer through T-net and matrix multiplication.
The second MLP layer is a multi-layer perceptron and is configured to perform a second dimension-increasing operation on the key point cloud data set output by the second transform layer, and in a specific implementation, a characteristic dimension after the second dimension-increasing operation is 1024.
And the pooling layer is used for performing maximum pooling operation on the key point cloud data set output by the second MLP layer to obtain local-global face features.
And step S3, performing similarity measurement on the local-global face features obtained in the step S2 and the face features of a candidate face, and judging whether the three-dimensional face point cloud data to be recognized is the candidate face according to a similarity measurement result.
In one embodiment, a Nearest Neighbor Distance Ratio (NNDR) may be used to perform similarity measurement between local-global face features and face features of candidate faces, the obtained nearest neighbor distance ratio is compared with a preset comparison threshold, and when the nearest neighbor distance ratio is greater than the preset threshold, a face corresponding to the three-dimensional face point cloud data to be recognized may be determined as a corresponding candidate face. In a specific implementation, the preset comparison threshold may be set to a plurality of different step values, and the final preset comparison threshold is determined according to the recognition accuracy of the result.
Experiments are respectively carried out on the disclosed three-dimensional face data set Bosphorus data set through the three-dimensional face recognition method, the three-dimensional face recognition method based on voxel representation of the depth image and the large-range 3D face recognition method, wherein the experiment environment comprises a normal environment and a low-light environment, and the recognition rate in the corresponding experiment environment is obtained. Among them, the three-dimensional face recognition method (3d face recognition based on volumetric rendering of range image) based on voxel representation of depth image is the method disclosed in Koushik Dutta, Debotosh Bhattacharjee, Mita nasiprari, and Anik Poddar in 2019 on Advanced Computing and Systems for Security; the wide-area 3D face recognition method (Towards large-scale 3D face recognition) is the method disclosed by Syed Zulqarinin Gilani and Ajmal Mian in 2016International Conference on Digital Image Computing, Techniques and Applications (DICTA). As shown in table 1, compared with two three-dimensional face recognition methods in the prior art, the three-dimensional face recognition method of the present embodiment improves the recognition rate in both the normal environment and the low-light environment.
TABLE 1
Compared with the prior art, the method has the advantages that the feature extraction is carried out on the three-dimensional face point cloud data through the RP-net network model, the local features and the global features of the face are captured simultaneously in the feature extraction of the RP-net network model, the local features can describe the face features in more detail, and the accuracy rate of the three-dimensional face recognition can be improved. In addition, the surface normal vector is added into the three-dimensional face point cloud data input into the RP-net network model, so that the three-dimensional face characteristics under the weak light or dark environment can be enhanced, and the high identification accuracy can be realized in the weak light and dark environment.
Based on the same invention concept, the invention also provides a three-dimensional face recognition device. Please refer to fig. 3, which is a schematic structural diagram of the three-dimensional face recognition apparatus of this embodiment, the apparatus includes a data obtaining module 10, a feature obtaining module 20, and a matching module 30, wherein the data obtaining module 10 is configured to obtain three-dimensional face point cloud data to be recognized, and pre-process the three-dimensional face point cloud data, where the pre-process includes a face cropping operation, a face smoothing operation, and a data enhancement operation, and a specific flow of steps of the face cropping operation, the face smoothing operation, and the data enhancement operation is the same as that of the face cropping operation, the face smoothing operation, and the data enhancement operation described in the foregoing method embodiment, and is not described herein again;
the feature obtaining module 20 is configured to input the three-dimensional face point cloud data preprocessed in the data obtaining module 10 into an RP-net network model to obtain local-global face features, where a network structure and a process flow of steps included in the RP-net network model are completely the same as those of the RP-net network model described in the above method embodiment, and are not described herein again.
The matching module 30 is configured to perform similarity measurement on the local-global face features obtained in the feature obtaining module 20 and face features of a candidate face, and determine whether the three-dimensional face point cloud data to be recognized is the candidate face according to a similarity measurement result.
For the device embodiments, reference is made to the description of the method embodiments for relevant details, since they correspond essentially to the method embodiments.
Based on the same inventive concept, the invention also provides an electronic device, which can be a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.), and other terminal devices. The electronic equipment comprises one or more processors and a memory, wherein the processors are used for executing the three-dimensional face recognition method of the program implementation method embodiment; the memory is for storing a computer program executable by the processor. The electronic equipment can also comprise a display screen which is used for displaying the retrieval result image obtained by the processor.
Based on the same inventive concept, the present invention further provides a computer-readable storage medium, corresponding to the aforementioned embodiments of the three-dimensional face recognition method, wherein the computer-readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the three-dimensional face recognition method described in any of the above embodiments.
This application may take the form of a computer program product embodied on one or more storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of the storage medium of the computer include, but are not limited to: phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by a computing device.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, to those skilled in the art, changes and modifications may be made without departing from the spirit of the present invention, and it is intended that the present invention encompass such changes and modifications.
Claims (10)
1. A three-dimensional face recognition method is characterized by comprising the following steps:
acquiring three-dimensional face point cloud data to be recognized;
inputting the three-dimensional face point cloud data to be recognized into an RP-net network model to obtain local-global face features, wherein the RP-net network model comprises a sampling module, a grouping module and a feature extraction module, and the sampling module is used for sampling the three-dimensional face point cloud data to be recognized to obtain a plurality of key points;
the grouping module is used for taking each key point as a circle center and taking the characteristic extraction radius value r as a radius to obtain a plurality of circular areas; grouping the three-dimensional face point cloud data to be recognized according to the circular area to obtain a plurality of key point cloud data sets;
the feature extraction module comprises a first transformation layer, a local feature marking layer, a first MLP layer, a second transformation layer, a second MLP layer and a pooling layer, wherein the first transformation layer is used for carrying out alignment operation on each key point cloud data set; the local feature marking layer is used for rotating the key point cloud data set output by the first conversion layer around a three-dimensional coordinate axis by T rotation angles { theta } k 1,2, T, obtaining a rotating point cloud data set Q 'corresponding to three-dimensional coordinate axes' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) (ii) a Respectively collecting the rotation point cloud data set Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Projecting on three planes where three-dimensional coordinate axes are located to obtain corresponding projection point cloud data sets; respectively averagely dividing the projection point cloud data set into N b ×N b Each grid is calculated, and the point cloud data quantity in each grid is calculated to obtain a corresponding distribution matrix D x ，D y ，D z (ii) a According to the distribution matrix D respectively x ，D y ，D z Obtaining the corresponding central moment mu mn And shannon entropy e; respectively according to the central moment mu mn Obtaining the sub-feature descriptors f corresponding to three-dimensional coordinate axes according to the Shannon entropy e x (θ k )，f y (θ k )，f z (θ k ) (ii) a Sub-feature descriptor f x (θ k )，f y (θ k )，f z (θ k ) Polymerizing to obtain a local feature descriptor f; the first MLP layer is used for performing dimension raising operation on the key point cloud data set output by the first transformation layer according to the local feature descriptor f; the second transformation layer is used for carrying out alignment operation on the key point cloud data set output by the first MLP layer; the second MLP layer is used for performing dimension increasing operation on the key point cloud data set output by the second conversion layer; the pooling layer is used for performing maximum pooling operation on the key point cloud data set output by the second MLP layer to obtain the local-global personA face feature;
and carrying out similarity measurement on the local-global face features and the face features of a candidate face to obtain a similarity measurement value, and judging whether the three-dimensional face point cloud data to be identified is the candidate face according to the similarity measurement value.
2. The method of claim 1, wherein after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the steps of:
horizontally slicing the three-dimensional face point cloud data to be recognized to obtain a plurality of horizontal contour maps;
aiming at each horizontal contour map, placing a plurality of detection points on a horizontal contour edge line, setting a detection circle by taking each detection point as a circle center, obtaining detection distances from two intersection points of the detection circle and the horizontal contour edge line to corresponding detection points, and determining the detection point corresponding to the largest detection distance as a nose tip candidate point;
determining the nose tip candidate point with the maximum detection distance as a nose tip point;
and calculating the distance between each data point in the three-dimensional human face point cloud data to be recognized and the nose tip point, and removing the data points with the distance larger than a preset distance.
3. The method of claim 1, wherein after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the steps of: and acquiring a median coordinate of the data points in the same field aiming at each data point in the three-dimensional face point cloud data to be recognized, and replacing the coordinate of the data point with the median coordinate.
4. The method of claim 3, wherein: and aiming at each data point in the three-dimensional face point cloud data to be recognized, acquiring a median coordinate of the data point in the same field, and replacing the coordinate of the data point with the median coordinate, and further comprising the following steps: and filling the missing holes in the three-dimensional human face point cloud data to be recognized through cubic interpolation.
5. The method of claim 1, wherein after acquiring the three-dimensional face point cloud data to be recognized, the method further comprises the steps of: and acquiring a normal vector of each data point in the three-dimensional face point cloud data to be recognized, and adding the normal vector into the three-dimensional face point cloud data to be recognized.
6. Method according to claim 1, characterized in that the central moment μ mn Is expressed as
Wherein the content of the first and second substances,
d (i, j) denotes the ith row and jth column of the distribution matrix D, D ═ D x ,D y ,D z ]，μ mn ＝[μ 11 ,μ 12 ,μ 21 ,μ 22 ]。
7. The method of claim 1, wherein the similarity metric is a nearest neighbor distance ratio.
8. A three-dimensional face recognition apparatus, comprising:
the data acquisition module is used for acquiring three-dimensional face point cloud data to be recognized;
the system comprises a feature acquisition module, a feature extraction module and a feature extraction module, wherein the feature acquisition module is used for inputting the three-dimensional face point cloud data to be recognized into an RP-net network model to obtain local-global face features, the RP-net network model comprises a sampling module, a grouping module and a feature extraction module, and the sampling module is used for sampling the three-dimensional face point cloud data to be recognized to obtain a plurality of key points;
the grouping module is used for taking each key point as a circle center and taking the characteristic extraction radius value r as a radius to obtain a plurality of circular areas; grouping the three-dimensional face point cloud data to be recognized according to the circular area to obtain a plurality of key point cloud data sets;
the feature extraction module comprises a first transformation layer, a local feature marking layer, a first MLP layer, a second transformation layer, a second MLP layer and a pooling layer, wherein the first transformation layer is used for carrying out alignment operation on each key point cloud data set; the local feature marking layer is used for rotating the key point cloud data set output by the first conversion layer around a three-dimensional coordinate axis by T rotation angles { theta } k 1,2, T, obtaining a rotating point cloud data set Q 'corresponding to three-dimensional coordinate axes' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) (ii) a Respectively collecting the rotation point cloud data set Q' x (θ k )，Q′ y (θ k )，Q′ z (θ k ) Projecting on three planes where three-dimensional coordinate axes are located to obtain corresponding projection point cloud data sets; respectively averagely dividing the projection point cloud data set into N b ×N b Each grid is calculated, and the point cloud data quantity in each grid is calculated to obtain a corresponding distribution matrix D x ，D y ，D z (ii) a According to the distribution matrix D respectively x ，D y ，D z Obtaining the corresponding central moment mu mn And shannon entropy e; respectively according to the central moment mu mn Obtaining the sub-feature descriptors f corresponding to three-dimensional coordinate axes according to the Shannon entropy e x (θ k )，f y (θ k )，f z (θ k ) (ii) a Sub-feature descriptor f x (θ k )，f y (θ k )，f z (θ k ) Polymerizing to obtain a local feature descriptor f; the first MLP layer is used for performing dimension raising operation on the key point cloud data set output by the first transformation layer according to the local feature descriptor f; the second transformation layer is used for carrying out alignment operation on the key point cloud data set output by the first MLP layer; the second MLP layer is used for performing dimension increasing operation on the key point cloud data set output by the second conversion layer; the pooling layer is used for the placePerforming maximum pooling operation on the key point cloud data set output by the second MLP layer to obtain the local-global face features;
and the matching module is used for carrying out similarity measurement on the local-global face features and the face features of a candidate face to obtain a similarity measurement value, and judging whether the three-dimensional face point cloud data to be identified is the candidate face according to the similarity measurement value.
9. An electronic device, comprising:
a processor;
a memory for storing a computer program for execution by the processor;
wherein the processor, when executing the computer program, implements the steps of the method of any one of claims 1-7.
10. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed, performs the steps of the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210615124.5A CN114842543A (en) | 2022-06-01 | 2022-06-01 | Three-dimensional face recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210615124.5A CN114842543A (en) | 2022-06-01 | 2022-06-01 | Three-dimensional face recognition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114842543A true CN114842543A (en) | 2022-08-02 |
Family
ID=82572657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210615124.5A Pending CN114842543A (en) | 2022-06-01 | 2022-06-01 | Three-dimensional face recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114842543A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243374A (en) * | 2015-11-02 | 2016-01-13 | 湖南拓视觉信息技术有限公司 | Three-dimensional human face recognition method and system, and data processing device applying same |
| CN110147721A (en) * | 2019-04-11 | 2019-08-20 | 阿里巴巴集团控股有限公司 | A kind of three-dimensional face identification method, model training method and device |
| WO2020199693A1 (en) * | 2019-03-29 | 2020-10-08 | 中国科学院深圳先进技术研究院 | Large-pose face recognition method and apparatus, and device |
| WO2021051539A1 (en) * | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Face recognition method and apparatus, and terminal device |
-
2022
- 2022-06-01 CN CN202210615124.5A patent/CN114842543A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243374A (en) * | 2015-11-02 | 2016-01-13 | 湖南拓视觉信息技术有限公司 | Three-dimensional human face recognition method and system, and data processing device applying same |
| WO2020199693A1 (en) * | 2019-03-29 | 2020-10-08 | 中国科学院深圳先进技术研究院 | Large-pose face recognition method and apparatus, and device |
| CN110147721A (en) * | 2019-04-11 | 2019-08-20 | 阿里巴巴集团控股有限公司 | A kind of three-dimensional face identification method, model training method and device |
| WO2021051539A1 (en) * | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Face recognition method and apparatus, and terminal device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Pflug et al. | Ear biometrics: a survey of detection, feature extraction and recognition methods | |
| JP6681729B2 (en) | Method for determining 3D pose of object and 3D location of landmark point of object, and system for determining 3D pose of object and 3D location of landmark of object | |
| CN108182397B (en) | Multi-pose multi-scale human face verification method | |
| CN110728196B (en) | Face recognition method and device and terminal equipment | |
| JP2013012190A (en) | Method of approximating gabor filter as block-gabor filter, and memory to store data structure for access by application program running on processor | |
| CN111091075B (en) | Face recognition method and device, electronic equipment and storage medium | |
| Islam et al. | A review of recent advances in 3D ear-and expression-invariant face biometrics | |
| CN102509138A (en) | Authentication system based on second-generation ID card and human face feature recognition and working method therefor | |
| CN108960088A (en) | The detection of facial living body characteristics, the recognition methods of specific environment | |
| US20200218885A1 (en) | Detecting artificial facial images using facial landmarks | |
| Soltanpour et al. | Multimodal 2D–3D face recognition using local descriptors: pyramidal shape map and structural context | |
| CN112686191B (en) | Living body anti-counterfeiting method, system, terminal and medium based on three-dimensional information of human face | |
| Li et al. | An efficient 3D face recognition method using geometric features | |
| CN106778491A (en) | The acquisition methods and equipment of face 3D characteristic informations | |
| CN110866442B (en) | Real-time face recognition-based testimony-of-person integrated checking system and method | |
| Kordelas et al. | Viewpoint independent object recognition in cluttered scenes exploiting ray-triangle intersection and SIFT algorithms | |
| CN114842543A (en) | Three-dimensional face recognition method and device, electronic equipment and storage medium | |
| Zhang et al. | Face recognition using SIFT features under 3D meshes | |
| Hasan et al. | 3-d face recognition using improved 3d mixed transform | |
| Gao et al. | Expression robust 3D face recognition by matching multi-component local shape descriptors on the nasal and adjoining cheek regions | |
| CN112395919B (en) | Off-line signature authentication method and system based on multi-feature metric learning | |
| Liu et al. | Recognizing occluded 3D faces using an efficient ICP variant | |
| Purahong et al. | Hybrid Facial Features with Application in Person Identification. | |
| Pears | RBF shape histograms and their application to 3D face processing | |
| Gavrilova | Computational geometry and image processing in biometrics: On the path to convergence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |