CN111553195B - Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP - Google Patents
Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP Download PDFInfo
- Publication number
- CN111553195B CN111553195B CN202010255272.1A CN202010255272A CN111553195B CN 111553195 B CN111553195 B CN 111553195B CN 202010255272 A CN202010255272 A CN 202010255272A CN 111553195 B CN111553195 B CN 111553195B
- Authority
- CN
- China
- Prior art keywords
- coordinate
- pixel point
- bitmap
- scale
- ulbp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a three-dimensional face shielding distinguishing method based on a multi-bitmap tangent plane and multi-scale uLBP (equivalent pattern LBP, uniform LBP), which comprises the following steps: firstly, inputting three-dimensional face point cloud and obtaining a depth map of the three-dimensional face point cloud; secondly, calculating the bitmap gray value of the depth map and obtaining 8 bitmap cutting planes; extracting 6, 7 and 8 bitmap planes and synthesizing to obtain a multi-bitmap cut plane; performing 3*4 blocking on the multi-bit map-cut plane according to x, y coordinates; extracting a multi-scale LBP feature descriptor in each sub-region and reducing dimensions to obtain a uLBP feature descriptor; extracting feature descriptors of 12 sub-regions of the input face, and connecting the feature descriptors in series to obtain a feature vector based on a multi-bitmap tangent plane and a multi-scale uLBP; and training and testing the occlusion by adopting a Bosphorus database in a support vector machine to finish judgment and identification of the occlusion. The invention has stronger detection and classification performance, simple principle and easy realization.
Description
Technical Field
The invention belongs to the field of three-dimensional image recognition in computer vision, and particularly relates to a three-dimensional face occlusion detection and classification method based on a multi-bitmap tangent plane and multi-scale uLBP.
Background
The three-dimensional face occlusion discrimination technology is a technology for discriminating face occlusion by a computer based on three-dimensional data of a face. The technology has great application potential in the research fields of human-computer interaction, identity verification and the like. Compared with two-dimensional data, the three-dimensional data of the human face is not influenced by factors such as light, posture, angle and the like, and simultaneously contains richer geometric information and topological features, so that human face recognition research based on the three-dimensional human face data has gained more extensive attention in recent years. Facial changes due to occlusion are important factors that should be considered for face recognition applications. Occlusion may be caused by hair or external objects, such as glasses, hats, scarves, or the subject's hand. When a face region is partially occluded, its recognition performance may be drastically degraded because the occlusion causes the loss of discrimination information. Therefore, the problem of occlusion in a three-dimensional face is to be solved.
However, the research of the three-dimensional face occlusion discrimination algorithm still faces multiple difficulties. Feature selection and extraction are one of the difficulties in research. In the current three-dimensional face occlusion discrimination research, methods applied by researchers are mainly classified into methods based on a face curve or a radial curve, methods based on a face model and a threshold, multi-modal methods, methods based on point cloud geometric information, and the like. In the method based on the facial curve or the radial curve, the structure of the characteristics depends on the positions of feature points of the five sense organs, and different shielding effects on different types can be achieved due to different curve selection modes; the multi-modal method applies other information such as color, brightness and the like except point cloud, but is also limited by the other information, and needs to additionally acquire RGB images and the like of the point cloud of the human face; the method based on the point cloud geometric information can describe the geometric characteristics of face deformation, wherein the characteristics comprise normal vectors, curvatures, shape indexes and the like, but the data volume is large. The method based on the multi-bitmap tangent plane and the multi-scale uLBP is selected, the data size is small, the principle is simple, and the operation speed is high.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the problems in the prior art, the invention provides the three-dimensional face occlusion distinguishing method based on the multi-bitmap tangent plane and the multi-scale uLBP, which only needs face point cloud data and does not need any additional information, represents the occlusion condition of a face and improves the accuracy of three-dimensional face occlusion distinguishing by generating a depth map, extracting the bitmap tangent plane and carrying out multi-scale uLBP characteristic extraction on the multi-bitmap tangent plane obtained by synthesis.
The technical scheme is as follows: the invention provides a three-dimensional face shielding distinguishing method based on a multi-bitmap tangent plane and multi-scale uLBP (equivalent pattern LBP, uniform LBP), which comprises the following steps:
(1) Calculating and obtaining a depth map of an input three-dimensional face point cloud;
(2) Calculating the bitmap gray value of the depth map and obtaining 8 bitmap tangent planes, assuming that the gray value at the pixel point of the depth map is T (p), calculating the gray value of each bit of the pixel point as follows:
T(p)＝r 8 ·2 7 +r 7 ·2 6 +r 6 ·2 5 +r 5 ·2 4 +r 4 ·2 3 +r 3 ·2 2 +r 2 ·2 1 +r 1 ·2 0 in the formula r 8 ，r 7 ，r 6 ，r 5 ，r 4 ，r 3 ，r 2 ，r 1 The gray values of the pixel points at the 8 th, 7 th, 6 th, 5 th, 4 th, 3 th, 2 th and 1 st bits respectively;
(3) Extracting 6, 7 and 8 bitmap planes and synthesizing to obtain a multi-bitmap cut plane, wherein the calculation expression of the gray value G (p) at the pixel point p of the multi-bitmap cut plane is as follows:
G(p)＝r 8 +r 7 +r 6 ,
obtaining a plurality of bitmap planes of the input face samples according to the formula;
(4) According to the vertical and horizontal coordinates of the image, namely x, y coordinates, 3*4 partitioning is carried out on the multi-position image cutting plane;
(5) Respectively traversing pixel points in 12 sub-regions of a multi-bit graph cut plane to extract multi-scale LBP (local binary pattern) feature descriptors;
(6) Extracting multi-scale LBP feature descriptor S of each sub-region j And j = 1-12, performing dimensionality reduction operation to obtain a 59-dimensional multi-scale uLBP feature descriptor k j J =1 to 12, j represents a sub-region number;
(7) Inputting each multi-scale uLBP feature descriptor k of 12 sub-regions of the human face j And (4) connecting in series to obtain a feature vector K based on the multi-bitmap tangent plane and the multi-scale uLBP, and taking the feature vector K as a final feature vector of each face point cloud sample.
Further, after the step 7, the method further comprises the following steps: and 8, training and testing the support vector machine by adopting a Bosphorus database, and inputting data as the final characteristics extracted in the step (7) to finish occlusion judgment and identification.
Further, the database used for training and testing is the Bosphorus database.
Further, in extracting the multi-scale LBP feature descriptors from each subarea of the multi-position graph cutting plane, the pixel point q is a central pixel point.
Further, the method for 3*4 partitioning the multi-bit map cutting plane based on the x, y coordinates of the depth map in step (4) is as follows:
(41) The maximum and minimum values of x and y coordinates of a bitmap tangent plane can be obtained according to the size of the depth map;
(42) Equally dividing an x coordinate 3 of the depth map into equal parts, and equally dividing a y coordinate 4 of the depth map into equal parts;
(43) The 1 st patch is marked from 0 to 1/3,y coordinate, the 2 nd patch is marked from 0 to 1/4 coordinate, the 3 rd patch is marked from 0 to 1/3,y coordinate, the 4 th patch is marked from 3/4 to 4/4 coordinate, the 5 th patch is marked from 0/3 to 2/3,y coordinate, the 6 th patch is marked from 1/4 to 2/3,y coordinate, the x coordinate is from 1/3 to 2/3,y coordinate from 2/4 to 3/4 to be marked as the 7 th small block, the x coordinate is from 1/3 to 2/3,y coordinate from 3/4 to 4/4 to be marked as the 8 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 0 to 1/4 to be marked as the 9 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 1/4 to 2/4 to be marked as the 10 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 2/4 to 3/4 to be marked as the 11 th small block, and the x coordinate is from 2/3 to 3/3,y coordinate from 3/4 to 4/4 to be marked as the 12 th small block;
(44) The division of 12 sub-regions of the input face is completed.
Further, the method for extracting the multi-scale LBP feature descriptors of the multi-bitmap tangent plane sub-regions in the step (5) comprises:
(51) Determining a central pixel point q, and assuming that the coordinate of the central pixel point q is (x) q ，y q ) The gray value is G (q);
(52) Selecting neighborhood pixel point q of center pixel point q i I = 1-16, the selection and numbering method of the neighborhood pixels is as follows: the coordinate is (x) q -1，y q Pixel point of-1) is q 1 The coordinate is (x) q ，y q Pixel point of-1) is q 2 The coordinates are (x) q +1，y q Pixel point of-1) is q 3 The coordinate is (x) q +1，y q ) Has a pixel point of q 4 The coordinate is (x) q +1，y q + 1) pixel point is q 5 The coordinate is (x) q ，y q + 1) pixel point is q 6 The coordinate is (x) q -1，y q + 1) pixel point is q 7 The coordinate is (x) q -1，y q ) Has a pixel point of q 8 The coordinate is (x) q -2，y q ) Has a pixel point of q 9 The coordinate is (x) q -2，y q Pixel point of-2) is q 10 The coordinate is (x) q ，y q -2) pixel point q 11 The coordinates are (x) q +2，y q Pixel point of-2) is q 12 The coordinate is (x) q +2，y q ) Has a pixel point of q 13 The coordinate is (x) q +2，y q + 2) pixel point is q 14 The coordinate is (x) q ，y q + 2) pixel point is q 15 The coordinate is (x) q -2，y q + 2) pixel point is q 16 ；
(53) Computing multiscale LBP feature descriptors s i (q), i =1 to 16, and the computational expression is:
wherein, G (q) i ) Is a neighborhood pixel q of pixel q i G (q) is the gray value of pixel point q;
(54) Will be 16 s i (q) two 8-bit binary numbers having values connected in the order of i =1 to 8, i =9 to 16
And
(55) Will be provided with
Conversion to decimalCounting to obtain two decimal numbers in the range of 0-255
And
the calculation expression is as follows:
(56) Traversing all pixel points in the sub-area as central pixel points according to the principle that the x and y coordinates are from small to large to perform the calculation, and then counting the number of the pixel points
The occurrence times of different values in the range of 0-255 are connected according to the sequence from 0-255 from small to large, each subarea obtains a 256-dimensional eigenvector, namely the multi-scale LBP feature descriptor S of the subarea j J =1 to 12, where j denotes a sub-region number.
Further, in the step 7, based on the multi-bitmap tangent plane and the feature vector K of the multi-scale uLBP, the expression is: k = [ k ] 1 k 2 k 3 k 4 k 5 k 6 k 7 k 8 k 9 k 10 k 11 k 12 ]。
Has the advantages that: the invention extracts a Multi-bitmap cut plane of a three-dimensional face depth map, and performs block Multi-scale LBP feature descriptor extraction and dimension reduction on the Multi-bitmap cut plane to obtain Multi-scale uLBP features (Multiscale Uniform LBP features of Multi Bit-plane Slicing, mulBP-MBPs). muLBP-MBPs features extraction of multi-scale uLBP descriptors was performed using a synthetic multi-bit map-cut plane. Not only is the characteristic dimension reduced, but also the information such as the positions and the edges of the face surface and the shielding area can be completely reflected. The algorithm has the advantages of simple principle, small calculation amount and quick judgment of the shielded area. And based on the x, y coordinates of the point cloud, the input face and the block 3*4 are subjected to blocking operation, and 12 sub-regions of the input face can be divided. The blocking method can uniformly separate the five sense organs and other areas of the face, is more robust to the positions and sizes of the five sense organs, has smaller bitmap gray value change caused by different distribution of the five sense organs, and can more sensitively reflect the bitmap forms and the change of the feature descriptors caused by shielding. After the blocking is finished, feature extraction and dimension reduction are respectively carried out in the small blocks, and after multi-scale uLBP feature vectors under the conditions of no shielding and different shielding are obtained, the multi-scale uLBP feature vectors are input into the SVM for training. And finally, realizing the shielding judgment of the three-dimensional face based on the multi-bitmap tangent plane and the multi-scale uLBP. The invention respectively provides innovation for the selection of the original characteristics and the calculation of the statistical characteristics, and obtains better effect in a discrimination experiment.
Drawings
FIG. 1 is an exemplary diagram of a portion of a face from the Bosphorus face library used in the examples;
FIG. 2 is a flow chart of a three-dimensional face occlusion discrimination method of the present invention;
FIG. 3 is a three-dimensional face depth map of the present invention;
FIG. 4 is a cut plane extraction of each bit of a sample of the present invention;
FIG. 5 is a multi-bit map cut plan view resulting from the synthesis of 6, 7, 8 bit map cut planes of the present invention;
FIG. 6 is a statistical chart illustrating the occlusion discrimination accuracy verification according to the present invention.
Detailed Description
The invention is further elucidated with reference to the drawings and the embodiments.
As shown in FIGS. 1-6, matlab R2017b is selected as a programming tool under a Windows operating system, and the method provided by the invention is tested based on Bosphorus three-dimensional face library published by Bogazici University in 2008. FIG. 1 shows an example diagram of no occlusion and 4 basic occlusions of an object in a Bosphorus library. Firstly, selecting 40 objects from a Bosphorus library optionally for training a classifier, wherein the Support Vector Machine (SVM) which is the most widely used classifier in the research in the field is selected as the classifier in the example; after training is complete, the remaining 60 subjects in the library are selected for testing. Meanwhile, each sample selects 200 three-dimensional face pictures under each condition of no-occlusion, hand-occlusion face, glasses occlusion, hair style occlusion and hand-occlusion mouth as training samples. The number of the samples to be detected in this example is sufficient (300 samples in total) and all 5 occlusion cases are included, so that the effectiveness of the identification method provided by the patent is verified.
As shown in fig. 2, the specific identification steps of the three-dimensional face discrimination method based on multi-bitmap tangent plane and multi-scale uLBP of the present invention are as follows:
step 1, inputting three-dimensional face point cloud and obtaining a depth map of the three-dimensional face point cloud;
step 2, calculating bitmap gray value of the depth map and obtaining 8 bitmap cut planes, assuming that the gray value of a pixel point p of the depth map is T (p), calculating gray calculation expressions of each bit of the pixel point as follows:
T(p)＝r 8 ·2 7 +r 7 ·2 6 +r 6 ·2 5 +r 5 ·2 4 +r 4 ·2 3 +r 3 ·2 2 +r 2 ·2 1 +r 1 ·2 0 ，
in the formula r 8 ，r 7 ，r 6 ，r 5 ，r 4 ，r 3 ，r 2 ，r 1 The gray values of the pixel points at the 8 th, 7 th, 6 th, 5 th, 4 th, 3 th, 2 th and 1 st bits respectively;
and 3, extracting 6, 7 and 8 bitmap planes and synthesizing to obtain a multi-bitmap cut plane, wherein the calculation expression of the gray value G (p) at the pixel point p of the multi-bitmap cut plane is as follows:
G(p)＝r 8 +r 7 +r 6 ,
obtaining a plurality of bitmap planes of the input face samples according to the formula;
and 4, according to the x and y coordinates, performing 3*4 partitioning on the multi-position graph cutting plane.
Step 4.1: the maximum and minimum values of x and y coordinates of a bitmap tangent plane can be obtained according to the size of the depth map;
step 4.2: equally dividing an x coordinate 3 of the depth map into equal parts, and equally dividing a y coordinate 4 of the depth map into equal parts;
step 4.3: the 1 st patch is marked from 0 to 1/3,y coordinate, the 2 nd patch is marked from 0 to 1/4 coordinate, the 3 rd patch is marked from 0 to 1/3,y coordinate, the 4 th patch is marked from 3/4 to 4/4 coordinate, the 5 th patch is marked from 0/3 to 2/3,y coordinate, the 6 th patch is marked from 1/4 to 2/3,y coordinate, the x coordinate is from 1/3 to 2/3,y coordinate from 2/4 to 3/4 to be marked as the 7 th small block, the x coordinate is from 1/3 to 2/3,y coordinate from 3/4 to 4/4 to be marked as the 8 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 0 to 1/4 to be marked as the 9 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 1/4 to 2/4 to be marked as the 10 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 2/4 to 3/4 to be marked as the 11 th small block, and the x coordinate is from 2/3 to 3/3,y coordinate from 3/4 to 4/4 to be marked as the 12 th small block;
step 4.4: the division of 12 sub-regions of the input face is completed.
And 5, traversing pixel points in 12 sub-regions of the multi-bit graph cut plane respectively to extract multi-scale LBP (local binary pattern) feature descriptors.
Step 5.1: determining a central pixel point q, and assuming that the coordinate of the central pixel point q is (x) q ，y q ) The gray value is G (q);
step 5.2: selecting neighborhood pixel point q of central pixel point q i I = 1-16, the selection and numbering method of the neighborhood pixel points is as follows: the coordinate is (x) q -1，y q Pixel point of-1) is q 1 The coordinate is (x) q ，y q Pixel point of-1) is q 2 The coordinate is (x) q +1，y q Pixel point of-1) is q 3 The coordinate is (x) q +1，y q ) Has a pixel point of q 4 The coordinate is (x) q +1，y q + 1) pixel point is q 5 The coordinate is (x) q ，y q + 1) pixel point is q 6 The coordinate is (x) q -1，y q + 1) pixel point is q 7 The coordinate is (x) q -1，y q ) Has a pixel point of q 8 The coordinate is (x) q -2，y q ) Has a pixel point of q 9 The coordinates are (x) q -2，y q Pixel point of-2) is q 10 The coordinates are (x) q ，y q Pixel point of-2) is q 11 The coordinate is (x) q +2，y q Pixel point of-2) is q 12 The coordinate is (x) q +2，y q ) Has a pixel point of q 13 The coordinate is (x) q +2，y q + 2) pixel point is q 14 The coordinate is (x) q ，y q + 2) pixel point is q 15 The coordinate is (x) q -2，y q + 2) pixel point is q 16 ；
Step 5.3: computing multiscale LBP feature descriptors s i (q), i =1 to 16, and the computational expression is:
wherein, G (q) i ) Neighborhood pixel point q being pixel point q i G (q) is the gray value of pixel point q;
step 5.4: will be 16 s i (q) two 8-bit binary numbers having values connected in the order of i =1 to 8, i =9 to 16
And
step 5.5: will be provided with
Converting into decimal number to obtain two decimal numbers in the range of 0-255
And
the calculation expression is as follows:
step 5.6: traversing all pixel points in the sub-area as central pixel points according to the principle that the x and y coordinates are from small to large to perform the calculation, and then counting the number of the pixel points
The occurrence times of different values in the range of 0-255 are connected according to the sequence from 0-255 to big, each subarea obtains a 256-dimensional eigenvector, namely a multi-scale LBP (local binary pattern) feature descriptor S of the subarea j J =1 to 12, where j denotes a sub-region number.
Step 6, extracting the obtained multi-scale LBP feature descriptors S of all the sub-regions j J = 1-12 equivalent mode calculation is carried out to obtain 59-dimensional multi-scale uLBP feature descriptor k j J =1 to 12,j denotes a sub-regionA serial number.
Step 6.1: describing subsequences from multi-scale LBP features
The hopping times are classified into equivalent modes, and the mode is discussed according to the situation:
(1) 0 jumps, namely 00000000 or 11111111,2 modes;
(2) 1 jump, 0 → 1 or 1 → 0, such as 00001111, or 11110000, in each case there are 7, for example if it is 1 first, then one jump must be completed at one of the following 7 positions, so there are 14 in both cases;
(3) And 2, jumping for 2 times:
0 → 1 → 0: the first jump is respectively at the position 2, the position 3 and the position 4.. When the position 8 is reached, the second jump is respectively at 6,5.. And 1 possible jump positions, so that 6+5+... Once. +1=21 modes are possible;
1 → 0 → 1: similarly, there are 21 modes, so there may be 42 modes for 2 hops;
(4) More than 2 hops are counted as 1 mode;
in conclusion, for the uLBP, there are 59 patterns possible, namely 59-dimensional feature vectors;
step 6.2: according to the classification method, the feature descriptors belonging to the same mode are subjected to subsequence
I.e. in decimal format
Number of occurrenceThe numbers are overlapped to generate 59-dimensional feature vector k j 。
Step 7, inputting each multi-scale uLBP feature descriptor k of 12 sub-regions of the human face j Serially connected as a final feature vector of each face point cloud sample, i.e. the feature vector K = [ K ] based on multi-bitmap tangent plane and multi-scale uLBP 1 k 2 k 3 k 4 k 5 k 6 k 7 k 8 k 9 k 10 k 11 k 12 ]。
And 8: and (3) training and testing in a support vector machine by adopting a Bosphorus database, and inputting data as the final characteristics extracted in the step (7) to finish the shielding judgment of the three-dimensional face.
The effect verification is carried out on the embodiment: the most widely used classifier in the research in the field, support Vector Machine (SVM), was selected as the classifier. Optionally, 40 objects are selected from the Bosphorus database for training the classifier, and the remaining 60 objects are used for testing the discrimination accuracy of 5 types of occlusions. The Bosphorus database is the most commonly used database in the field of three-dimensional face occlusion processing, and in occlusion samples, 100 more ideal objects are provided, and each object has 5 samples, including: 1 sample without occlusion; class 4 occlusion samples, hand occlusion eyes, eye occlusion, hand occlusion mouth, and hair style occlusion. Only 1 face sample is selected for training and testing in each condition of each object, so that the number of samples in the training set is 200, and the number of samples in the testing set is 300. To reduce the error, the test was repeated 20 times, and the average was taken as the final recognition rate. The average value of the discrimination accuracy rates of the 5 cases was calculated to obtain an average discrimination rate of 92.05%, as shown in the verification effect of fig. 5.
The technical means disclosed in the invention scheme are not limited to the technical means disclosed in the above embodiments, but also include the technical scheme formed by any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and such improvements and modifications are also considered to be within the scope of the present invention.
Claims (6)
1. The three-dimensional face shielding distinguishing method based on the multi-bitmap tangent plane and the multi-scale uLBP is characterized by comprising the following steps of: the discrimination method comprises the following steps:
step 1, calculating and obtaining a depth map of an input three-dimensional face point cloud;
step 2, calculating the bitmap gray value of the depth map and obtaining 8 bitmap cutting planes, assuming that the gray value at a pixel point p of the depth map is T (p), and calculating the gray calculation expression of each bit of the pixel point as follows:
T(p)＝r 8 ·2 7 +r 7 ·2 6 +r 6 ·2 5 +r 5 ·2 4 +r 4 ·2 3 +r 3 ·2 2 +r 2 ·2 1 +r 1 ·2 0 ，
in the formula r 8 ，r 7 ，r 6 ，r 5 ，r 4 ，r 3 ，r 2 ，r 1 The gray values of the pixel points at the 8 th, 7 th, 6 th, 5 th, 4 th, 3 th, 2 th and 1 st bits respectively;
and 3, extracting 6, 7 and 8 bitmap planes and synthesizing to obtain a multi-bitmap cut plane, wherein the calculation expression of the gray value G (p) at the pixel point p of the multi-bitmap cut plane is as follows:
G(p)＝r 8 +r 7 +r 6 ，
obtaining a plurality of bitmap planes of the input face samples according to the formula;
step 4, according to x, y coordinates, 3*4 partitioning is carried out on the multi-position graph cutting plane, and 12 sub-areas are obtained;
step 5, traversing pixel points in 12 sub-regions of a multi-bit graph cut plane respectively to extract multi-scale LBP feature descriptors;
step 6, extracting the obtained multi-scale LBP feature descriptors S of all the sub-areas j J = 1-12, and performing dimensionality reduction operation to obtain 59-dimensional multi-scale uLBP feature descriptor k j J =1 to 12, j represents a sub-region number;
step 7, inputting each multi-scale uLBP feature descriptor k of 12 sub-regions of the human face j Serially connecting to obtain features based on multi-bitmap tangent plane and multi-scale uLBPAnd the vector K is used as a final feature vector of each human face point cloud sample.
2. The method for discriminating three-dimensional face occlusion based on multi-bitmap tangent plane and multi-scale uLBP according to claim 1, characterized in that: after the step 7, the method further comprises the following steps: and 8, training and testing by using a support vector machine, wherein input data are the final characteristics extracted in the step 7, so as to finish occlusion judgment and identification.
3. The method for discriminating three-dimensional face occlusion based on multi-bitmap tangent plane and multi-scale uLBP according to claim 1, characterized in that: and extracting a multi-scale LBP feature descriptor from each subarea of the multi-position graph cutting plane, wherein a pixel point q is a central pixel point.
4. The method for discriminating three-dimensional face occlusion based on multi-bitmap tangent plane and multi-scale uLBP according to claim 1, characterized in that: the method for performing 3*4 blocking on the multi-bit map cutting plane based on the x, y coordinates of the depth map in the step 4 comprises the following steps:
(41) The maximum and minimum values of x and y coordinates of a bitmap tangent plane can be obtained according to the size of the depth map;
(42) Equally dividing an x coordinate 3 of the depth map into equal parts, and equally dividing a y coordinate 4 of the depth map into equal parts;
(43) The 1 st patch is marked from 0 to 1/3,y coordinate, the 2 nd patch is marked from 0 to 1/4 coordinate, the 3 rd patch is marked from 0 to 1/3,y coordinate, the 4 th patch is marked from 3/4 to 4/4 coordinate, the 5 th patch is marked from 0/3 to 2/3,y coordinate, the 6 th patch is marked from 1/4 to 2/3,y coordinate, the x coordinate is from 1/3 to 2/3,y coordinate from 2/4 to 3/4 to be marked as the 7 th small block, the x coordinate is from 1/3 to 2/3,y coordinate from 3/4 to 4/4 to be marked as the 8 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 0 to 1/4 to be marked as the 9 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 1/4 to 2/4 to be marked as the 10 th small block, the x coordinate is from 2/3 to 3/3,y coordinate from 2/4 to 3/4 to be marked as the 11 th small block, and the x coordinate is from 2/3 to 3/3,y coordinate from 3/4 to 4/4 to be marked as the 12 th small block;
(44) The division of 12 sub-regions of the input face is completed.
5. The method for discriminating three-dimensional face occlusion based on multi-bitmap tangent plane and multi-scale uLBP according to claim 1, characterized in that: the method for extracting the multi-scale LBP feature descriptors of the multi-bitmap tangent plane sub-regions in the step 5 comprises the following steps:
(51) Determining a central pixel point q, and assuming that the coordinate of the central pixel point q is (x) q ，y q ) The grey scale value is G (q);
(52) Selecting neighborhood pixel point q of center pixel point q i I = 1-16, the selection and numbering method of the neighborhood pixels is as follows: the coordinate is (x) q -1，y q -1) pixel point q 1 The coordinate is (x) q ，y q -1) pixel point q 2 The coordinates are (x) q +1，y q Pixel point of-1) is q 3 The coordinate is (x) q +1，y q ) Has a pixel point of q 4 The coordinate is (x) q +1，y q + 1) pixel point is q 5 The coordinates are (x) q ，y q The pixel point of + 1) is q 6 The coordinate is (x) q -1，y q The pixel point of + 1) is q 7 The coordinates are (x) q -1，y q ) Has a pixel point of q 8 The coordinate is (x) q -2，y q ) Has a pixel point of q 9 The coordinate is (x) q -2，y q Pixel point of-2) is q 10 The coordinate is (x) q ，y q Pixel point of-2) is q 11 The coordinate is (x) q +2，y q Pixel point of-2) is q 12 The coordinate is (x) q +2，y q ) Has a pixel point of q 13 The coordinate is (x) q +2，y q + 2) pixel point is q 14 The coordinate is (x) q ，y q + 2) pixel point is q 15 The coordinates are (x) q -2，y q + 2) pixel point is q 16 ；
(53) Computing multiscale LBP feature descriptors s i (q), i =1 to 16, and the computational expression is:
wherein, G (q) i ) Neighborhood pixel point q being pixel point q i G (q) is the gray value of pixel point q;
(54) Will be 16 s i (q) two 8-bit binary numbers having values connected in the order of i =1 to 8, i =9 to 16
And
(55) Will be provided with
Converting into decimal number to obtain two decimal numbers in the range of 0-255
And
the calculation expression is as follows:
(56) Traversing all pixel points in the sub-area as central pixel points according to the principle that the x and y coordinates are from small to large to perform the calculation, and then counting the number of the pixel points
The occurrence times of different values in the range of 0-255 are connected according to the sequence from 0-255 to big, each subarea obtains a 256-dimensional eigenvector, namely a multi-scale LBP (local binary pattern) feature descriptor S of the subarea j J =1 to 12, where j denotes a sub-region number.
6. The method for discriminating three-dimensional face occlusion based on multi-bitmap tangent plane and multi-scale uLBP according to claim 1, characterized in that: in the step 7, the expression of the feature vector K based on the multi-bitmap tangent plane and the multi-scale uLBP is as follows: k = [ K ] 1 k 2 k 3 k 4 k 5 k 6 k 7 k 8 k 9 k 10 k 11 k 12 ]。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255272.1A CN111553195B (en) | 2020-04-02 | 2020-04-02 | Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255272.1A CN111553195B (en) | 2020-04-02 | 2020-04-02 | Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111553195A CN111553195A (en) | 2020-08-18 |
| CN111553195B true CN111553195B (en) | 2022-11-15 |
Family
ID=72000274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010255272.1A Active CN111553195B (en) | 2020-04-02 | 2020-04-02 | Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111553195B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052120B (en) * | 2021-04-08 | 2021-12-24 | 深圳市华途数字技术有限公司 | Entrance guard's equipment of wearing gauze mask face identification |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886091A (en) * | 2019-01-08 | 2019-06-14 | 东南大学 | Three-dimensional face expression recognition methods based on Weight part curl mode |
| CN110532915A (en) * | 2019-08-20 | 2019-12-03 | 东南大学 | Three-dimensional face based on normal vector azimuth local entropy blocks method of discrimination |
-
2020
- 2020-04-02 CN CN202010255272.1A patent/CN111553195B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886091A (en) * | 2019-01-08 | 2019-06-14 | 东南大学 | Three-dimensional face expression recognition methods based on Weight part curl mode |
| CN110532915A (en) * | 2019-08-20 | 2019-12-03 | 东南大学 | Three-dimensional face based on normal vector azimuth local entropy blocks method of discrimination |
Non-Patent Citations (1)
| Title |
|---|
| 融合局部特征的面部遮挡表情识别;王晓华等;《中国图象图形学报》;20161116(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111553195A (en) | 2020-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359538B (en) | Training method of convolutional neural network, gesture recognition method, device and equipment | |
| CN107742102B (en) | Gesture recognition method based on depth sensor | |
| Aldoma et al. | OUR-CVFH–oriented, unique and repeatable clustered viewpoint feature histogram for object recognition and 6DOF pose estimation | |
| WO2015161816A1 (en) | Three-dimensional facial recognition method and system | |
| CN110334762B (en) | Feature matching method based on quad tree combined with ORB and SIFT | |
| Li et al. | Expression-robust 3D face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns | |
| Min et al. | Efficient detection of occlusion prior to robust face recognition | |
| CN103996052B (en) | Three-dimensional face gender classification method based on three-dimensional point cloud | |
| CN108062543A (en) | A kind of face recognition method and device | |
| US9443137B2 (en) | Apparatus and method for detecting body parts | |
| WO1997044757A2 (en) | Principal component analysis of image/control-point location coupling for the automatic location of control points | |
| CN107992800A (en) | A kind of fingerprint image quality determination methods based on SVM and random forest | |
| JP2005535017A (en) | Extended Isomap using Fisher linear discriminant function and Kernel Fisher linear discriminant function | |
| CN106611158A (en) | Method and equipment for obtaining human body 3D characteristic information | |
| Angadi et al. | Face recognition through symbolic modeling of face graphs and texture | |
| Alsawwaf et al. | In your face: person identification through ratios and distances between facial features | |
| Gürel et al. | Design of a face recognition system | |
| CN110532915B (en) | Three-dimensional face shielding discrimination method based on normal vector azimuth local entropy | |
| CN111553195B (en) | Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP | |
| Abid et al. | Dynamic hand gesture recognition from Bag-of-Features and local part model | |
| CN111488811A (en) | Face recognition method and device, terminal equipment and computer readable medium | |
| Abdullah-Al-Wadud et al. | Skin segmentation using color distance map and water-flow property | |
| Dai et al. | An Improved ORB Feature Extraction Algorithm Based on Enhanced Image and Truncated Adaptive Threshold | |
| Lv et al. | A 3D face recognition method using region-based extended local binary pattern | |
| Zhou et al. | Shape matching based on rectangularized curvature scale-space maps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |