CN108898547B - Single-sample-based face image virtual sample expansion method and system - Google Patents
Single-sample-based face image virtual sample expansion method and system Download PDFInfo
- Publication number
- CN108898547B CN108898547B CN201810675044.2A CN201810675044A CN108898547B CN 108898547 B CN108898547 B CN 108898547B CN 201810675044 A CN201810675044 A CN 201810675044A CN 108898547 B CN108898547 B CN 108898547B
- Authority
- CN
- China
- Prior art keywords
- virtual sample
- sample
- virtual
- face image
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 239000011159 matrix material Substances 0.000 claims abstract description 106
- 230000009466 transformation Effects 0.000 claims abstract description 45
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 10
- 238000010586 diagram Methods 0.000 claims description 32
- 238000012545 processing Methods 0.000 claims description 14
- 230000001815 facial effect Effects 0.000 claims description 4
- 230000003416 augmentation Effects 0.000 claims 1
- 238000005286 illumination Methods 0.000 abstract description 6
- 238000011426 transformation method Methods 0.000 abstract description 3
- 238000012549 training Methods 0.000 description 29
- 230000000694 effects Effects 0.000 description 10
- 238000012360 testing method Methods 0.000 description 9
- 238000002474 experimental method Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 7
- 238000004088 simulation Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000036544 posture Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000012733 comparative method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 206010052804 Drug tolerance Diseases 0.000 description 1
- 206010034719 Personality change Diseases 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000026781 habituation Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Abstract
The invention discloses a face image virtual sample expansion method and system based on a single sample. The method and system superimpose non-Negative Matrix Factorization (NMF) on the mirror transformation, sliding window method and bit image method to produce rich virtual samples. The invention integrates the advantages of three sample expansion methods, namely a mirror image transformation method, a window sliding method and a bit plane method, and improves the robustness to the posture, the expression and the illumination. The virtual sample generated by the window sliding method and the bit plane method is subjected to mirror image transformation again, and the image subjected to mirror image transformation contains more information different from the original image, so that the original image information is fully mined and utilized; and the original image and the virtual samples generated by the mirror image transformation, the window sliding method and the bit plane method are subjected to non-negative matrix decomposition (NMF) to reconstruct a new image, so that the accuracy of face recognition can be improved.
Description
Technical Field
The invention relates to the field of sample expansion, in particular to a method and a system for expanding a face image virtual sample based on a single sample.
Background
Most mature face recognition technologies are directed at the situation of many samples, these existing recognition methods depend on the number of samples in a training sample set and their representativeness to a great extent, and in an actual application environment, a face image database has some potential problems, for example, it is difficult to obtain a plurality of different face samples of each person due to the difficulty of sample acquisition; or the limitation of the storage space of the database, the number of the stored sample images of each person is limited, and even each person has only one image, so that the problem of single sample is caused. The single-sample face recognition means that only one face image for training is stored in a face training database for each person, and identity of an unpredictable face image is recognized through the single face image, and recognition problems such as identity card authentication, driving license authentication, passport authentication and the like belong to single-sample face recognition. Under such conditions, the performance of the conventional face recognition technology is seriously reduced, even the conventional face recognition technology cannot work, and a relatively ideal recognition effect is difficult to obtain. Therefore, research on face recognition in the case of a single sample is of importance and necessity.
In recent years, the research of researchers at home and abroad on the single-sample problem in the face recognition problem is currently divided into two types: one is to use only one face image of each person and use various methods to generate a virtual sample of the person, so as to expand the number of samples of the person in the training sample set. However, although the sample expansion method enriches training samples, the generated virtual samples have too high similarity with the original image, the existing face information of the original image is not fully mined, and the improvement of the recognition rate is limited. In the other method, feature extraction is researched, features are extracted from a face image by adopting various technologies, and useful face feature information is obtained from the original image as much as possible, so that the recognition effect is improved.
Disclosure of Invention
The invention aims to provide a single-sample-based face image virtual sample expansion method and a single-sample-based face image virtual sample expansion system, which are used for expanding the number of samples and improving the accuracy of face recognition.
In order to achieve the purpose, the invention provides the following scheme:
a human face image virtual sample expansion method based on a single sample comprises the following steps:
acquiring a face image;
carrying out horizontal mirror image transformation on the face image to obtain a first virtual sample; the horizontal mirror image transformation is to exchange the left half part and the right half part of the image by taking the vertical central axis of the image as the center;
selecting a window size and a sliding step length of a sliding window, wherein the window size and the sliding step length are both smaller than the side length of the face image;
intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample;
processing the face image by a bit plane method to obtain a third virtual sample;
respectively carrying out horizontal mirror image transformation on the second virtual sample and the third virtual sample to obtain a corresponding fourth virtual sample and a corresponding fifth virtual sample;
respectively carrying out non-negative matrix factorization reconstruction on the face image, the first virtual sample, the second virtual sample and the third virtual sample to obtain a sixth virtual sample, a seventh virtual sample, an eighth virtual sample and a ninth virtual sample which correspond to the first virtual sample and the second virtual sample;
determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample.
Optionally, the intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample specifically includes:
calculating the sliding times according to the window size, the sliding step length and the size of the face image;
and intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
Optionally, the processing the face image by using a bit plane method to obtain a third virtual sample specifically includes:
acquiring a bit plane map of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram;
selecting the bit plane map according to a bit height threshold value to obtain a selected bit plane map;
and combining the selected bit plane maps to obtain a third virtual sample.
Optionally, the performing non-negative matrix factorization reconstruction on the face image to obtain a corresponding sixth virtual sample specifically includes:
acquiring a gray value of the face image;
determining a non-negative matrix according to the gray value;
determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
A single-sample based face image virtual sample expansion system, the system comprising:
the face image acquisition module is used for acquiring a face image;
the first transformation module is used for carrying out horizontal mirror image transformation on the face image to obtain a first virtual sample; the horizontal mirror image transformation is to exchange the left half part and the right half part of the image by taking the vertical central axis of the image as the center;
the selection module is used for selecting the window size and the sliding step length of a sliding window, and the window size and the sliding step length are both smaller than the side length of the face image;
the intercepting module is used for intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample;
the processing module is used for processing the face image by a bit plane method to obtain a third virtual sample;
the second transformation module is used for respectively carrying out horizontal mirror image transformation on the second virtual sample and the third virtual sample to obtain a corresponding fourth virtual sample and a corresponding fifth virtual sample;
the non-negative matrix decomposition and reconstruction module is used for respectively carrying out non-negative matrix decomposition and reconstruction on the face image, the first virtual sample, the second virtual sample and the third virtual sample to obtain a corresponding sixth virtual sample, a corresponding seventh virtual sample, a corresponding eighth virtual sample and a corresponding ninth virtual sample;
the determining module is used for determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample.
Optionally, the intercepting module specifically includes:
the calculating unit is used for calculating the sliding times according to the window size, the sliding step length and the size of the face image;
and the intercepting unit is used for intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
Optionally, the processing module specifically includes:
the bit plane image acquisition unit is used for acquiring a bit plane image of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram;
the selecting unit is used for selecting the bit plane diagram according to the bit height threshold value to obtain the selected bit plane diagram;
and the combination unit is used for combining the selected bit plane maps to obtain a third virtual sample.
Optionally, the non-negative matrix factorization reconstruction module specifically includes:
the gray value acquisition unit is used for acquiring the gray value of the face image;
the non-negative matrix determining unit is used for determining a non-negative matrix according to the gray value;
a first matrix and second matrix determining unit for determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and the reconstruction unit is used for reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
Compared with the prior art, the invention has the following technical effects:
the invention is researched aiming at the face recognition under the condition of a single sample. The traditional face recognition method mainly depends on the number of training samples, so that the feature extraction is not accurate in single sample, and the recognition rate is low. Therefore, the sample expansion method was studied to directly enrich the number of training samples. However, the correlation between the original image and the virtual sample generated by the conventional sample expansion method is high, and the function of the face information included in the original image cannot be fully exerted. Accordingly, the present invention proposes a new sample expansion method and system that superimposes non-Negative Matrix Factorization (NMF) on the mirror transform, sliding window method, and bit-image method to produce rich virtual samples. The invention integrates the advantages of three sample expansion methods, namely a mirror image transformation method, a window sliding method and a bit plane method, and improves the robustness to the posture, the expression and the illumination. The virtual sample generated by the window sliding method and the bit plane method is subjected to mirror image transformation again, and the image subjected to mirror image transformation contains more information different from the original image, so that the original image information is fully mined and utilized; and the original image and the virtual samples generated by the mirror image transformation, the window sliding method and the bit plane method are subjected to non-negative matrix decomposition (NMF) to reconstruct a new image, so that the accuracy of face recognition can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flowchart of a method for expanding a virtual face image sample based on a single sample according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a face image and a virtual sample generated by mirror image transformation thereof according to an embodiment of the present invention;
FIG. 3 is a face image and a virtual sample generated by a sliding window method thereof according to an embodiment of the present invention;
FIG. 4 is a face image and 8 bit-plane views thereof according to an embodiment of the present invention;
FIG. 5 is a partial virtual sample of bit plane synthesis according to an embodiment of the present invention;
FIG. 6 is an ORL face library partial face image and its NMF reconstructed image;
FIG. 7 is a partial face image of a FERET face library and its NMF reconstructed image;
FIG. 8 shows 10 virtual samples corresponding to a person in the ORL face library;
FIG. 9 shows 10 virtual samples corresponding to a person in the FERET face library;
FIG. 10 is an ORL face library showing average recognition rates corresponding to different r values;
FIG. 11 is an ORL face library, the time for reconstructing an image by NMF corresponding to different r values;
FIG. 12 shows the average recognition rates corresponding to different r values in the FERET face library;
fig. 13 is a FERET face library, showing the time for reconstructing an image by NMF corresponding to different r values;
FIG. 14 is an ORL face library comparing the recognition rates of the present invention and the comparison method;
FIG. 15 is a FERET face library comparing the recognition rates of the present invention and the comparison method;
fig. 16 is a block diagram illustrating a virtual sample expansion system for a human face image based on a single sample according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a flowchart of a method for expanding a virtual face image sample based on a single sample according to an embodiment of the present invention. As shown in fig. 1, a method for expanding a virtual sample of a face image based on a single sample includes:
step 101: and acquiring a human face image.
Step 102: carrying out horizontal mirror image transformation on the face image to obtain a first virtual sample; the horizontal mirror transformation is to exchange the left half and the right half of the image with the vertical central axis of the image as the center.
The mirror image transformation is a kind of geometric transformation, and can reduce the influence of the head rotation on the recognition effect to a certain extent while not introducing interference information. The mirror transformation of an image includes three types: horizontal mirror image, vertical mirror image and diagonal mirror image, and the three transformations generate virtual samples as shown in fig. 2, and the transformations do not change the shape of the original image. In consideration of the habituation of face image shooting, horizontal mirror image transformation is adopted, which reflects some possible changes of the face to a certain extent and can provide richer feature information for the face sample.
Horizontal mirroring is the exchange of the left and right halves of an image around the vertical central axis of the image. If the size of the original image (face image) is a × b, where (x)0,y0) Corresponds to a coordinate point in the horizontal mirror image as (x1, y1), and the relationship therebetween satisfies the following equation:
step 103: and selecting the window size and the sliding step length of the sliding window, wherein the window size and the sliding step length are both smaller than the side length of the face image.
Step 104: and intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample. Calculating the sliding times according to the window size, the sliding step length and the size of the face image; and intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
The sliding window method is to select a certain window size and sliding step length to slide on the width and height of the image. The method is completely based on the sample information of the image, interference information is not introduced, the influence of external noise is avoided, and the inherent information of the original image is maintained and enhanced to the maximum extent by the expanded sample. The method adopts a sliding window method, and concretely comprises the following steps:
let the sample image size be a × b, the sliding window size be x × y, and x < a, y < b. The window starts to slide from the upper left corner of the image, and the distances that the window can slide in the two directions of the width and the height of the sample image are a-x and b-y respectively;
setting the sliding step length of the window in the width direction and the height direction as x respectively1，y1And x is1≤a-x，y1≤b-y。
According to the set sample image size, the sliding window size and the sliding step size, the number n of times that the window can respectively slide in the width direction and the height direction of the sample image is calculated1And n2The specific calculation is as follows.
And according to the calculated sliding times, sliding the window on the original image along the width and the height by corresponding step lengths, namely intercepting a series of virtual samples to sequentially expand the training sample set.
Taking a face sample of the ORL face library as an example, the image size is 92 × 112, and if the obtained virtual sample contains as much original image information as possible, the window size needs to be set slightly larger, for example, the window size is set to 84 × 104, and the sliding step lengths in the width and height directions are both set to 8, so that n calculated in this way is1And n2All the values of (2) are obtained, and the window is slid from the upper left corner of the original image to obtain 4 virtual samples, as shown in fig. 3, which can be regarded as a plurality of human face sub-images of a person.
Step 105: and processing the face image by a bit plane method to obtain a third virtual sample. Acquiring a bit plane map of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram; selecting the bit plane map according to a bit height threshold value to obtain a selected bit plane map; and combining the selected bit plane maps to obtain a third virtual sample.
In a computer, each pixel of an image is converted into 8-bit binary data for storage, and values on bits with the same weight are sequentially taken from the high order to form a bit plane, so that an image can be decomposed into 8 bit planes, as shown in fig. 4.
The image information distributed by each different bit plane is different, and only a plurality of bit plane images with higher bits are distributed with visually meaningful information, wherein the higher bit planes contain the obvious outline information of the image; the bit plane of the middle bit represents the background information of the image; the low bit planes cover the detail information of the image, but the randomness is stronger. The original image is denoted by A, and the generated 8 bit plane images are sequentially denoted by A8，A7，A6…A1The 8 images can be recombined into a series of new images according to different combinations, the combination formula is shown as formula (4), and alpha can be adjustedi(0≤αiAnd less than or equal to 1) to obtain virtual samples generated under different combinations.
A'＝α8A8+α7A7+α6A6+…+α1A1 (4)
Combining the bit images according to different weights to obtain a series of virtual samples as shown in FIG. 5, wherein the weights from left to right are sequentially set to be alpha8～α1＝1,α8～α2＝1,α8～α3＝1,...,α8～α6As can be seen from 1, the images formed by the combination of the lower bit planes and the combination of no lower bit planes have no significant visual difference, and the discrimination information included in the lower bit planes is not much, and the effect on the structure of the image information is not so large, and the middle and high bit planes can be selected for the structure when the virtual image is constructed. Setting the weight alpha of the bit plane8～α4＝1,α3～α1And (5) selecting the upper 5 bit planes to generate a virtual sample according to the weighted sum combination.
Step 106: and respectively carrying out horizontal mirror image transformation on the second virtual sample and the third virtual sample to obtain a fourth virtual sample and a fifth virtual sample which correspond to each other.
Step 107: and respectively carrying out nonnegative matrix factorization reconstruction on the face image, the first virtual sample, the second virtual sample and the third virtual sample to obtain a sixth virtual sample, a seventh virtual sample, an eighth virtual sample and a ninth virtual sample which correspond to the first virtual sample, the second virtual sample and the third virtual sample. Taking a face image as an example:
acquiring a gray value of the face image;
determining a non-negative matrix according to the gray value;
determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
The processing of the first virtual sample, the second virtual sample, and the third virtual sample is the same as above.
Introduction to NMF theory
Non-Negative Matrix Factorization (NMF) means that for a Non-negative Matrix V, two Non-negative matrices W and H are found such that their product is approximately equal to the original Matrix V. Described in mathematical language as: given an m × n non-negative matrix V, an m × r non-negative matrix W and an r × n non-negative matrix H are found such that W and H satisfy equation (5). The original matrix V can be seen as a weighted sum of linear combinations of all column vectors in the left matrix W, while the elements of all column vectors in the right matrix H are the weight coefficients. Thus, W is a non-negative basis matrix and H is a non-negative coefficient matrix. In addition, r is generally selected to satisfy
Therefore, the dimensionality of both the matrix W and the matrix H is smaller than that of the original matrix V, the original matrix V is replaced by the coefficient matrix H, the dimensionality reduction is achieved, and r is the dimensionality after dimensionality reduction and is the number of base images corresponding to the non-negative base matrix W.
V≈WH (5)
The decomposition result of the NMF has exact physical meaning, and for a face image, the non-negative matrix V of m × n can be understood as: v is a matrix formed by n images of m × 1 dimension, each column of which represents the gray level of a face image, and the gray level of the face image is a non-negative value. NMF is a partial decomposition method, for a human face image, the result of decomposition is the sub-features of the human face, and the sub-features conform to the human thinking that the part forms the whole.
According to equation (5), the non-negative matrices W and H are approximations of the original image V, let
Y＝WH (6)
Then Y is the reconstructed image. And the approximation degree of the reconstructed image and the original image is related to the value of r. The initial r value was set to 60. Fig. 6 and 7 below show partial original images and their corresponding reconstructed images in the ORL and FERET face libraries, respectively. r is the number of basis matrices represented by the W matrix and the characteristic dimension of the H matrix. Different W and H matrices can be obtained by adjusting the size of r, for a face image, W is [ W1, W2., wr ] is a base image, H is [ H1, H2., hr ] is a weight coefficient, and each face in V is a linear combination of the base images.
By comparing the original image with the reconstructed image, the eyebrow, eye, nose and mouth information of the reconstructed image of the human face can be seen visually to be more prominent. The base matrix W and the coefficient matrix H of the NMF method are required to be non-negative, so that the weight coefficients represented by the elements of all the column vectors in H are additive combinations for linear combinations of all the column vectors in W, and are not subjected to subtraction operation, and the reconstructed face image has better clamping performance, so that the expression effect of the reconstructed face image is better, and the representation of the face image information is more compact and has less redundancy.
Step 108: determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample. Fig. 8 illustrates 10 virtual samples corresponding to a person in the ORL face library, and fig. 9 illustrates 10 virtual samples corresponding to a person in the fee face library.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention integrates the advantages of three sample expansion methods, namely, a mirror image transformation method, a window sliding method and a bit plane method, and improves the robustness to the posture, the expression and the illumination;
performing mirror image transformation on the virtual samples generated by the window sliding method and the bit plane method again, wherein the image subjected to mirror image transformation contains more information different from the original image, so that the original image information is fully mined and utilized;
the non-negative matrix decomposition NMF is carried out on the original image and virtual samples generated by the mirror image transformation, the window sliding method and the bit plane method to reconstruct a new image, the characterization capability of the face image is enhanced due to the non-negative constraint of the NMF, the face information contained in the reconstructed image is compact and less redundant by selecting the optimal r value, and a good foundation is laid for the subsequent feature extraction and classification and identification.
Simulation of experiment
1. And (4) experimental environment. The simulation experiment environment is Intel (R) core (TM) i5-2520M CPU, 6GB memory, Win7 operating system, and Matlab R2015b software.
2. And (5) introducing a face library. The ORL face library has 40 types of objects, 10 images are taken by each person, single-sample face recognition is carried out in the experiment, 10 samples of each person are taken as a training set in turn, the kth sample of each person is taken as an original sample every time, and the rest 9 images are taken as a test set. Thus, the original training set A contains 40 samples and the test set Y contains 360 samples. If virtual samples are generated by using the method, each sample has 10 virtual samples including the original image, then the virtual sample set E of the face library has 480 training samples.
The FERET face library selects 200 types of objects, 7 images of each person, and 1400 images in total. In the experiment, each person selects 1 sample to perform single-sample face recognition. Because the facial images in the facial library have large changes in expression, posture, illumination and the like, the first front face image of each person is selected as a training set in the experiment, and the rest 6 front face images are used as test sets and recorded as testAll. Wherein, the 2 nd to 5 th images of each person are images with changed postures and are recorded as test 1; the 6 th image of each person is an image with changed expression and is recorded as test 2; the 7 th image of each person was an image of the change in illumination, denoted test3, as shown in Table 1. Thus, the original training set a contains 200 samples, the test set test1 has 200 samples, test2 has 800 samples, and test3 has 200 samples. The virtual sample set E contains 2400 training samples.
Table 1 test set partitioning of FERET face library
| Test set | test1 | test2 | test3 |
| Image information | Attitude change | Changes in expression | Variation of illumination |
| Position of | Sheets 2 to 5 | No. 6 | 7 th sheet |
The experiment was compared to a standard. After a new face training set is expanded, the sizes of training samples and testing samples are unified to be 64 multiplied by 80, feature extraction is carried out according to a WPD-HOG pyramid feature extraction method, and then classification and recognition are carried out through an SVM. The experimental simulation of single-sample face recognition is carried out on an ORL face database and a FERET face database, the influence of the NMF-based virtual sample reconstruction method on the recognition effect when the dimension r value is different is researched, and the recognition rate is compared with the traditional three sample expansion methods of mirror transformation, a sliding window method and a bit plane method.
And setting a contrast experiment, and verifying the effectiveness of the NMF-based virtual sample reconstruction method on an ORL face library and a FERET face library. The specific settings are as follows:
the invention comprises the following steps: generating three corresponding virtual samples A1, A2 and A3 for the original sample A by using mirror image transformation, window sliding and a bit plane method respectively; then, mirror image transformation is carried out on the virtual samples A2 and A3 corresponding to the two window sliding methods and the bit plane method again to generate two virtual samples M1 and M2; in addition, NMF reconstruction of A, A1, a2, A3 all produced new virtual samples N1, N2, N3, N4. The final new training sample set E ═ A, A1, a2, A3, M1, M2, N1, N2, N3, N4, and this method of enriching the training sample set with NMF reconstruction virtual samples is experimentally denoted as "invention".
Original: and (3) expanding the training sample set without any method, directly carrying out subsequent feature extraction and classification identification, and marking the comparison experiment as 'original'.
Mirroring: the original training sample A is subjected to mirror image transformation to obtain a virtual sample A1, A and A1 together form a new training sample set B { A, A1}, and the sample expansion method is simplified as mirror image.
Window: the original training sample a is subjected to window sliding to obtain a virtual sample a2, a and a2 together form a new training sample set C ═ a, a2}, and this sample expansion method is abbreviated as "window".
Bit plane: the bit plane method is performed on the original training sample a to obtain a virtual sample A3, a and A3 together form a new training sample set D ═ a, a1}, and this sample extension method is abbreviated as "bit plane".
4. And (5) analyzing an experimental result.
First, the influence of the r value on the recognition rate was investigated.
The optimal r values of different face libraries are selected differently, and the influence of different r values of the ORL face library and the FERET face library on the recognition rate and the running time required by the NMF to reconstruct an image along with the increase of the r values are researched respectively. The new sample expansion method proposed herein performs experiments on the ORL face library, and the corresponding recognition rates when the r values take different values are shown in table 2.
TABLE 2ORL face library, influence of r-value on recognition rate
Wherein a comparison of the corresponding average recognition rates is clearly shown in fig. 10. The time for reconstructing one image by NMF for different r values is shown in fig. 11. The new sample expansion method proposed herein is performed on the FERET face library, and the corresponding recognition rates when the r value takes different values are shown in table 3, where the comparison of the average recognition rates represented by the testAll training set is shown in fig. 12. The time for reconstructing one image by NMF for different r values is shown in fig. 13.
TABLE 3FERET face library, influence of r-value on recognition rate
According to the above experimental simulation on the ORL and FERET face libraries, it can be seen that the face recognition method based on the NMF reconstructed virtual sample proposed herein has the following features:
the recognition rates of the two face libraries are increased along with the increase of r, and when r is increased to a certain degree, the recognition rate is reduced. When the r value is too small, the reconstructed image contains too little face information, and the face cannot be effectively represented; when the value of r is too large, the reconstructed image is doped with much redundant information and some information disturbing the recognition rate. Therefore, too small or too large r value is detrimental to recognition.
The more samples the face library contains, the larger the optimal r value. From the experimental results, for the ORL face library with the sample number of 400, the best effect is achieved when the r value is 50; for the FERET face library with 1400 samples, the r value is selected to be 400, so that the effect is the best. This is because the NMF is for the whole training set, the feature extraction of each image is closely related to the whole training set, and the samples are not independent. Thus, as the number of training set samples increases, the optimal r value increases accordingly.
The larger the value of r, the more time is required to reconstruct an image. From the experimental results of fig. 6-9 and 6-11, it can be seen that as the r value increases, the time to reconstruct the image increases. The reconstructed image, the number of the base images of the non-negative matrix W of the r value is the characteristic dimension of H, and the larger the r value is, the more complicated the reconstructed image is, and therefore, the more computer running time is required.
Secondly, the effectiveness of the proposed method was investigated:
the experimental results of the present invention and the comparison method on the ORL face library are shown in fig. 14. As can be seen, the new sample expansion method proposed by the present invention has a recognition rate about 3% -8% higher than that of the comparative method. The experimental results of the present invention and the comparative method on the FERET face library are shown in FIG. 15. As can be seen, the new sample expansion method proposed by the present invention has a recognition rate about 3% -5% higher than that of the comparative method. The effectiveness of the invention is proved by experimental simulation of two face libraries.
As shown in fig. 16, the present invention further provides a single-sample-based face image virtual sample expansion system, which includes:
the face image obtaining module 1601 is configured to obtain a face image.
A first transformation module 1602, configured to perform horizontal mirror transformation on the face image to obtain a first virtual sample; the horizontal mirror transformation is to exchange the left half part and the right half part of the image by taking the vertical central axis of the image as the center.
A selecting module 1603, configured to select a window size and a sliding step size of a sliding window, where the window size and the sliding step size are both smaller than the side length of the face image.
And an intercepting module 1604, configured to intercept the face image according to the window size and the sliding step length to obtain a second virtual sample.
The intercepting module 1604 specifically includes:
the calculating unit is used for calculating the sliding times according to the window size, the sliding step length and the size of the face image;
and the intercepting unit is used for intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
A processing module 1605, configured to process the face image by a bit plane method, so as to obtain a third virtual sample.
The processing module 1605 specifically includes:
the bit plane image acquisition unit is used for acquiring a bit plane image of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram;
the selecting unit is used for selecting the bit plane diagram according to the bit height threshold value to obtain the selected bit plane diagram;
and the combination unit is used for combining the selected bit plane maps to obtain a third virtual sample.
A second transformation module 1606, configured to perform horizontal mirror transformation on the second virtual sample and the third virtual sample, respectively, to obtain a fourth virtual sample and a fifth virtual sample that correspond to each other.
A non-negative matrix factorization reconstruction module 1067, configured to perform non-negative matrix factorization reconstruction on the face image, the first virtual sample, the second virtual sample, and the third virtual sample, respectively, to obtain a sixth virtual sample, a seventh virtual sample, an eighth virtual sample, and a ninth virtual sample, which correspond to each other.
The non-negative matrix factorization reconstruction module 1607 specifically includes:
the gray value acquisition unit is used for acquiring the gray value of the face image;
the non-negative matrix determining unit is used for determining a non-negative matrix according to the gray value;
a first matrix and second matrix determining unit for determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and the reconstruction unit is used for reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
A determining module 1608 for determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. A face image virtual sample expansion method based on a single sample is characterized by comprising the following steps:
acquiring a face image;
performing horizontal mirror image transformation on the face image to obtain a first virtual sample; the horizontal mirror image transformation is to exchange the left half part and the right half part of the image by taking the vertical central axis of the image as the center;
selecting a window size and a sliding step length of a sliding window, wherein the window size and the sliding step length are both smaller than the side length of the face image;
intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample;
processing the face image by a bit plane method to obtain a third virtual sample;
respectively carrying out horizontal mirror image transformation on the second virtual sample and the third virtual sample to obtain a corresponding fourth virtual sample and a corresponding fifth virtual sample;
respectively carrying out non-negative matrix factorization reconstruction on the face image, the first virtual sample, the second virtual sample and the third virtual sample to obtain a sixth virtual sample, a seventh virtual sample, an eighth virtual sample and a ninth virtual sample which correspond to the first virtual sample and the second virtual sample;
determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample.
2. The method for expanding the virtual samples of the face images according to claim 1, wherein the intercepting the face images according to the window size and the sliding step length to obtain a second virtual sample specifically comprises:
calculating the sliding times according to the window size, the sliding step length and the size of the face image;
and intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
3. The method for expanding the virtual samples of the face images according to claim 1, wherein the processing the face images by a bit plane method to obtain a third virtual sample specifically comprises:
acquiring a bit plane map of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram;
selecting the bit plane map according to a bit height threshold value to obtain a selected bit plane map;
and combining the selected bit plane maps to obtain a third virtual sample.
4. The method for expanding virtual samples of a human face image according to claim 1, wherein the performing non-negative matrix factorization reconstruction on the human face image to obtain a corresponding sixth virtual sample specifically comprises:
acquiring a gray value of the face image;
determining a non-negative matrix according to the gray value;
determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
5. A human face image virtual sample expansion system based on single sample is characterized in that the system comprises:
the face image acquisition module is used for acquiring a face image;
the first transformation module is used for carrying out horizontal mirror image transformation on the face image to obtain a first virtual sample; the horizontal mirror image transformation is to exchange the left half part and the right half part of the image by taking the vertical central axis of the image as the center;
the selection module is used for selecting the window size and the sliding step length of a sliding window, and the window size and the sliding step length are both smaller than the side length of the face image;
the intercepting module is used for intercepting the face image according to the window size and the sliding step length to obtain a second virtual sample;
the processing module is used for processing the face image by a bit plane method to obtain a third virtual sample;
the second transformation module is used for respectively carrying out horizontal mirror image transformation on the second virtual sample and the third virtual sample to obtain a corresponding fourth virtual sample and a corresponding fifth virtual sample;
the non-negative matrix decomposition and reconstruction module is used for respectively carrying out non-negative matrix decomposition and reconstruction on the face image, the first virtual sample, the second virtual sample and the third virtual sample to obtain a corresponding sixth virtual sample, a corresponding seventh virtual sample, a corresponding eighth virtual sample and a corresponding ninth virtual sample;
the determining module is used for determining a virtual sample set of the face image; the set of virtual samples of the face image includes the face image, the first virtual sample, the second virtual sample, the third virtual sample, the fourth virtual sample, the fifth virtual sample, the sixth virtual sample, the seventh virtual sample, the eighth virtual sample, and a ninth virtual sample.
6. The system for expanding virtual samples of facial images according to claim 5, wherein the intercepting module specifically comprises:
the calculating unit is used for calculating the sliding times according to the window size, the sliding step length and the size of the face image;
and the intercepting unit is used for intercepting the face image according to the sliding times and the sliding step length to obtain a second virtual sample.
7. The system for virtual sample augmentation of human face images according to claim 5, wherein the processing module specifically comprises:
the bit plane image acquisition unit is used for acquiring a bit plane image of the face image; the bit plane diagram comprises a lower bit plane diagram, a middle bit plane diagram and an upper bit plane diagram;
the selecting unit is used for selecting the bit plane diagram according to the bit height threshold value to obtain the selected bit plane diagram;
and the combination unit is used for combining the selected bit plane maps to obtain a third virtual sample.
8. The system for expanding virtual samples of facial images according to claim 5, wherein the non-negative matrix factorization reconstruction module specifically comprises:
the gray value acquisition unit is used for acquiring the gray value of the face image;
the non-negative matrix determining unit is used for determining a non-negative matrix according to the gray value;
a first matrix and second matrix determining unit for determining a first matrix and a second matrix according to the non-negative matrix; obtaining the product of the first matrix and the second matrix as the non-negative matrix;
and the reconstruction unit is used for reconstructing the face image according to the first matrix and the second matrix to obtain a sixth virtual sample.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810675044.2A CN108898547B (en) | 2018-06-27 | 2018-06-27 | Single-sample-based face image virtual sample expansion method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810675044.2A CN108898547B (en) | 2018-06-27 | 2018-06-27 | Single-sample-based face image virtual sample expansion method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108898547A CN108898547A (en) | 2018-11-27 |
| CN108898547B true CN108898547B (en) | 2022-06-07 |
Family
ID=64346221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810675044.2A Active CN108898547B (en) | 2018-06-27 | 2018-06-27 | Single-sample-based face image virtual sample expansion method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108898547B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784207B (en) * | 2018-12-26 | 2020-11-24 | 深圳云天励飞技术有限公司 | Face recognition method, device and medium |
| CN110503146B (en) * | 2019-08-21 | 2021-12-14 | 杭州比智科技有限公司 | Data enhancement method and device, computing equipment and computer storage medium |
| TW202219895A (en) | 2020-11-09 | 2022-05-16 | 財團法人工業技術研究院 | Recognition system and image augmentation and training method thereof |
| CN112836433B (en) * | 2021-02-18 | 2023-03-14 | 南昌航空大学 | Construction method and size identification method of high-temperature alloy grain size identification model |
| CN113222889B (en) * | 2021-03-30 | 2024-03-12 | 大连智慧渔业科技有限公司 | Industrial aquaculture counting method and device for aquaculture under high-resolution image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700076A (en) * | 2015-02-13 | 2015-06-10 | 电子科技大学 | Face image virtual sample generating method |
| CN105426836A (en) * | 2015-11-17 | 2016-03-23 | 上海师范大学 | Single-sample face recognition method based on segmented model and sparse component analysis |
| CN107239741A (en) * | 2017-05-10 | 2017-10-10 | 杭州电子科技大学 | A kind of single sample face recognition method based on sparse reconstruct |
-
2018
- 2018-06-27 CN CN201810675044.2A patent/CN108898547B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700076A (en) * | 2015-02-13 | 2015-06-10 | 电子科技大学 | Face image virtual sample generating method |
| CN105426836A (en) * | 2015-11-17 | 2016-03-23 | 上海师范大学 | Single-sample face recognition method based on segmented model and sparse component analysis |
| CN107239741A (en) * | 2017-05-10 | 2017-10-10 | 杭州电子科技大学 | A kind of single sample face recognition method based on sparse reconstruct |
Non-Patent Citations (2)
| Title |
|---|
| Algorithm for Single Sample Face Recognition Based on Sample Augments and Double Subspace Decision Fusion;Yang Jun等;《Journal of Data Acquisition & Processing》;20150131;1-17 * |
| 基于单样本的人脸识别算法研究;吴凡;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20150115(第1期);I138-1232 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108898547A (en) | 2018-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108898547B (en) | Single-sample-based face image virtual sample expansion method and system | |
| US10891511B1 (en) | Human hairstyle generation method based on multi-feature retrieval and deformation | |
| Huang et al. | Gait recognition with shifted energy image and structural feature extraction | |
| Luu et al. | Contourlet appearance model for facial age estimation | |
| CN106326871B (en) | A kind of robust human face recognition methods decomposed based on dictionary with rarefaction representation | |
| Du et al. | Face aging simulation and recognition based on NMF algorithm with sparseness constraints | |
| Chen et al. | Face age estimation using model selection | |
| US20080014563A1 (en) | Method for Recognising Faces by Means of a Two-Dimensional Linear Disriminant Analysis | |
| CN103413117B (en) | A kind of incremental learning face identification method keeping Non-negative Matrix Factorization based on local | |
| Koniusz et al. | Spatial coordinate coding to reduce histogram representations, dominant angle and colour pyramid match | |
| CN103279936A (en) | Human face fake photo automatic combining and modifying method based on portrayal | |
| CN105389343B (en) | A kind of vectorization dimension reduction method | |
| CN108256449B (en) | Human behavior identification method based on subspace classifier | |
| CN109993199B (en) | Processing method for high-order tensor data | |
| Hou et al. | Disentangled representation for age-invariant face recognition: A mutual information minimization perspective | |
| CN112200147A (en) | Face recognition method and device, computer equipment and storage medium | |
| CN102289679B (en) | Method for identifying super-resolution of face in fixed visual angle based on related characteristics and nonlinear mapping | |
| Chen et al. | Nuclear norm based two-dimensional sparse principal component analysis | |
| Sun et al. | A genetic algorithm based feature selection approach for 3D face recognition | |
| Hai-Long et al. | Combining wavelet transform and orthogonal centroid algorithm for ear recognition | |
| Dwivedi et al. | Color image compression using 2-dimensional principal component analysis (2DPCA) | |
| CN114495239A (en) | Forged image detection method and system based on frequency domain information and generation countermeasure network | |
| Kose et al. | Block based face recognition approach robust to nose alterations | |
| Onifade et al. | A Model of Correlated Ageing Pattern for Age Ranking | |
| Wang et al. | Gender classification using selected independent-features based on genetic algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |