CN105354558A - Face image matching method - Google Patents

Abstract

The face image matching method of the present invention relates to image data processing. It is a face image matching method based on two SURF and shape contexts. The SURF algorithm is used for rough matching to obtain scale difference and direction difference information, and then the information is used for precise matching by the SURF algorithm. , use the shape context algorithm to remove the false match for the obtained matching result, the steps are: determine the face area; generate the reconstructed integral image; match the SURF feature twice; generate the shape context descriptor, remove the false match, and complete the face image matching . The method of the invention overcomes the defects of few feature points, few matching points and low correct rate in the existing face image matching method.

Description

Face Image Matching Method

technical field

The technical solution of the invention relates to image data processing, specifically a face image matching method.

Background technique

Face image matching is an important branch in the field of image matching. With the advent of the automated information age, face image matching has more and more applications in real life. Because face information is unique, difficult to forge and easy to collect, it is widely used in access control systems, video surveillance and identity verification technologies.

Most of the existing face image matching algorithms are based on extracting the local features of the face, and use the local features of the face for matching. Principal component analysis (hereinafter referred to as PCA method) is the most commonly used face local feature extraction method. In 1991, Turk and others at the University of California used the PCA method to propose a classic "eigenface" face image matching algorithm, and achieved good results. However, the PCA method only considers the second-order statistical information of the image data, fails to utilize the high-order statistical information in the data, and ignores the nonlinear correlation between multiple pixels. In 2004, David Lowe proposed the scale-invariant feature transformation algorithm (hereinafter referred to as the SIFT algorithm), which identifies potential scale and rotation-invariant key points through the Gaussian differential function, and determines the position and scale through a fine-fitting model. The local gradient direction of the image assigns one or more directions to each key point position, which realizes the invariance of scale and rotation. In the neighborhood of each key point, the local gradient of a certain scale image has a SIFT Indicates that large local deformation and illumination changes can be tolerated. But the SIFT algorithm is inefficient and slow. In 2006, Bay et al. in Switzerland proposed the SpeededUpRobustFeature algorithm (hereinafter referred to as the SURF algorithm) to improve the SIFT algorithm, using the maximum value of the Hessian matrix determinant to detect feature points, and simplifying the DOH (that is, the Hessian matrix) with the method of integral image box filtering. The calculation of the convolution in the determinant value) greatly improves the efficiency of the algorithm. However, the SURF algorithm still has the problem that few feature points are detected and the number of matching pairs obtained is also small.

Due to the use of PCA method, SIFT algorithm or SURF algorithm for feature extraction, the existing face image matching methods have the defects of few feature points, few matching points and low accuracy rate, especially for people with posture, expression and illumination changes. Face images, the number of feature points and the accuracy rate need to be improved. Therefore, it is of great significance to study face image matching methods with many feature points and high accuracy.

Contents of the invention

The technical problem to be solved by the present invention is to provide a face image matching method, which is a face image matching method based on twice SURF algorithm and shape context, referred to as TSURF+SC (TwiceSURF+ShapeContext), and uses the SURF algorithm to roughly match to obtain the scale difference and direction difference information, and then use the information to perform accurate matching with the SURF algorithm, and use the shape context algorithm to remove the wrong match for the matching result, which overcomes the existing face image matching methods that have few feature points, few matching points and high accuracy. Not high flaws.

The technical solution adopted by the present invention to solve the technical problem is: the face image matching method is a face image matching method based on twice SURF and shape context, and the specific steps are as follows:

The first step is to determine the face area:

Input two face images of the same size of the same person, one of the images with expression, posture and illumination changes is used as the image to be matched, and the other standard frontal face image is used as the template image, and the image to be matched and the template image are used Both are scaled to 1/8 of the template image, and then a face detection search box with a size of 20×20 pixels is used to scan the above two images from left to right and from top to bottom, and each scanned sub-image , Use the frontal face detector that comes with OpenCV to determine whether it is a face image, if so, mark it as a face area, scan the above two images each time, enlarge the face detection search box by 10%, and then start again Scan once and repeat until the face detection search box expands to half the size of the above image and stop scanning, then convert all marked face sub-images from RGB to YCrCb color space, and for each pixel in it Cr, The Cb component is used for skin color verification, and the skin color condition used for verification is shown in formula (1),

133≤Cr≤173∩77≤Cb≤127(1),

In the formula, Cr and Cb respectively represent the hue and saturation of the image in the YCrCb color space,

In the above-mentioned scanned image, the area where more than 40% of the pixels satisfy the formula (1) is determined as the face area, that is, the area of interest, and then the face area determined in the image to be matched and the template image is enlarged by 8 times, and restored to original size;

In the second step, the reconstructed integral image is generated:

Convert the face area determined in the first step above to RGB space, and then use formula (2) to convert it into a grayscale image,

X=0.299R+0.587G+0.114B (2),

In the above formula, R, G and B are the red, green and blue channels of the RGB space, respectively, and X represents the gray value in the gray image;

Then calculate the saliency factor of each pixel in the grayscale image to obtain the saliency factor map, as shown in formula (3),

σ(X _c )=magn×arctan(V/X _c ) (3),

In the above formula, magn is the magnification factor, σ(X _c ) is the salience factor of the pixel in the face image, V is the pixel X _c in the face image and the eight neighbors X _{i (} i =0,...,7), the calculation method is shown in formula (4),

$\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; 7</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

The sum of the pixel values of all points in the rectangular area formed by the origin of the upper left corner of the significant factor map and this point is used as the pixel value of the point on the integral map to generate a reconstructed integral image, as shown in formula (5),

$\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

In the above formula, IN(X _c ) is the pixel value at X _c in the reconstructed integral image, the coordinates of X _c are (x, y), and the value of IN(X _c ) is equal to (0,0) in the significant factor map The sum of all pixel values in the rectangular area formed by point, (x,0) point, (0,y) point and (x,y) point;

The third step, two SURF feature matching:

The matching process first detects SURF features and generates descriptors, then performs a rough match to obtain scale difference and direction difference information, and finally uses this information to perform an accurate match. The specific steps are as follows:

(1) Generate SURF descriptor:

Use box filter templates of different sizes and the reconstructed integral image obtained in the second step above to obtain the Hessian matrix determinant response images of different scales of the saliency factor map, and then use 3D non-maximum value suppression on these response images for detection , determine the point with the maximum response value as the feature point, the scale of the feature point is the scale of the corresponding response image, set the size of the box filter template as L×L pixels, the initial size L=9 pixels, and the corresponding response image scale s=1.2, then use L=15 pixels, 21 pixels, and 27 pixels in turn, and the corresponding response image scale s can be calculated by formula (6),

s=1.2×L/9(6),

After obtaining the position and scale s of the feature points, for each feature point, in a circular area with the feature point as the center and a radius of 6s, the Haar wavelet template with a size of 4s×4s pixels is used to perform a response operation on the significant factor map. Here s needs to be rounded to an integer, and then use a fan-shaped sliding window with a feature point as the center and an opening angle of π/3 to rotate around the feature point with a step size of 0.2 radians. Every time it turns, the image Haar in the sliding window is counted. The accumulative value ∑dx+∑dy of the wavelet horizontal and vertical response values dx and dy, the direction with the largest response accumulative value is taken as the main direction of the feature point. The pixel-sized image is divided into 4×4 sub-blocks, and each sub-block uses a Haar template with a size of 2s×2s pixels to calculate the response value, and the cumulative sum and absolute value of the response values in the horizontal direction x and vertical direction y are respectively counted. Value accumulation and ∑dx, ∑|dx|, ∑dy, ∑|dy| form a feature vector, that is, a SURF descriptor. Each feature point generates a total of 4×4×4=64-dimensional SURF descriptors;

(2) SURF algorithm rough matching:

For each feature point in the matching image, calculate the Euclidean distance between it and all feature point SURF descriptors in the template image, record the nearest neighbor distance d1 and the second nearest neighbor distance d2, when d1/d2<th1 and d1<th2 , the point and its nearest neighbor are recorded as a pair of matching points and stored in the initial matching set, thus completing the rough matching of the SURF algorithm. The above th1 and th2 are preset thresholds;

(3) SURF algorithm exact match:

Calculate and count the scale difference and angle difference between the matching point pairs in the initial matching set obtained in step (2) above. The scale mentioned here is the scale of the feature point, and the angle is the main direction of the feature point, and then calculate the scale of all matching point pairs The mean value ds and standard deviation dc of the difference, calculate the mean value dO of the angle difference, clear the initial matching set, and calculate the scale difference tds and angle difference tdO between it and all the feature points in the template image for each feature point in the image to be matched. The scale difference and angle difference between each part of the matching image and the template image should be consistent, and the scale difference and angle difference between the matching pairs should fall within a certain range. If tds and tdO satisfy the conditions of formula (7),

(ds-1.5dc)<tds<(ds+1.5dc)∩(dO-π/6)<tdO<(dO+π/6)(7),

Then calculate the Euclidean distance between the two matching points, redo the above (2) rough matching step of the SURF algorithm, and store the obtained matching points in the matching set. If the formula (7) is not satisfied, skip the pair of matching points , no longer calculate their Euclidean distance;

The fourth step is to remove the mismatch:

Use the shape context algorithm to eliminate the mismatching results obtained in the third step above. The specific method is as follows:

(1) Generate shape context descriptor:

Use the feature points in the matching set obtained in the third step above that belong to the image to be matched and the template image as the sampling points of the two images, and calculate and record the distance from all other sampling points in the image to it, and The angle it forms is then divided into 6 intervals after normalizing the distance, and the angle [0,2π] is divided into 12 intervals. These 6 distance intervals and 12 angle intervals form 72 blocks. By counting the number of sampling points falling in each block, a 72-dimensional shape context descriptor is generated;

(2) Eliminate false matches and complete face image matching:

Calculate the Euclidean distance d _sc of the shape context descriptor between each pair of matching points in the matching set obtained in the third step, find the mean value w and standard deviation f of the Euclidean distance, and then treat the matching pairs sc that do not satisfy the formula (8) as Eliminate false matches, and the remaining matching set is the final matching set.

d _sc ≤ w+f(8),

At this point, the face image matching is completed.

In the face image matching method described above, the magnification factor magn=10 in the formula (3).

In the face image matching method described above, in the rough matching of the SURF algorithm, the preset thresholds are: th1=0.6, th2=0.3.

In the face image matching method above, in the precise matching of the SURF algorithm, th1=0.7 and th2=0.4 are taken when redoing the rough matching step of the SURF algorithm in the above (2) step.

The beneficial effects of the present invention are: compared with the prior art, the outstanding substantive features and remarkable progress of the present invention are as follows:

(1) The method of the present invention is a face image matching method based on SURF algorithm and shape context, utilizes SURF algorithm rough matching to obtain scale difference and direction difference information, then utilizes these information to carry out SURF algorithm accurate matching, uses shape to the matching result obtained The context algorithm removes false matches, and overcomes the defects of few feature points, few matching points and low accuracy in the existing face image matching methods.

(2) The method of the present invention sets the face area as the area of interest, and only matching the face area can save a lot of time and improve the matching efficiency.

(3) The method of the present invention carries out reconstruction transformation to image when generating integral image, makes eyes, eyebrows, nose, mouth contain the position of a large amount of facial features more prominent, has more feature points, thereby increased effective feature point quantity .

(4) The method of the present invention uses the scale difference and direction difference information obtained by rough matching to perform precise matching, so that more matching points can be obtained, and the accuracy rate is improved.

(5) The method of the present invention uses the shape context algorithm to perform an operation of eliminating false matches on the results obtained by exact matching, thereby further improving the accuracy rate.

The following examples further demonstrate the outstanding substantive features and remarkable progress of the present invention.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Fig. 1 is a schematic block diagram of the process of the method of the present invention.

Fig. 2 is the curves of the number of matching pairs between the TSURF+SC method of the present invention and the existing face image matching method under different thresholds.

Fig. 3 is the accuracy rate curve of the TSURF+SC method of the present invention and the existing face image matching method under different thresholds.

detailed description

The embodiment shown in FIG. 1 shows that the flow of the method of the present invention is: determine the face area → generate a reconstructed integral image → match SURF features twice → generate a shape context descriptor, remove mismatches, and complete face image matching.

The embodiment shown in Fig. 2 shows that when different thresholds th1 and th2=0.4, the matching pair number curves of the TSURF+SC method of the present invention method and the existing face image matching method under different thresholds, the method of the present invention is shown in the figure The TSURF+SC method has the largest number of correct matching pairs, indicating that its algorithm effect is better than the existing face image matching methods.

The embodiment shown in Fig. 3 shows that when different thresholds th1 and th2=0.4, the accuracy rate curves of the inventive method TSURF+SC method and the existing face image matching method under different thresholds, the figure shows the inventive method TSURF The +SC method has the highest correct matching rate, indicating that its algorithm is better than the existing face image matching methods.

Example

The face image matching method of the present embodiment is a face image matching method based on twice SURF and shape context, and the specific steps are as follows:

The first step is to determine the face area:

Input two face images of the same size of the same person, one of the images with expression, posture and illumination changes is used as the image to be matched, and the other standard frontal face image is used as the template image, and the image to be matched and the template image are used Both are scaled to 1/8 of the template image, and then a face detection search box with a size of 20×20 pixels is used to scan the above two images from left to right and from top to bottom, and each scanned sub-image , Use the frontal face detector that comes with OpenCV to determine whether it is a face image, if so, mark it as a face area, scan the above two images each time, enlarge the face detection search box by 10%, and then start again Scan once and repeat until the face detection search box expands to half the size of the above image and stop scanning, then convert all marked face sub-images from RGB to YCrCb color space, and for each pixel in it Cr, The Cb component is used for skin color verification, and the skin color condition used for verification is shown in formula (1),

133≤Cr≤173∩77≤Cb≤127(1),

In the formula, Cr and Cb respectively represent the hue and saturation of the image in the YCrCb color space,

In the above-mentioned scanned image, the area where more than 40% of the pixels satisfy the formula (1) is determined as the face area, that is, the area of interest, and then the face area determined in the image to be matched and the template image is enlarged by 8 times, and restored to original size;

In the second step, the reconstructed integral image is generated:

Convert the face area determined in the first step above to RGB space, and then use formula (2) to convert it into a grayscale image,

X=0.299R+0.587G+0.114B (2),

In the above formula, R, G and B are the red, green and blue channels of the RGB space, respectively, and X represents the gray value in the gray image;

Then calculate the saliency factor of each pixel in the grayscale image to obtain the saliency factor map, as shown in formula (3),

σ(X _c )=magn×arctan(V/X _c ) (3),

In the above formula, magn is the magnification factor, magnification factor magn=10, σ(X _c ) is the salience factor of the pixel in the face image, V is the pixel point X _c in the face image and the eighth centered on the pixel point X _c The gray level difference of the neighborhood Xi ( _i =0,...,7), the calculation method is shown in the formula (4),

$\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; 7</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

The sum of the pixel values of all points in the rectangular area formed by the origin of the upper left corner of the significant factor map and this point is used as the pixel value of the point on the integral map to generate a reconstructed integral image, as shown in formula (5),

$\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

In the above formula, IN(X _c ) is the pixel value at X _c in the reconstructed integral image, the coordinates of X _c are (x, y), and the value of IN(X _c ) is equal to (0,0) in the significant factor map The sum of all pixel values in the rectangular area formed by point, (x,0) point, (0,y) point and (x,y) point;

The third step, two SURF feature matching:

The matching process first detects SURF features and generates descriptors, then performs a rough match to obtain scale difference and direction difference information, and finally uses this information to perform an accurate match. The specific steps are as follows:

(1) Generate SURF descriptor:

Use box filter templates of different sizes and the reconstructed integral image obtained in the second step above to obtain the Hessian matrix determinant response images of different scales of the saliency factor map, and then use 3D non-maximum value suppression on these response images for detection , determine the point with the maximum response value as the feature point, the scale of the feature point is the scale of the corresponding response image, set the size of the box filter template as L×L pixels, the initial size L=9 pixels, and the corresponding response image scale s=1.2, then use L=15 pixels, 21 pixels, and 27 pixels in turn, and the corresponding response image scale s can be calculated by formula (6),

s=1.2×L/9(6),

After obtaining the position and scale s of the feature points, for each feature point, in a circular area with the feature point as the center and a radius of 6s, the Haar wavelet template with a size of 4s×4s pixels is used to perform a response operation on the significant factor map. Here s needs to be rounded to an integer, and then use a fan-shaped sliding window with a feature point as the center and an opening angle of π/3 to rotate around the feature point with a step size of 0.2 radians. Every time it turns, the image Haar in the sliding window is counted. The cumulative value of wavelet horizontal square line and vertical direction response value dx, dy ∑dx+∑dy, the direction with the maximum response cumulative value is taken as the main direction of the feature point, after the main direction is obtained, take the feature point as the center and move 20s× The 20s pixel-sized image is divided into 4×4 sub-blocks, and each sub-block uses a Haar template with a size of 2s×2s pixels to calculate the response value, and the cumulative sum and sum of the response values in the horizontal direction x and vertical direction y are respectively counted Absolute value accumulation sums ∑dx, ∑|dx|, ∑dy, ∑|dy| to form a feature vector, that is, a SURF descriptor. Each feature point generates a total of 4×4×4=64-dimensional SURF descriptors;

(2) SURF algorithm rough matching:

For each feature point in the matching image, calculate the Euclidean distance between it and all feature point SURF descriptors in the template image, record the nearest neighbor distance d1 and the second nearest neighbor distance d2, when d1/d2<th1 and d1<th2 , record the point and its nearest neighbor as a pair of matching points, and store them in the initial matching set, thus completing the rough matching of the SURF algorithm. The above-mentioned th1 and th2 are preset thresholds, th1=0.6, th2=0.3;

(3) SURF algorithm exact match:

Calculate and count the scale difference and angle difference between the matching point pairs in the initial matching set obtained in step (2) above. The scale mentioned here is the scale of the feature point, and the angle is the main direction of the feature point, and then calculate the scale of all matching point pairs The mean value ds and standard deviation dc of the difference, calculate the mean value dO of the angle difference, clear the initial matching set, and calculate the scale difference tds and angle difference tdO between it and all the feature points in the template image for each feature point in the image to be matched. The scale difference and angle difference between each part of the matching image and the template image should be consistent, and the scale difference and angle difference between the matching pairs should fall within a certain range. If tds and tdO satisfy the conditions of formula (7),

(ds-1.5dc)<tds<(ds+1.5dc)∩(dO-π/6)<tdO<(dO+π/6)(7),

Then calculate the Euclidean distance between the two matching points, redo the rough matching step of the above (2) step SURF algorithm, wherein th1=0.7, th2=0.4, the matching point obtained is stored in the matching set, if the formula (7 ), then skip the pair of matching points and no longer calculate their Euclidean distance;

The fourth step is to remove the mismatch:

Use the shape context algorithm to eliminate the mismatching results obtained in the third step above. The specific method is as follows:

(1) Generate shape context descriptor:

Use the feature points in the matching set obtained in the third step above that belong to the image to be matched and the template image as the sampling points of the two images, and calculate and record the distance from all other sampling points in the image to it, and The angle it forms is then divided into 6 intervals after normalizing the distance, and the angle [0,2π] is divided into 12 intervals. These 6 distance intervals and 12 angle intervals form 72 blocks. By counting the number of sampling points falling in each block, a 72-dimensional shape context descriptor is generated;

(2) Eliminate false matches and complete face image matching:

Calculate the Euclidean distance d _sc of the shape context descriptor between each pair of matching points in the matching set obtained in the third step, find the mean value w and standard deviation f of the Euclidean distance, and then treat the matching pairs sc that do not satisfy the formula (8) as Eliminate false matches, and the remaining matching set is the final matching set.

d _sc ≤ w+f(8),

At this point, the face image matching is completed.

This embodiment is implemented using VS2005 and OPENCV2.0 platforms, and the matching experiments are carried out on images with expression changes, posture changes, illumination changes, and simultaneous changes in expression and posture in the IMM of the Technical University of Denmark and the GeorgiaTech face database. Among them, the IMM database contains 40 people, 6 images per person, and the image size is 640×480. The GeorgiaTech face database contains 50 people, 15 images per person, and the image size is 640×480. The processor used in the experiment is Intel Core I3, 4G memory. In order to verify the advantages of the method of the present invention in terms of the number of feature points and the accuracy rate, this embodiment selects the classic matching algorithm SURF algorithm, the SURF algorithm combined with the RANSAC algorithm, the improved SURF algorithm combined with the RANSAC algorithm and this method TSURF+SC for comparison, and records each The number of matching points and the number of wrong matches obtained by a group of experimental pictures at different thresholds th1 and th2 respectively, the experimental data shows that the method of this embodiment exceeds the comparison algorithm by more than 80% on average in the number of matching points, and the number of wrong matches accounts for the number of matching points The objective ratios are also lower than the comparison algorithm, but the running time of the algorithm is 80% higher than the original SURF algorithm, and the algorithm efficiency needs to be improved in future research. Table 1 lists the matching data of face images with posture changes in a group of IMM database images at empirical thresholds th1=0.7, th2=0.4, and Table 2 shows a group of face images with illumination changes in the GeorgiaTech database. Matching data when empirical thresholds th1=0.7, th2=0.4.

Table 1 Attitude change matching effect

Table 2 Lighting change matching effect

TSURF+SC in the table is the method of the present invention adopted in this embodiment.

The results show that the number of matching pairs and correct rate obtained by the method of this embodiment are better than those of the three comparison algorithms. Although the matching time is higher than the previous two algorithms, it provides more correct matches than the previous two algorithms, and the algorithm efficiency needs to be within Continue to improve in future research.

Claims (4)

1. Humanface image matching method, is characterized in that: be the Humanface image matching method based on twice SURF and Shape context, concrete steps are as follows:

The first step, determine human face region:

Input the facial image of the formed objects of two width same persons, to a wherein width espressiove, the image of attitude and illumination variation is as image to be matched, another width standard front face facial image is as template image, image to be matched and template image are all carried out zooming to 1/8 of template image, then the Face datection search box of 20 × 20 pixel sizes is used to scan respectively above-mentioned two width images from top to bottom from left to right, to the subimage that each scans, the obverse face detection device carried with OpenCV judges it whether as facial image, if, then be labeled as human face region, one time has been scanned respectively to above-mentioned two width images at every turn, Face datection search box is amplified 10%, rescan one time again, repetition like this, until stop scanning when Face datection search box expands the half size of above-mentioned image to, then from RGB, YCrCb color space is transformed into all face subimages be labeled, to the Cr of each pixel wherein, Cb component carries out colour of skin checking, verify that colour of skin condition used is as shown in formula (1),

133≤Cr≤173∩77≤Cb≤127(1)，

In formula, Cr, Cb represent tone and the saturation degree of image in YCrCb color space respectively,

The region that the pixel in the image of above-mentioned scanning with more than 40% meets formula (1) is defined as human face region, i.e. area-of-interest, again the human face region determined in image to be matched and template image is amplified 8 times, return to original size size;

Second step, generates the integral image of reconstruct:

Again rgb space is transformed into the human face region determined in the above-mentioned first step, then utilizes formula (2) to be converted to gray level image,

X＝0.299R+0.587G+0.114B(2)，

In above formula, R, G and B are the redness of rgb space, green and blue channel respectively, and X represents the gray-scale value in gray level image;

Then calculate the significant factor of each pixel in gray level image, obtain significant factor figure, as shown in formula (3),

σ(X _c)＝magn×arctan(V/X _c)(3)，

In above formula, magn is amplification coefficient, σ (X _c) be the significant factor of the pixel in facial image, V is pixel X in facial image _cwith with pixel X _ccentered by eight neighborhood X _i(i=0 ..., 7) grey scale difference, computing method as shown in formula (4),

$V={\mathrm{\&Sigma;}}_{i=0}^7(X_i-X_c)---\left(4\right),$

In the rectangular area that significant factor figure upper left corner initial point and this point are formed the pixel value of pixel value sum this point on integrogram a little, generate the integral image of reconstruct, as shown in formula (5),

$IN\left(X_c\right)={\mathrm{\&Sigma;}}_{i=0}^{i\&le;x}{\mathrm{\&Sigma;}}_{j=0}^{j\&le;y}\mathrm{\&sigma;}\left(X_c\right)(i,j)---\left(5\right),$

IN (X in above formula _c) be X in the integral image of reconstruct _cthe pixel value at place, X _ccoordinate be (x, y), IN (X _c) value equal (0,0) point in significant factor figure, (x, 0) point, (0, y) put with (x, y) all pixel values in the rectangular area that forms with;

3rd step, twice SURF characteristic matching:

First matching process detects SURF feature and generates descriptor, and then carry out once thick coupling and obtain yardstick difference and direction difference information, finally recycle these information and carry out an exact matching, concrete steps are as follows:

(1) SURF descriptor is generated:

The integral image of the reconstruct obtained with box Filtering Template and the above-mentioned second step of different size asks for the Hessian matrix determinant response image of the different scale of significant factor figure, on these response images, adopt 3D non-maxima suppression to detect afterwards, the point with response maximum value is defined as unique point, the yardstick of this unique point is the yardstick of respective response image, if box Filtering Template size is L × L pixel, original dimension L=9 pixel, the response image yardstick s=1.2 of its correspondence, then L=15 pixel is used successively, 21 pixels, the size of 27 pixels, response image yardstick s corresponding respectively can be calculated by formula (6),

s＝1.2×L/9(6)，

After the position obtaining unique point and yardstick s, to each unique point, centered by unique point, 6s be radius border circular areas in response computing is carried out to the Haar small echo template that significant factor figure size is 4s × 4s pixel, here s need be rounded to integer, then one is used centered by unique point, subtended angle is the fan-shaped moving window of π/3, rotate around unique point with step-length 0.2 radian, often forward a place to, image Haar small echo horizontal direction and vertical direction response dx in statistics moving window, the accumulated value ∑ dx+ ∑ dy of dy, there is the principal direction of direction as unique point of peak response accumulated value, after obtaining principal direction, centered by unique point, along principal direction, the image of 20s × 20s pixel size is divided into 4 × 4 sub-blocks, each sub-block utilizes Haar template that size is 2s × 2s pixel to carry out the calculating of response, and its cumulative sum and absolute value cumulative sum ∑ dx are added up respectively to the response of horizontal direction x and vertical direction y, ∑ | dx|, ∑ dy, ∑ | dy|, morphogenesis characters vector, i.e. SURF descriptor, the SURF descriptor that each unique point symbiosis becomes 4 × 4 × 4=64 to tie up,

(2) SURF algorithm slightly mates:

Treat each unique point in matching image, calculate the Euclidean distance between all unique point SURF descriptors in it and template image, record nearest neighbor distance d1 and time nearest neighbor distance d2, as d1/d2<th1 and d1<th2 time, this point and its arest neighbors are recorded as a pair match point, stored in initial matching collection, complete SURF algorithm thus and slightly mate, above-mentioned th1 and th2 is the threshold value preset;

(3) SURF algorithm exact matching:

Calculate and add up initial matching that above-mentioned (2) step obtains and concentrate yardstick difference between matching double points and differential seat angle, yardstick mentioned here is the yardstick of unique point, angle is unique point principal direction, then average ds and the standard deviation dc of all matching double points yardstick differences is calculated, calculate the average dO of differential seat angle, initial matching collection is emptied, treat each unique point in matching image and calculate yardstick difference tds in it and template image between all unique points, differential seat angle tdO, yardstick difference between image to be matched and template image each several part and differential seat angle should be consistent, mate between yardstick difference and differential seat angle should fall within the specific limits, if tds and tdO meets formula (7) condition,

(ds-1.5dc)<tds<(ds+1.5dc)∩(dO-π/6)<tdO<(dO+π/6)(7)，

Then calculate the Euclidean distance between two match points, above-mentioned (2) step SURF algorithm of reforming slightly mates step, by the match point that obtains stored in set of matches, if do not meet formula (7), then skip this to match point, no longer calculate their Euclidean distance;

4th step, remove error hiding:

Utilize Shape context algorithm to carry out the rejecting of error hiding to the matching result that above-mentioned 3rd step obtains, concrete grammar is as follows:

(1) Shape context descriptor is generated:

The sampled point of unique point as two width images of image to be matched and template image will be belonged in the set of matches obtained in above-mentioned 3rd step, each sampled point is calculated and records other sampled point in all figure to its distance and its angulation, then 6 intervals will be divided into after range normalization, and by angle [0,2 π] be divided into 12 intervals, 72 blocks are constituted with 12 angular interval between these 6 distance regions, add up the sampled point number dropped in each block, just generate the Shape context descriptor of 72 dimensions;

(2) reject error hiding, complete facial image coupling:

Calculate the Euclidean distance d of Shape context descriptor between often pair of match point in the set of matches obtained in the 3rd step _sc, obtain Euclidean distance average w and standard deviation f, the coupling then sc not being met formula (8) weeds out being used as error hiding, and remaining set of matches is then final set of matches,

d _sc≤w+f(8)，

So far facial image coupling is completed.

2. Humanface image matching method according to claim 1, is characterized in that: the amplification coefficient magn=10 in described formula (3).

3. Humanface image matching method according to claim 1, is characterized in that: during described SURF algorithm slightly mates, the threshold value preset: th1=0.6, th2=0.3.

4. Humanface image matching method according to claim 1, is characterized in that: in described SURF algorithm exact matching, gets th1=0.7, th2=0.4 when above-mentioned (2) step SURF algorithm of reforming slightly mates step.