CN101477684A - Process for reconstructing human face image super-resolution by position image block - Google Patents

Abstract

The invention provides a method for reconstructing face image super-resolution by utilizing position image blocks. The method comprises the following steps: dividing a low-resolution face image and face images in high-low resolution training sets into mutually overlapped image blocks; calculating the optimal value of each divided image blocks input in the low-resolution face image during the linear restoration of the position block of each sample image in the low-resolution training set; replacing the position blocks of the sample images in the low-resolution training set by the position blocks of the sample images in the high-resolution training set, which correspond to each position block of the sample images in the low-resolution training set, and compositing image blocks of high-resolution in a weighting manner; and splicing the composited image blocks of high-resolution into a whole image according to the position of the image blocks in the face image. The method which reconstructs a high-resolution image block in the same position by utilizing the image block in the same position of each sample image in a training set directly has the advantage that the manifold learning step or the feature extraction step which are common in similar algorithms are avoided, thereby greatly saving operation time, reducing complexity; and the quality of the composited high-resolution image is improved.

Description

A kind of face image super-resolution method that utilizes the position image block to rebuild

Technical field

The present invention relates to a kind of image processing method, be specifically related to a kind of face image super-resolution method that utilizes the position image block to rebuild.

Background technology

Super-resolution is meant by a frame or multiframe low-resolution image and reconstructs a frame or multiframe high-definition picture.The super-resolution rebuilding algorithm that is applicable to various images is not directed to the algorithm of a certain class graphical design and rebuilds effective.

U.S. Ka Naiji-Baker (document 1:S.Baker of Mei Long university in 2000, T.Kanade, Hallucinating Faces, in:Proc.of Inter.Conf.on Automatic Face and GestureRecognit ion, Grenoble, France, 2000, pp.83-88.) at first develop single frames face image super-resolution algorithm (Face hallucination) based on machine learning, select the level of facial image gaussian pyramid and the derivative and the laplacian pyramid of vertical direction, as the feature space of facial image.Obtain mapping by off-line learning, this mapping has reflected the gradation of image corresponding relation of former figure under different resolution, and as based on the prior imformation of discerning, but still there is bigger noise in the high-resolution human face image that obtains at some position with this.Calendar year 2001 Liu (document 2:C.Liu, H.Shum, and C.Zhang, A Two-Step Approachto Hallucinating Faces:Global Parametric Model and Local NonparametricModel, in Proc.Of CRPR, Vol.1, pp.192-198,2001) developed one two step algorithm, at first under Gauss's hypothesis, utilize the global parameter model to obtain preliminary high-resolution human face, under the Markov field hypothesis, utilize local nonparametric model to obtain residual image then, at last the two addition is obtained net result.The method of Baker and Liu has not only been used complicated probability model and has been made that the computing quantitative change is big, and obtaining of net result needs the down-sampling function simultaneously, and the down-sampling function is difficult to obtain in practice.Chang (document 3:Hong Chang in 2004, etal.Super-resolution through neighbor embedding. //Proceedings of the IEEE Computer Society Conference on Computer Visionand Pattern Recognition, Washington, 2004.United States:Institute ofElectrical and Electronics Engineers Computer Society, 2004:1275-1282) at first neighborhood thought in manifold learning Locally Linear Embedding (LLE) algorithm is introduced super-resolution rebuilding, propose neighborhood and embed (Neighbor Embedding) algorithm, ask for the method for data neighborhood point according to LLE, choose the low-resolution image of neighborhood piece reconstruct input in the low resolution training set, and suppose that high low-resolution image has similar manifold structure, weights under the low-resolution spatial are applied to high resolution space, reconstruct high-definition picture.Scholar afterwards, when the method for using image block is carried out the super-resolution rebuilding of single width people face, nearly all used the neighborhood image piece, just nearly all embed on the algorithm basis at neighborhood, suppose that no longer high low-resolution image has similar manifold structure, and the mode that adopts iteration obtains reconstruct weights (the document 4:Sung Won Park under the high resolving power, Savvides M.Robust.Super-Resolutionof Face Images by Iterative Compensating Neighborhood Relationships. //Biometrics Symposium, 2007.United States:Digital Object Identifier, 2008:1-5), the result that neighborhood embedding algorithm is obtained compensates (document 5:Wei Liu again, Dahua Lin, Xiaoou Tang.Neighbor combination and transformation forhallucinating faces. //IEEEInternational Conference on Multimedia andExpo, Netherlands, 2005.United States:Institute of Electrical andElectronics Engineers Computer Society, 2005:478-484) or the like.Wang (document 6:Xiaogang Wang in 2005, et al.Hallucinating Face by Eigentransformation.IEEE Transactions on Systems Man and Cybernetics, 2005,35 (3): 425-434) a series of low-resolution images and corresponding high-definition picture are carried out principal component analysis (PCA) obtains people's face shape and texture model is tried to achieve the model coefficient for the treatment of the reconstructed image correspondence, thereby utilize these coefficients and Model Reconstruction facial image. because the weights that obtain are at general image, it is poor slightly to rebuild precision, only be that people's face area-of-interest image reconstruction quality is higher, all the other zones are fuzzyyer, and image carried out principal component analysis (PCA) earlier, just lost non-characteristic information, and these non-characteristic informations also are useful concerning super-resolution rebuilding.Zhuang (document 7:Yueting Zhuang in 2007, et al.Hallucinating faces:LPH super-resolutionand neighbor reconstruction for residue compensation.Pattern Recognition, 2007,40:3178-3194) utilize Locality Preserving Projection (LPP) to extract global feature earlier, the preliminary high-definition picture that obtains through radial basis function carries out image block neighborhood residual error at last and obtains last result again.The method of Zhuang has been introduced LPP in the first step, so just lost a part of non-characteristic information, and these characteristic informations are contributive to the detail recovery of image, thereby the not good enough compensation of high-definition picture effect of the first step, use neighborhood to embed algorithm at last and carry out residual compensation, the high-definition picture effect of Huo Deing depends on the residual compensation in second step at last, and calculated amount is bigger.

Summary of the invention

The object of the invention is to provide a face image super-resolution method that utilizes the position image block to rebuild, and it is big that method of the present invention solves existing similar algorithm operation quantity, the dissatisfactory problem of high-resolution human face picture quality of acquisition.

For achieving the above object, the technical solution used in the present invention is:

1) low resolution facial image, the high low resolution training set facial image with input is divided into overlapped image block respectively;

2), calculate by each sample image block of locations of low resolution training set and carry out the linear optimum weights of rebuilding for each low-resolution image piece in the low resolution facial image of input;

3), replace with high resolving power training set sample image block of locations one to one, use step 2 with low resolution training set sample image block of locations) weights obtained, the high-definition picture piece is synthesized in weighting;

4) with synthetic high-definition picture piece according to its position at facial image, be spliced into general image.

Overlapped image block partiting step of the present invention is as follows:

Low resolution facial image X with input _L, m facial image Y in the high low resolution training set _H, Y _LBe expressed as overlapped image block matrix form, be respectively

Wherein N is the number of institute's divided image square, and the m maximal value is M, and M is the right number of the high low-resolution image of training set, X ^P(i j) is positioned at the image block at the capable j row of i place in the presentation video block matrix, (i j) has embodied the position feature of institute's divided image piece at people's face, establishes each image block size and is n * n, and four adjacent image pieces of each image block are expressed as X ^P(i-1, j), X ^P(i+1, j), X ^P(i, j+1), X ^P(i, j-1), n is a positive integer, when n is odd number, each image block X _L ^P(i, j) the overlapping region size that is adjacent image block is (n-1)/2, then its corresponding high-definition picture piece X _H ^P(i, j) size is qn * qn, the overlapping region size that is adjacent image block is [q (n-1)/2] * [q (n-1)/2]; When n was even number, the overlapping region size was n/2, and then its corresponding high-definition picture block size is qn * qn, and the overlapping region size is [qn/2] * [qn/2], and q is extraction yield or enlargement factor.

Block of locations of the present invention is meant: for each low-resolution image piece in the low resolution facial image of input

Each sample image is at (i, j) image block of position in the training set

Perhaps with (i, j) 8 adjacent neighborhood image pieces of position all are called

Block of locations.

The present invention is as follows by the optimum weight calculation method of the low-resolution image piece of the linear reconstruction input of each sample image block of locations of low resolution training set:

Optimum weights computing formula is: $w(i,j)=\frac{Z^{-1}C}{C^T{Z}^{-1}C},$ Z=(X-Y) wherein ^T(X-Y), X=X _L ^P(i, j) C ^T, C is the unit column vector, and Y is the matrix of a M * D dimension, and its each row are by Y _L ^MP(i j) forms, and D is Y _L ^MP((i j) is each w to w for i, dimension j) _m(i, the M right-safeguarding value vector that j) is combined into often in actual calculation adopt a method more efficiently, promptly ask linear system equation Zw (i, j)=C.

The synthetic high-definition picture block method of weighting of the present invention is as follows:

Obtain the low-resolution image piece X of input _L ^P(i, j) Dui Ying high-definition picture piece X _H ^P(i, j) computing formula is: ${X_H}^P(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{Y_H}^{\mathrm{mP}}(i,j)w_m(i,j).$

The present invention is the super-resolution algorithms at facial image, promptly by the low resolution facial image of an input, reconstruct high-resolution facial image, avoided similar algorithm search step, and the image block that directly utilizes each sample image same position of training set is rebuild same position high-definition picture piece, thereby saves operation time and complexity greatly; Avoided manifold learning or characteristic extraction step common in the similar algorithm, further saved operation time and reduced complexity, and promoted the quality of the high-definition picture that is synthesized.

Description of drawings

Fig. 1 is a process flow diagram of the present invention.

Fig. 2 is the division methods of overlapping image block.

Fig. 3 is result's contrast of the present invention and neighborhood embedding grammar (Neighbor Embedding document [3]), and wherein K is a neighborhood piece number in the neighborhood embedding grammar; Wherein (a) input low-resolution image (32 * 24); (b) utilize during K=50 neighborhood to embed arithmetic result (128 * 96); (c) utilize during K=150 neighborhood to embed arithmetic result (128 * 96); (d) utilize during K=400 neighborhood to embed arithmetic result (128 * 96); (e) result of the present invention (128 * 96); (f) true picture (128 * 96);

Fig. 4 is result's contrast of the present invention and document 6,8 methods; (a) Shu Ru low-resolution image (32 * 24); (b) traditional interpolation oversubscription result (128 * 96); (c) document 6 arithmetic result (128 * 96); (d) document 8 arithmetic result (128 * 96); (e) result of the present invention (128 * 96); (f) true picture (128 * 96).

Embodiment

Below in conjunction with accompanying drawing the present invention is described in further detail.

Referring to Fig. 1, provide this invention technical scheme concrete steps below:

Step 1: with the low-resolution image X of input _L, m image Y in the high low resolution training set _H, Y _LBe expressed as overlapped image block matrix form, be respectively

Wherein N is the number of institute's divided image square, and m is M to the maximum, and M is right number, X of the high low-resolution image of training set ^P(i j) is positioned at the image block at the capable j row of i place in the presentation video block matrix.(i j) has embodied the position feature of institute's divided image piece at people's face, establishes each image block size and is n * n, and four adjacent image pieces of each image block are expressed as X ^P(i-1, j), X ^P(i+1, j), X ^P(i, j+1), X ^P(i, j-1), n is a positive integer, image block is divided with expression and is seen accompanying drawing 2.To the low-resolution image training set

, when n is odd number, each image block X _L ^P(i, j) the overlapping region size that is adjacent image block is (n-1)/2, then its corresponding high-definition picture piece X _H ^P(i, j) size is qn * qn, the overlapping region size that is adjacent image block is [q (n-1)/2] * [q (n-1)/2]; When n was even number, the overlapping region size was n/2, and then its corresponding high-definition picture block size is qn * qn, and the overlapping region size is [qn/2] * [qn/2], and q is extraction yield or enlargement factor.To different images X, work as i, when j got maximal value respectively separately, image block was positioned at the image boundary position, and its size might be less than other image block, and for these boundary image pieces, disposal route is the same.

(i j) has embodied the position feature of institute's divided image piece at people's face, and (i, j) identical image block is called the block of locations of this position.Described block of locations is meant: for each low-resolution image piece in the low resolution facial image of input

, each sample image is at (i, j) image block of position in the training set , perhaps with (i, j) 8 adjacent neighborhood image pieces of position all are called

Block of locations.

Step 2: the low-resolution image for the treatment of operation

Each image block X _L ^P(i, j) all carry out following operation (image and image block all adopt vector form to express):

Optimum weight w is calculated in operation 1. _m(i j), makes following two formulas set up

$X_L^P(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_m(i,j)\·Y_L^{\mathrm{mP}}(i,j)---\left(1\right)$

$\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_m(i,j)=1---\left(2\right)$

The computing method of optimum weights wherein:

For obtaining w _m(i, j), convolution (1), we set up as shown in the formula:

$\mathrm{\θ}=\underset{w_m(i,j)}{\arg \min }{\left|\right|X^P(i,j)-\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_m(i,j)Y^{\mathrm{mP}}(i,j)\left|\right|}^2$

Make Z=(X-Y) ^T(X-Y), X=X wherein ^P(i, j) C ^T, C is the unit column vector, and Y is the matrix of a M * D dimension, and its each row are by Y _L ^MP(i j) forms, and D is Y _L ^MP(i, dimension j).(i j) is each w to w _m(i, the M right-safeguarding value vector that j) is combined into use method of Lagrange multipliers to obtain

$w(i,j)=\frac{Z^{-1}C}{C^T{Z}^{-1}C}$

Often in actual calculation adopt a method more efficiently, promptly ask linear system equation zw (i, j)=C.Optimum weight w is used in operation 2. _m(i j), utilizes following formula to obtain X _L ^P(i, j) Dui Ying high-definition picture piece X _H ^P(i, j)

${X_H}^P(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{Y_H}^{\mathrm{mP}}(i,j)w_m(i,j)$

Step 3: N the high-definition picture piece X that previous step is obtained suddenly _H ^P(i j), splices according to its position in image array.Asking for of adjacent image piece institute lap pixel value, employing is got Mean Method and is got final product.

The general low resolution location drawing gets 3 * 3 as block size, and the adjacent image piece overlaps 1 pixel.

The present invention can be applicable to numerous areas, for example:

(1) video monitoring on bank, traffic, sub-district, customs, airport.

(2) video conference.

(3) police criminal detection is used for improving the captured suspect's in the site of the accident picture quality.

(4) handset image transmission.

(5) certificate identification is restored as the people's face on the certificates such as I.D., driver's license, passport.

It is big why the present invention can solve similar algorithm operation quantity, and the reason of the dissatisfactory problem of high-resolution human face picture quality of acquisition is:

1, avoided similar algorithm search step, promptly avoiding searching in training set the image block that satisfies a certain rule comes, and the image block that directly utilizes each sample image same position of training set is rebuild same position high-definition picture piece, thereby saves operation time and complexity greatly;

2, avoided introducing manifold learning or characteristic extraction step (as PCA, LLE or LPP etc.), further saved operation time and complexity and promoted the quality of the high-definition picture that is synthesized.Similar algorithm is usually introduced manifold learning or characteristic extraction step in order to obtain the relation between the high low resolution facial image, and the model parameter that obtains does not so comprise non-characteristic information, but not characteristic information is contributive to people's face to the detail recovery of image.Similar algorithm is owing to introduce manifold learning or characteristic extraction step, causes the high-definition picture quality that obtains not ideal enough, usually needs the residual compensation in second step further to promote picture quality, even but such effect still is not so good as the present invention.

Below be the experiment contrast.

Adopted the extensive face database of CAS-PEAL-R1 (document 8:Wen Gao, et al.The CAS-PEALLarge-Scale authority of office Chinese Face Database and Baseline Evaluations.IEEETransactions on System Man, and Cybernetics (Part A), 2008, (38): 149-161), (face is subjected to light even under same illumination condition to have selected 200 Different Individual at random for use, there is not " negative and positive face " situation), usual expression front face image, demarcate and two centers of having alignd by hand, three unique points in face center, intercept human face region as required, and with picture size unification to 128 * 96 sizes.Adopt average Downsapling method to reduce the low-resolution image that resolution obtains 32 * 24 sizes all sample images.

The high-definition picture that utilizes the present invention high-definition picture that obtains and the document [3] that representative row property is arranged, [6], [8] middle algorithm to obtain carries out the contrast of picture quality, operation time, sees Fig. 3, Fig. 4 respectively.

Table 1 is for contrasting the computing time of the present invention and document [3], [6], [8] method.

Table 1. uses same training set on the computing machine of 3.0G CPU, add the time that generates training set, and algorithms of different generates the used time contrast of a vertical frame dimension image in different resolution

Claims (5)

1, a kind of face image super-resolution method that utilizes the position image block to rebuild is characterized in that comprising the steps:

1) low resolution facial image, the high low resolution training set facial image with input is divided into overlapped image block respectively;

2), calculate by each sample image block of locations of low resolution training set and carry out the linear optimum weights of rebuilding for each low-resolution image piece in the low resolution facial image of input;

3), replace with high resolving power training set sample image block of locations one to one, use step 2 with low resolution training set sample image block of locations) weights obtained, the high-definition picture piece is synthesized in weighting;

4) with synthetic high-definition picture piece according to its position at facial image, be spliced into general image.

2, a kind of face image super-resolution method that utilizes the position image block to rebuild according to claim 1, it is characterized in that: said overlapped image block partiting step is as follows:

Low resolution facial image X with input _L, m facial image Y in the high low resolution training set _H, Y _LBe expressed as overlapped image block matrix form, be respectively ${\left\{{X_L}^P(i,j)\right\}}_{p=1}^N,$ ${\left\{{X_L}^{\mathrm{mP}}(i,j)\right\}}_{p=1}^N,$ ${\left\{{Y_H}^{\mathrm{mP}}(i,j)\right\}}_{p=1}^N,$ Wherein N is the number of institute's divided image square, and the m maximal value is M, and M is the right number of the high low-resolution image of training set, X ^P(i j) is positioned at the image block at the capable j row of i place in the presentation video block matrix, (i j) has embodied the position feature of institute's divided image piece at people's face, establishes each image block size and is n * n, and four adjacent image pieces of each image block are expressed as X ^P(i-1, j), X ^P(i+1, j), X ^P(i, j+1), X ^P(i, j-1), n is a positive integer, when n is odd number, each image block X _L ^P(i, j) the overlapping region size that is adjacent image block is (n-1)/2, then its corresponding high-definition picture piece X _H ^P(i, j) size is qn * qn, the overlapping region size that is adjacent image block is [q (n-1)/2] * [q (n-1)/2]; When n was even number, the overlapping region size was n/2, and then its corresponding high-definition picture block size is qn * qn, and the overlapping region size is [qn/2] * [qn/2], and q is extraction yield or enlargement factor.

3, a kind of face image super-resolution method that utilizes the position image block to rebuild according to claim 1 is characterized in that described block of locations is meant: for each low-resolution image piece in the low resolution facial image of input

Each sample image is at (i, j) image block of position in the training set Perhaps with (i, j) 8 adjacent neighborhood image pieces of position all are called

Block of locations.

4, a kind of face image super-resolution method that utilizes the position image block to rebuild according to claim 1 is characterized in that described optimum weight calculation method by the linear low-resolution image piece of rebuilding input of each sample image block of locations of low resolution training set is as follows:

Optimum weights computing formula is: $w(i,j)=\frac{Z^{-1}C}{C^T{Z}^{-1}C},$ Z=(X-Y) wherein ^T(X-Y), X=X _L ^P(i, j) C ^T, C is the unit column vector, and Y is the matrix of a M * D dimension, and its each row are by Y _L ^MP(i j) forms, and D is Y _L ^MP((i j) is each w to w for i, dimension j) _m(i, the M right-safeguarding value vector that j) is combined into often in actual calculation adopt a method more efficiently, promptly ask linear system equation Zw (i, j)=C.

5, a kind of face image super-resolution method that utilizes the position image block to rebuild according to claim 1 is characterized in that the synthetic high-definition picture block method of described weighting is as follows:

Obtain the low-resolution image piece X of input _L ^P(i, j) Dui Ying high-definition picture piece X _H ^P(i, j) computing formula is: ${X_H}^P(i,j)=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{Y_H}^{\mathrm{mP}}(i,j)w_m(i,j).$

Images