CN101216889A - A face image super-resolution method based on fusion of global features and local details - Google Patents

Abstract

The invention discloses a face mage super-resolution method which fuses global features and local detail information. The invention can synthesize a high-resolution face image according to a low-resolution face image based on a sample image. Firstly, a local maintaining mapping algorithm and a radial basic function return algorithm are combined together to get a global high-resolution face image; then a neighborhood reconstruction method is adopted to synthesize a high-resolution face residual image block and consequently form a high-resolution face residual image by combination; finally, the high-resolution face residual image is overlapped to the high-resolution face image to obtain a final super-resolution effect. The technology provided by the invention can synthesize the clearer high-resolution face image, improve the recognition of the face image and have important application significances on video monitoring, face recognition and other aspects.

Description

A face image super-resolution method based on fusion of global features and local details

technical field

The invention relates to digital image processing, in particular to a face image super-resolution method for fusing global features and local detail information.

Background technique

Face super-resolution technology is a special kind of image super-resolution technology. The current image super-resolution technology can be roughly divided into two categories, namely, image super-resolution based on reconstruction and image super-resolution based on learning. The latter is more effective than the former. In recent years, some representative learning-based image super-resolution techniques have emerged. The main idea of these methods is to perform image super-resolution based on a sample image library containing pairs of high-resolution and low-resolution images. Freeman et al. proposed a sample-based method. They learned the relationship between low-resolution images and corresponding high-resolution images through Markov Networks, and used the learned relationship to process other low-resolution images. Super-resolution, this work was announced at the IEEE International Conference on Computer Vision in 1999 (IEEE International Conference on Computer Vision, (1999) 1182-1189). Hertzmann et al proposed a general local feature conversion method called "Image Analogies" at the 2001 ACM Graphics Conference (ACM SIGGRAPH 2001). They used a multi-scale auto-regression (Multi-scale Auto-regression) method to learn the local similarity between high-low resolution image pairs, and based on this, image super-resolution was performed. These methods are more suitable for super-resolution problems of general images, because they do not consider the special properties of face images. Baker and Kanade announced the first "Face Hallucination" technology at the 2000 IEEE International Conference on Automatic Face and Pose Recognition. They used the spatial distribution of face image gradient information as prior knowledge, and used Bayesian The inference means obtains a high-resolution face image from a low-resolution face image. This approach relies on very complex probabilistic models. Ce Liu announced a two-step face fantasy method at the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai Marriott, Hawaii (2001) 192-198 to achieve the same Purpose. Wang et al. developed an efficient face super-resolution technique based on the Eigen-transformation algorithm. Based on Principal Component Analysis (PCA), they approximate the input face image with a linear combination of low-resolution sample face images and solve for the combination coefficients. By keeping these combination coefficients and replacing low-resolution face images with high-resolution sample face images, high-resolution face images can be synthesized through linear combination. However, this method only considers the global image features, while ignoring the local detail information, causing the synthesized image to be unclear in the local area and lacking detail features. This work appeared in the 2005 IEEE Transactions (IEEE Trans.on Systems , Man, and Cybernetics, Part-C.2005, 35(3): 425-434).

In video surveillance applications, due to the low resolution of the video and the fact that the face is too far away from the lens, the image resolution of the face part is too low and the recognition is too poor, which makes it difficult to identify people. Face super-resolution technology can reasonably "deduce" high-resolution face images based on low-resolution face images, increase the recognition of faces in images, and will be widely used in the fields of face recognition and video surveillance.

Contents of the invention

The object of the present invention is to provide a method for super-resolution of human face images that fuses global features and local detail information, comprising the following steps:

1) According to the sample data of low-resolution and high-resolution face images, a transformation vector is established using a locality-preserving mapping algorithm;

2) Using a local preserving mapping algorithm to extract the intrinsic features of the low-resolution sample face image;

3) Using radial basis function regression to establish a correlation between the intrinsic features of the low-resolution face image and the corresponding high-resolution face image;

4) Project the input low-resolution face image onto the transformation vector to obtain intrinsic features, and use this intrinsic feature as the input of the radial basis function to obtain a global high-resolution face image;

5) Establish high-resolution and low-resolution sample residual image block matrices according to the sample image;

6) Calculate the weight of the low-resolution residual image block reconstructed from the k nearest low-resolution sample residual image blocks, and then replace the low-resolution residual image block with a high-resolution sample residual image block, Weighted synthesis of high-resolution residual image patches;

7) Combining high-resolution residual image blocks and smoothing with a linear smoothing operator to obtain a high-resolution residual face image;

8) Add the high-resolution residual face image obtained in step 7) to the global high-resolution face image obtained in step 4) to obtain the final super-resolution result.

The described establishment of the conversion vector by adopting the local preserving mapping algorithm comprises the following steps:

1) Let P be n pieces of high-resolution sample face images, P=p ₁ ,...,p _n , and the dimension is m; Q is the corresponding n pieces of low-resolution sample face images, Q=q ₁ ,..., q _n , the dimension is d, using principal component analysis to reduce the dimensionality of the low-resolution sample face image Q, and obtain the feature vector U and feature coefficient V;

2) Calculate the distance between any two face images q _i and q _j in the subspace represented by the eigenvector U, and select k nearest neighbors for each image q in the sample space, and construct a structure that reflects the local topology of the data set Neighborhood map;

3) If the face image q _i is one of the k neighbors of the face image q _j or the face image q _j is one of the k neighbors of the face image q _i , the weight is set to W _ij =‖q _i -q _j ‖ ² , otherwise W _ij =0;

4) The formula for calculating the conversion vector is: QLQ ^T a ¹ =λQDQ ^T a ¹ where D _ii =∑ _j W _ji , and L=DW is the Laplacian matrix, and the eigenvalue obtained by solving is assumed to be λ _i (l= 1,...L), with $S=\&lsqb;a_1^1,...,a_{\mathrm{<figure-callout\; id="1"\; label="h"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">h</figure-callout>}}^1\&rsqb;$ represents the eigenvector corresponding to the smallest first h eigenvalues, then the transformation vector that maps the original high-dimensional face image to the low-dimensional manifold space is expressed as A=US.

The described adopting the part-preserving mapping algorithm to extract the eigenfeatures of the low-resolution sample face images: calculate the eigenfeatures ^ytr of the sample low-resolution face images by the formula ^ytr = ^ATQ , wherein Q is the low-resolution The sample face image, A is the transformation vector established by the locality preserving mapping algorithm.

The method of establishing a correlation between the eigenfeatures of the low-resolution face image and the corresponding high-resolution face image using the radial basis function regression is as follows: the basic form of the radial basis function is $p_j=\underset{i=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{w}_{j}k({\mathrm{the\; y}}_i^{\mathrm{tr}},{\mathrm{the\; y}}_j^{\mathrm{tr}}),$ in $k({\mathrm{the\; y}}_i^{\mathrm{tr}},{\mathrm{the\; y}}_j^{\mathrm{tr}})=\exp ({\left|\right|{\mathrm{the\; y}}_i^{\mathrm{tr}}-{\mathrm{the\; y}}_j^{\mathrm{tr}}\left|\right|}^2/{2\mathrm{\&sigma;}}^2),$ And the constant σ can be calculated by the following formula ${\mathrm{\&sigma;}}^2=\frac{(\mathrm{ma}{x}_{\underset{j=1,...,\mathrm{no}}{i=1,...,\mathrm{no}}}K(i,j)-\mathrm{ma}{x}_{\underset{j=1,...,\mathrm{no}}{i=1,...,\mathrm{no}}}K(i,j))}{\mathrm{no}/\mathrm{nbs}},$ Among them, n represents the number of sample images, nbs represents the number of k nearest neighbors, and the radial basis function in matrix form is expressed as P=WK, wherein the correlation coefficient is W= ${\mathrm{<figure-callout\; id="1"\; label="w"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">w</figure-callout>}}_1,...,w_{\mathrm{no}},K=\left(\begin{array}{ccc}k({\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}},{\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}})& ...& k({\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}},{\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}})\\ ...& ...& ...\\ k({\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}},{\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}})& ...& k({\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}},{\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}})\end{array}\right),$ Use y _i=1,...n ^tr and P=p ₁ ,...p _n to train the radial basis function and obtain the correlation coefficient W.

The described generation global high-resolution face image comprises the steps:

1) Project the input low-resolution face image q ⁱⁿ onto the conversion vector A, and obtain the coordinate y ⁱⁿ of q ⁱⁿ in the low-dimensional manifold space, y ⁱⁿ = ^AT q ⁱⁿ ;

2) Take y ⁱⁿ as input data, calculate the matrix K according to k(y _i ^tr , y _j ^tr ), and calculate the global high-resolution face image p ^out according to P=WK.

The described method for establishing high-resolution and low-resolution sample residual image block matrices according to sample images is as follows: Let I ^l be a sample low-resolution face image, and I ^h be a corresponding global high-resolution face image, then The low-resolution residual face R ^l is R ^l =I ^l -D(I ^h ), where D( ) is a downsampling function; the high-resolution residual face R ^h is R ^h =I ^o - I ^h , where I ^o is the original high-resolution face image, let I ^l be the low-resolution sample residual face image again, let I ^h be the high-resolution sample residual face image, divide them into the same number Each pair of high-low resolution image small blocks satisfies a one-to-one correspondence; define P _t ^l (i, j) as the low-resolution residual small block in I ^l , and the center is v _ij ^l , P _t ^h (i, j) is the high-resolution residual block in I ^h , the center is v _ij ^h , and P _t ^l (i, j) is fixed to n ^l × n ^l size, n ^l is an odd number , fix P _t ^{h (} i, j) to n ^h ×n ^h size, n ^h = λn ^l between n ^l and n ^h , where λ is the scaling factor; The overlapping size is set to (n ^l -1)/2, and the overlapping size between high-resolution small blocks in I ^h is set to (odd(n ^h )-1)/2, where the function of odd(x) is to find The largest odd number not greater than x; once v _ij ^l is determined, P _t ^l (i, j) is known, and the position of P _t ^h (i, j) is also determined, the coordinates of the center point of the small block can be calculated as follows :

${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; ,</span>\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&lambda;v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1,1</span>}<span\; class="notranslate">\; )</span>$

k _a and k _b are:

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.$ ${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.$

Among them, mod( ) is a modulo operation;

Each residual face image is an image block matrix, and the image block located in row i and column j in the matrix is denoted as P _t (i, j).

Calculate the weight of the input low-resolution residual image patch reconstructed from the k nearest low-resolution sample residual image patches, and then replace the low-resolution residual image patch with a high-resolution sample residual image patch, weighted synthesis The high-resolution residual image block includes the following steps:

1) Subtract the down-sampled global high-resolution face image from the input low-resolution image to obtain the input low-resolution residual face image R _in ^l , divide R _in ^l into overlapping residual small Block P _t ⁱⁿ (i, j), i=1,...,r; j=1,...,c, where r and c are the number of rows and columns of the image block matrix respectively, and the initial values of i and j are both 1;

2) If i>r or j>c, the algorithm terminates;

Otherwise, for the current P _t ⁱⁿ (i, j), find its k-nearest neighbors in P _t(m) ^l (i, j) based on Euclidean distance, m=1,...,n, these k-nearest neighbors are denoted as P _t(k) ^l (i, j), k=1,..., K, K≤n;

3) Calculate weights for P _t(k) ^l (i, j); this is done by ${\mathrm{\&Sigma;}}_{k=1}^K{w}_k=1$ Under the constraints, the reconstruction error of P _t ⁱⁿ (i, j) is minimized, and the objective function is ${\mathrm{\&epsiv;}=\left|\right|P_t^{\mathrm{in}}(i,j)-\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{w}_k{P}_{t\left(k\right)}^l(i,j)\left|\right|}^2,$ Define C as $C=P_t^{\mathrm{in}}(i,j)\&Center\; Dot;\mathrm{ones}(1,K)-\&lsqb;P_{t\left(1\right)}^l(i,j),...,P_{t\left(K\right)}^l(i,j)\&rsqb;,$ where ones is a row vector whose K elements are all 1, then the local covariance matrix G can be expressed as G=C ^T C , and the input low-resolution residual is reconstructed from the k nearest low-resolution sample residual image blocks The weight w=(G ^-1 ones (K, 1))/(ones (K, 1) ^T G ^-1 ones (K, 1)) of image block, wherein w is the weight vector of K dimension;

4) Calculate the high-resolution residual image block based on w $P_t^{\mathrm{out}}(i,j):P_t^{\mathrm{out}}(i,j)=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}w\left(k\right)P_{t\left(k\right)}^h(i,j);$

5) If j<c, set j=j+1; otherwise, set i=i+1 and j=1, go to step 2).

The described generation high-resolution residual face image comprises the steps:

1) Superimpose the residual image small blocks to obtain the initial high-resolution residual face image;

2) Assuming that R ^h is a simple superposition of all residual image blocks, a smoothing operator SMO is defined at each pixel position to smooth R ^h (x, y). SMO is defined as:

Where 1≤p≤r, 1≤q≤c, r and c are the number of rows and columns of the image block matrix respectively, and the smoothing operation for R ^h (x, y) is a linear operation R ^h (x, y) = R ^h (x, y) · SMO (x, y).

The beneficial effect that the present invention has:

1) The local preservation mapping algorithm is linear, and a set of conversion vectors is given explicitly like principal component analysis. This shows that the locality-preserving mapping algorithm can easily deal with data outside the sample data set through a linear projection, and is better than the principal component analysis method in terms of generalization.

2) This method organically combines locality-preserving mapping and radial basis functions. The local preserving mapping captures the most essential features of the sample image, and the radial basis function regression establishes a relationship between the intrinsic features of the image and the original image. Compared with linear principal component analysis, the global high-resolution face image synthesized by this algorithm is closer to the real face, and the calculation efficiency is high.

3) The search strategy for kNN in the second stage is "position-dependent", which means that when searching for the data closest to the input image block at a specific position in the sample image block, only need to traverse the samples at the same position data. This will not cause a decrease in the quality of the face image, because the face image blocks that appear in the same position roughly reflect the features of the same part of the face, and kNN is also most likely to appear in these sample blocks. In addition, the kNN algorithm is applied to low-resolution image blocks. These features greatly reduce the computational complexity of kNN search.

Description of drawings

Fig. 1 is a schematic diagram of a global high-resolution human face generated by a local preservation mapping algorithm with different parameters in the present invention;

Fig. 2 is a one-to-one correspondence between high resolution and low resolution image residual small blocks in the present invention;

Fig. 3 is a schematic diagram of the mutual overlapping of four adjacent residual small blocks in the present invention;

Fig. 4 is a schematic diagram of the overlapping situation of residual small blocks on the entire image in the present invention;

Fig. 5 is a schematic diagram of the final super-resolution effect formed by superimposing the global high-resolution face and the high-resolution residual face image in the present invention;

Figure 6(a) is the original low-resolution image;

Figure 6(b) is a high-resolution image obtained by direct interpolation from the original low-resolution image;

Fig. 6 (c) is the high-resolution image obtained by adopting the method of the present invention;

Figure 6(d) is the real high-resolution image;

Figure 7(a) is the low-resolution original image actually taken in the stadium;

From left to right in Fig. 7 (b) is successively low-resolution original human face image, the high-resolution human face image that interpolation obtains, adopts the high-resolution human face image that method of the present invention obtains;

Figure 8(a) is the low-resolution original image actually taken in the laboratory;

In Fig. 8 (b), from left to right, it is the low-resolution original human face image, the high-resolution human face image obtained by interpolation, and the high-resolution human face image obtained by the method of the present invention;

Detailed ways

The face image super-resolution method that combines global features and local detail information is implemented as follows:

1) Use the locality-preserving mapping algorithm to extract the intrinsic features of the sample low-resolution face image. We validate the method described in the present invention on a database of Asian face images. The face images in the PF01 database come from a total of 107 volunteers, 56 males and 51 females, each with 17 images with different appearance features (1 frontal face, 4 images containing illumination changes, 8 images containing pose changes , and the remaining 4 pictures have expression changes). Of all the volunteers, 24 men and 8 women wore glasses. Most of the volunteers are between 20 and 30 years old. Since the purpose of the present invention is to perform super-resolution of frontal face images under uniform illumination conditions, remove face images with illumination and posture changes, and construct a new data set containing 321 images (reserve 1 frontal image for each person) image and 2 images with typical expression changes). All sample images have been pre-registered, and the pupils of the guarantors are roughly located in the same position of the images. On this basis, further manually adjust the image resolution to 96×128. Let P=p ₁ ,..., p ₆₀ be 60 high-resolution sample face images, the dimension is 12288; Q=q ₁ ,..., q ₆₀ are the corresponding 60 low-resolution sample face images, the dimension is 768 , use principal component analysis to reduce the dimensionality of Q, and get the feature vector U and feature coefficient V; calculate the distance between any two face images q _i and q _j in the subspace represented by U, and in the sample space for each Select 30 nearest neighbors for an image q to construct a neighborhood map reflecting the local topology of the data set; set the weight: if q _i is one of the k neighbors of q _j or q _j is one of the k neighbors of q _i , the weight Set to W _ij =‖q _i −q _j ‖ ² , otherwise W _ij =0; calculate transformation vector: solve the eigenvalue problem generalized as follows: QLQ ^T a ¹ =λQDQ ^T a ¹ , where D _ii =∑ _j W _ji , and L=DW is a Laplacian matrix, assuming the eigenvalue obtained from the solution is λ _l (l=1,...L), use $S=[a_1^1,...,a_{50}^1\&rsqb;$ represents the eigenvectors corresponding to the smallest first 50 eigenvalues, then the conversion vector for mapping the original high-dimensional face image to the low-dimensional manifold space can be expressed as A=US. The intrinsic feature y ^tr of the sample low-resolution face image can be calculated as y ^tr = ^AT Q.

2) Using the intrinsic features of the low-resolution sample image and the corresponding high-resolution sample image to train the radial basis function. The basic form of RBF is $p_j=\underset{i=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{w}_{j}k({\mathrm{the\; y}}_i^{\mathrm{tr}},{\mathrm{the\; y}}_j^{\mathrm{tr}}),$ in $k({\mathrm{the\; y}}_i^{\mathrm{tr}},{\mathrm{the\; y}}_j^{\mathrm{tr}})=\exp ({\left|\right|{\mathrm{the\; y}}_i^{\mathrm{tr}}-{\mathrm{the\; y}}_j^{\mathrm{tr}}\left|\right|}^2/{2\mathrm{\&sigma;}}^2),$ And the constant σ can be calculated by the following formula ${\mathrm{\&sigma;}}^2=\frac{(\mathrm{ma}{x}_{\underset{j=1,...,\mathrm{no}}{i=1,...,\mathrm{no}}}K(i,j)-\mathrm{ma}{x}_{\underset{j=1,...,\mathrm{no}}{i=1,...,\mathrm{no}}}K(i,j))}{\mathrm{no}/\mathrm{nbs}},$ Among them, n represents the number of sample images, which is 60 here, and nbs represents the number of nearest neighbors defined in step 2, which is 30 here. The RBF in matrix form is expressed as P=WK, where $W=[{\mathrm{<figure-callout\; id="1"\; label="w"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">w</figure-callout>}}_1,...,w_{\mathrm{no}}],K=\left(\begin{array}{ccc}k({\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}},{\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}})& ...& k({\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}},{\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}})\\ ...& ...& ...\\ k({\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}},{\mathrm{the\; <figure-callout\; id="1"\; label="y"\; filenames="S2008100591307E00021.png"\; state="\{\{state\}\}">y</figure-callout>}}_1^{\mathrm{tr}})& ...& k({\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}},{\mathrm{the\; y}}_{\mathrm{no}}^{\mathrm{tr}})\end{array}\right),$ Use y _i ^tr (i=1,...60) and P=[p ₁ ,...p ₆₀ ] to train RBF and get W.

3) Project the input low-resolution face image to the transformation vector. The dimension of the low-resolution face image is 768, and a total of 50 transformation vectors are extracted. The transformation vector forms a 768×50 matrix. The input low-resolution face image is a 1×768 vector. This vector and the transformation vector matrix After the operation, a 1×50 vector is obtained, which is the intrinsic feature of the input low-resolution face image. The above operation can be expressed as y ⁱⁿ = ^AT q ⁱⁿ , where A is the conversion vector, and q ⁱⁿ is the input low-resolution face image. resolution face image, ^yin is the intrinsic feature.

4) Using radial basis function regression to obtain a global high-resolution face image. Assuming that the dimension of the global high-resolution face image p ^out is N, p ^out can be calculated according to y ⁱⁿ and the previously obtained W, and the formula is p ^out = y ⁱⁿ W, where W is a 50×12288 matrix. Use k and h to represent the number of neighbors and the number of transformation vectors of each face image in the local preservation algorithm, respectively. We tested the algorithm on a sample dataset of 75 images, and synthesized global high-resolution faces at different k and h are shown in Fig. 1. The images in the first row are real high-resolution faces, and for the second, third, and fourth rows, the neighborhood sizes are specified as 5, 15, and 30, respectively. For each k, different numbers of transformation vectors from 10 to 70 are calculated to form a 3×7 face matrix. Figure 1 shows that when there are few conversion vectors, different k will make the synthesized face images very different; and when the number of conversion vectors gradually increases, the algorithm will converge to an optimized value, and the obtained results under different k The results tend to be consistent. This shows that compared with the size of the neighborhood, the number of transformation vectors is a more critical factor for the LPH algorithm. The best results are obtained when k=30 and h=70.

5) Synthesize the sample residual image by down-sampling and difference of the image, and further divide the residual image into small image blocks. Suppose I ^l is a sample low-resolution face image, and I ^h is the corresponding global high-resolution face image, then the low-resolution residual face R ^l is R ^l = I ^l -D(I ^h ), where D(·) is the downsampling function, which downsamples the image from 96×128 to 24×32; the high-resolution residual face R ^h is ^Rh = I ^o -I ^h , where I ^o is the original high-resolution High-rate face image, let I ^l be the low-resolution sample residual face image again, let I ^h be the high-resolution sample residual face image, divide them into the same number of small image blocks, each pair of high- The small blocks of the low-resolution image all satisfy the one-to ^- one correspondence, and the one-to-one correspondence between the small blocks of the high-resolution and low-resolution images is shown in Figure 2; define P _t ^l (i, j) as the low The resolution residual small block, the center is v _ij ^l , P _t ^h (i, j) is the high-resolution residual small block in I ^h , the center is v _ij ^h , and P _t ^l (i, j) is fixed n ^l ×n ^l size, n ^l is set to 3, ^{Ph t} ₍ i, j) is fixed to n ^h ×n ^h size, n ^h is set to 12, n ^h = λn is satisfied between n ^l and n ^{h l} , where λ is the scaling factor, here is 4; the overlap size between low-resolution patches in I ^l is set to (n ^l -1)/2=1, and the overlap between high-resolution patches in I ^h The size is set to (odd(n ^h )-1)/2=5, where the function of odd(x) is to find the largest odd number not greater than x; once v _ij ^l is determined, P _t ^l (i, j) is is known, and the position of P _t ^h (i, j) can also be uniquely determined, the coordinates of the center point of the small block can be calculated as follows:

${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; ,</span>\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&lambda;v</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1,1</span>}<span\; class="notranslate">\; )</span>$

k _a and k _b are:

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.$ ${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; mod</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.$

Among them, mod( ) is a modulo operation;

6) Extract the residual image according to the input low-resolution image, divide the residual image into small blocks, and search for 30 neighbors in the small blocks of the low-resolution sample image. Extracting the residual image of the input image is accomplished by downsampling the synthesized high-resolution global image with D( ) in the fifth step, and subtracting the downsampled image. Each residual face image is regarded as an image patch matrix, and the image patch located in row i and column j of the matrix is represented as P _t (i, j), each patch contains n×n pixels, and The size of the overlapping area between each small block and its four adjacent small blocks up, down, left, and right is (n-1)/2, and the image small block structure overlapping with each other is shown in FIG. 3 . Let P _t(k) ^l (i, j) be the 30 nearest neighbors of the input low-resolution residual block P _t ⁱⁿ (i, j) in the sample set, P _t(k) ^l (i, j ) can be directly calculated by Euclidean distance.

7) Calculate the weights w ₁ ,...w ₃₀ of reconstructing P _t ⁱⁿ (i, j) from the 30 nearest neighbors P _t(k) ^l (i, j), according to the low-resolution residual image block and the high-resolution One-to-one correspondence between high-rate residual image blocks Replace the 30 closest low-resolution residual image blocks with corresponding high-resolution residual image blocks, and synthesize the corresponding high-resolution residual image blocks with the same weight P _t ^out (i, j). Computing the weights for P _t(k) ^l (i, j) is done by ${\mathrm{\&Sigma;}}_{k=1}^{30}{w}_k=1$ Under the constraints, the reconstruction error of P _t ⁱⁿ (i, j) is minimized, and the objective function is $\mathrm{\&epsiv;}={\left|\right|P_t^{\mathrm{in}}(i,j)-\underset{k=1}{\overset{30}{\mathrm{\&Sigma;}}}{w}_k{P}_{t\left(k\right)}^l(i,j)\left|\right|}^2,$ This is a constraint-based least squares problem, and C is defined as $C=P_t^{\mathrm{in}}(i,j)\&Center\; Dot;\mathrm{ones}\left(\mathrm{1,30}\right)-\&lsqb;P_{t\left(1\right)}^l(i,j),...,P_{t\left(30\right)}^l(i,j)\&rsqb;,$ Among them, ones is a row vector whose 30 elements are all 1, then the local covariance matrix G can be expressed as G=C ^T C, and the solution of the constraint-based least squares problem above is w=(G ^-1 ones(30, 1 ))/(ones(30, 1) ^T G ^-1 ones(30, 1)), where w is a 30-dimensional weight vector. Calculate the high-resolution residual image block P _t ^out (i, j) based on w: $P_t^{\mathrm{out}}(i,j)=\underset{k=1}{\overset{30}{\mathrm{\&Sigma;}}}w\left(k\right)P_{t\left(k\right)}^h(i,j);$

8) Integrate the overlapping high-resolution residual image blocks together to form the entire high-resolution residual image, and use a smoothing operator to smooth the image, and then superimpose it with the global high-resolution face image to form the final Super resolution effect.

The synthesized residual patches overlap each other, which brings redundant image high-frequency features, which makes the synthesized face look sharper. We propose a linear smoothing operator based on the residual patch structure to solve this problem. Assuming that R ^h is a simple superposition of all residual image blocks, a smoothing operator SMO is defined at each pixel position (x, y) to smooth R ^h (x, y). SMO is defined as:

where 1 ≤ p ≤ r, 1 ≤ q ≤ c, r and c are the number of rows and columns of the image block matrix respectively, obviously SMO has the same dimension as R ^h . The smoothing operation for R ^h (x, y) is a simple linear operation R ^h (x, y)=R ^h (x, y)·SMO(x, y). According to this linear smoothing operator, for pixel positions where several adjacent small blocks overlap, the smoothed pixel value is the algebraic average of the corresponding pixel values of these small blocks. The overlapping residual blocks can be described by Fig. 4 . According to the image block matrix, the dark gray area in ^Rh is covered by 4 residual small blocks at the same time, and the light gray area is covered by 2 residual small blocks.

The final super-resolution result can be obtained by superimposing the smoothed high-resolution residual face with the previously obtained global high-resolution face image, as shown in Figure 5.

To validate the method described in the present invention, we provide two sets of super-resolution examples. That is, super-resolution is performed on the frontal unoccluded face images in the database and the actual captured images.

Example 1

Example of super-resolution of frontal unoccluded face images in the database:

The goal is to generate a corresponding high-resolution face from a low-resolution frontal face image with a neutral expression. Among the 107 volunteers in the database, 75 without glasses were selected, and their frontal face images were used as the experimental data set, of which 60 images were used to synthesize the sample set, and 15 images were used as test data. First, 60 high-resolution face images of 96×128 are down-sampled to 24×32, and these 60 high-low resolution face image pairs are used as sample data.

In the local preserving mapping algorithm, the number of transformation vectors is more critical than the size of the neighborhood. Here, the size of the neighborhood is fixed at 30, and the number of transformation vectors is set at 50. In the kNN search of the residual small block synthesis algorithm, the number of neighboring small blocks is also set to 30. When performing residual block synthesis, the high-low resolution sample residual image is divided into the same number of residual small blocks, the size of the low-resolution small block is 3×3, and the size of the high-resolution small block is 12×12 . In this set of experiments, an ideal global face image can be obtained using 50 transformation vectors. The residual face image contains high-frequency information of the image, which is used to compensate the detailed features of the global face. The final result is not only clear but also very close to the real image. The super-resolution result is shown in Figure 6. In Fig. 6, (a) is the low-resolution face image of input, (b) is the super-resolution result obtained by simple bicubic interpolation, (c) is the super-resolution result obtained by the method of the present invention, (d) is a real high-resolution face image.

Example 2

Super-resolution example of an actual captured image:

In order to further verify the effect of the method described in the present invention, we perform super-resolution on real-shot images. Still using the 60 high-low resolution face image pairs described in Example 1 as sample data, the number of transformation vectors in the local preserving mapping algorithm is 50, and the neighborhood size is 30. When compositing image residual blocks, the size of low-resolution small blocks is 3×3, the size of high-resolution small blocks is 12×12, and the number of adjacent small blocks is also set to 30.

Figure 7(a) is a low-resolution face image taken with a mobile phone in a stadium. The face area is manually extracted from this image and super-resolution is performed with different methods. The result is shown in Figure 7(b). From left to right, the three images in Fig. 7(b) are the original low-resolution image, the result of cubic B-spline interpolation, and the result obtained by the method of the present invention.

Figure 8(a) is an image randomly captured indoors, where the face area is very small. The super-resolution results of the above two methods are shown in Fig. 8(b). The three images in Fig. 8(b) from left to right are the original low-resolution image, the result of cubic B-spline interpolation, and The results obtained by the method of the present invention.

Claims (8)

1. the face image super-resolution method of amalgamation of global characteristics and local detail information is characterized in that comprising the steps;

1), adopt the local mapping algorithm that keeps to set up converting vector according to the sample data of low resolution and high-resolution human face image;

2) adopt the local intrinsic characteristics that keeps mapping algorithm to extract low resolution sample facial image;

3) adopt radial basis function to return between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image, to set up related;

4) the low resolution facial image with input projects to and obtains intrinsic characteristics on the converting vector, with the input of this intrinsic characteristics as radial basis function, obtains the high-resolution human face image of the overall situation;

5) set up high resolving power and low resolution sample residual image block matrix according to sample image;

6) calculate the weight of rebuilding the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, high resolving power residual image piece is synthesized in weighting;

7) combination high resolving power residual image piece also carries out smoothly obtaining high-resolution residual error facial image with the linear smoothing operator;

8) the overall high-resolution human face image addition that high-resolution residual error facial image that step 7) is obtained and step 4) obtain obtains final super-resolution result.

2. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that described employing part keeps mapping algorithm to set up converting vector and comprises the steps:

1) establishing P is n panel height resolution sample facial image, P=p ₁..., p _n, dimension is m; Q is corresponding n width of cloth low resolution sample facial image, Q=q ₁..., q _n, dimension is d, adopts principal component analysis (PCA) that low resolution sample facial image Q is carried out dimensionality reduction, obtains proper vector U and characteristic coefficient V;

2) in the subspace that proper vector U represents, calculate any two width of cloth facial image q _iAnd q _jBetween distance, and be that every width of cloth image q selects k arest neighbors in sample space, structure reflects the neighborhood figure of data set local topology;

3) if facial image q _iBe facial image q _jOne of k neighbour or facial image q _jBe facial image q _iOne of k neighbour, weight is set to W _Ij=‖ q _i-q _j‖ ², otherwise W _Ij=0;

4) formula of calculating converting vector is: QLQ ^Ta ¹=λ QDQ ^Ta ¹

D wherein _Ii=∑ _jW _Ji, and L=D-W is Laplce's matrix, establishing and finding the solution the eigenwert that obtains is λ _l(l=1 ... L), use $S=\&lsqb;a_1^1,...,a_h^1\&rsqb;$ Expression and preceding h minimum eigenwert characteristic of correspondence vector, the converting vector that then original higher-dimension facial image is mapped in the low-dimensional stream shape space is expressed as A=US.

3. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and 2 and local detail information is characterized in that the described local intrinsic characteristics that keeps mapping algorithm to extract low resolution sample facial image that adopts: by formula y ^Tr=A ^TQ calculates the intrinsic characteristics y of sample low resolution facial image ^Tr, wherein Q is a low resolution sample facial image, A adopts the local converting vector that keeps mapping algorithm to set up.

4. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information, it is characterized in that described employing radial basis function returns that to set up correlating method between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image as follows: the citation form of radial basis function is $p_j=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{w}_{j}k(y_i^{\mathrm{tr}},y_j^{\mathrm{tr}}),$ Wherein $k(y_i^{\mathrm{tr}},y_j^{\mathrm{tr}})=\exp ({\left|\right|y_i^{\mathrm{tr}}-y_j^{\mathrm{tr}}\left|\right|}^2/{2\mathrm{\&sigma;}}^2),$ And constant σ can be calculated by following formula ${\mathrm{\&sigma;}}^2=\frac{(\mathrm{ma}{x}_{\underset{j=1,...,n}{i=1,...,n}}K(i,j)-\mathrm{ma}{x}_{\underset{j=1,...,n}{i=1,...,n}}K(i,j))}{n/\mathrm{nbs}},$ Wherein n represents the number of sample image, and nbs represents the number of k arest neighbors, and the radial basis function of matrix form is expressed as P=WK, and wherein correlation coefficient is $W=w_1,...,w_n,K=\left(\begin{array}{ccc}k(y_1^{\mathrm{tr}},y_1^{\mathrm{tr}})& ...& k(y_1^{\mathrm{tr}},y_n^{\mathrm{tr}})\\ ...& ...& ...\\ k(y_n^{\mathrm{tr}},y_1^{\mathrm{tr}})& ...& k(y_n^{\mathrm{tr}},y_n^{\mathrm{tr}})\end{array}\right),$ Adopt y _{I=1 ... n} ^TrAnd P=p ₁... p _nTrain radial basis function and obtain correlation coefficient W.

5. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that the overall high-resolution human face image of described generation comprises the steps:

1) with the low resolution facial image q that imports ^InProject on the converting vector A, obtain q ^InCoordinate y in low-dimensional stream shape space ^In, y ^In=A ^Tq ^In

2) with y ^InAs the input data, according to k (y _i ^Tr, y _j ^Tr) compute matrix K, and calculate overall high-resolution human face image p according to P=WK ^Out

6. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that describedly setting up high resolving power and low resolution sample residual image block matrix method is as follows according to sample image: establish I ^lBe sample low resolution facial image, and I ^hBe corresponding overall high-resolution human face image, then residual error people's face R of low resolution ^lBe R ^l=I ^l-D (I ^h), wherein D () is the down-sampling function; High-resolution residual error people's face R ^hBe R ^h=I ^o-I ^h, I wherein ^oBe the original high resolution facial image, make I again ^lFor low resolution sample residual facial image, make I ^hBe high resolving power sample residual facial image, they are divided into the image fritter of similar number, each all satisfies one-to-one relationship to height-low-resolution image fritter; Definition P _t ^l(i j) is I ^lIn low resolution residual error fritter, the center is v _Ij ^l, P _t ^h(i j) is I ^hIn high resolving power residual error fritter, the center is v _Ij ^h, with P _t ^l(i j) is fixed as n ^l* n ^lSize, n ^lBe odd number, with P _t ^h(i j) is fixed as n ^h* n ^hSize, n ^lAnd n ^hBetween satisfy n ^h=λ n ^l, wherein λ is a zoom factor; I ^lOverlapping dimension between the middle low resolution fritter is made as (n ^l-1)/2, and I ^hOverlapping dimension between the middle high-resolution fritter is made as (odd (n ^h)-1)/2, wherein the effect of function odd (x) is the odd number that finds the maximum that is not more than x; In case determine v _Ij ^l, P _t ^l(i, j) be known, and P _t ^h(i, position j) determines that also the coordinate of fritter central point can be calculated as follows:

$v_{\mathrm{ij}}^{\text{l}}=(\underset{a=1}{\overset{i}{\mathrm{\&Sigma;}}}{k}_a,\underset{b=1}{\overset{j}{\mathrm{\&Sigma;}}}{k}_b)$ $V_{\mathrm{ij}}^h=(\mathrm{\&lambda;}\underset{a=1}{\overset{i}{\mathrm{\&Sigma;}}}{k}_a-1,\mathrm{\&lambda;}\underset{b=1}{\overset{j}{\mathrm{\&Sigma;}}}{k}_b-1)={\mathrm{\&lambda;v}}_{\mathrm{ij}}^l-\left(\mathrm{1,1}\right)$

k _aAnd k _bFor:

$k_a=\left\{\begin{array}{cc}(n^l+1)/2& \mathrm{mod}(a,2)=1\\ (n^l-1)/2& \mathrm{mod}(a,2)=0\end{array}\right.$

Wherein mod () is a modulo operation;

Every width of cloth residual error facial image is exactly a little block matrix of image, and the image fritter that is arranged in the capable j row of matrix i is expressed as P _t(i, j).

7. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information, it is characterized in that calculating the weight of rebuilding the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, the synthetic high resolving power residual image piece of weighting comprises the steps:

1) the low resolution residual error facial image R that the overall high-resolution human face image of subduction behind the down-sampling obtains importing on the low-resolution image of input _In ^l, with R _In ^lBe divided into overlapped residual error fritter P _t ^In(i, j), i=1 ..., r; J=1 ..., c, wherein r and c are respectively the line number and the columns of image block matrix, the initial value of i and j is 1;

2) if i＞r or j＞c, algorithm stops;

Otherwise, for current P _t ^In(i, j), at P _{T (m)} ^l(i finds its k neighbour based on Euclidean distance in j), m=1 ..., n, these k neighbor table are shown P _{T (k)} ^l(i, j), k=1 ..., K, K≤n;

3) be P _{T (k)} ^l(i j) calculates weight; This be by ${\mathrm{\&Sigma;}}_{k=1}^K{w}_k=1$ Minimize P under the constraint _t ^In(i, reconstruction error j) realizes that objective function is $\mathrm{\&epsiv;}={\left|\right|P_t^{\mathrm{in}}(i,j)-\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{w}_k{P}_{t\left(k\right)}^l(i,j)\left|\right|}^2,$ Definition C is $C=P_t^{\mathrm{in}}(i,j)\&CenterDot;\mathrm{ones}(1,K)-\&lsqb;P_{t\left(1\right)}^l(i,j),...,P_{t\left(K\right)}^l(i,j)\&rsqb;,$ Wherein ones is that K element is 1 row vector, and then local covariance matrix G can be expressed as G=C ^TC is rebuild the weight w=(G of the low resolution residual image piece of input by the immediate low resolution sample residual image block of k ^-1Ones (K, 1))/(ones (K, 1) ^TG ^-1Ones (K, 1)), wherein w is the weight vectors of K dimension;

4) calculate high-resolution residual image piece based on w $P_t^{\mathrm{out}}(i,j):P_t^{\mathrm{out}}(i,j)=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}w\left(k\right)P_{t\left(k\right)}^h(i,j);$

5) if j＜c then makes j=j+1; Otherwise make i=i+1 and j=1, change step 2).

8. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that described generation high resolving power residual error facial image comprises the steps:

1) the residual image fritter is stacked up obtains initial high resolving power residual error facial image;

2) suppose R ^hBe the simple superposition of all residual image pieces, definition smoothing operator SMO is to R on each location of pixels ^h(x y) carries out smoothly, and SMO is defined as:

1≤p≤r wherein, 1≤q≤c, r and c are respectively the line number and the columns of image block matrix, for R ^h(x, smooth operation y) is linear operation R ^h(x, y)=R ^h(x, y) SMO (x, y).

Images