CN103971332B - A kind of face image super-resolution restored method based on the constraint of HR-LLE weights - Google Patents

Abstract

The invention belongs to digital image processing field, particularly to a kind of face image super-resolution restored method based on the constraint of HR LLE weights.First obtain a large amount of HR face sample and residual error HR face and rebuild weights constraint relative to the average of its neighbour's sample；In process of reconstruction, it is utilized respectively global and local and averagely rebuilds weights constraint and traditional human face super-resolution based on LLE is rebuild weights method for solving carry out weights constraint.Algorithm for reconstructing is divided into the overall situation to rebuild and local detail compensates two parts.Here the purpose that the overall situation is rebuild is the basic feature that backout criterion face should possess；The purpose that local detail compensates is to rebuild facial image to have the personal characteristics distinguishing other faces.The method, when estimating the reconstruction weights based on LLE of target HR image, adds the constraint of HR LLE weights, makes weights at l₂Closer to real HR image reconstruction weights in norm.The method can obtain preferable image restoration result.

Description

A kind of face image super-resolution restored method based on the constraint of HR-LLE weights

Technical field

The invention belongs to digital image processing field, surpass particularly to a kind of facial image based on the constraint of HR-LLE weights Resolution restored method.

Background technology

In recent years, the technology such as human face detection and recognition is in multimedia application such as video monitoring, mobile terminal and network retrievals Middle play the most important effect.The performance of these multimedia application is had a very big impact by the quality of facial image. But, owing to being affected by image capture device and collection environment, particularly under uncontrollable natural environment, get Facial image the most second-rate, it is difficult to directly apply to follow-up detection and identification.Oversubscription is used after man face image acquiring Resolution is restored (Super Resolution, SR) technology raising quality of human face image and is particularly important.

Existing human face super-resolution recovery technique can be divided into two classes: based on the method rebuild and side based on study Method.In recent years, method based on study becomes the focus of research.Its main thought be by learning method set up low resolution and Mapping relations between high-definition picture, machine learning the prior information obtained replaces based on artificially determining in method for reconstructing The constraints of justice.Along with the development that manifold learning is theoretical, researcher proposes a series of facial image assumed based on manifold Super-Resolution algorithm.Being locally linear embedding into (Locally Linear Embedding, LLE) is to have generation in manifold learning A kind of Method of Nonlinear Dimensionality Reduction of table, is used for carrying out the Super-Resolution of image by Many researchers in recent years, achieves Certain achievement.On the basis of human face super-resolution restoration algorithm based on LLE is all built upon manifold hypothesis, i.e. high-resolution (High Resolution, HR) image (or image block) and corresponding low resolution (Low Resolution, LR) image (or Image block) there is similar local geometry.Specifically in LLE, then show as in LR and HR space corresponding pixel or When image block carries out linear expression by pixel about or block, weighted vector is equal.This hypothesis is applied to image oversubscription During resolution is restored, initially set up the most paired LR-HR learning sample storehouse；Then for LR image to be reconstructed, sample is utilized LR sample in storehouse carries out linear expression, it is thus achieved that LR weights coefficient.Process of reconstruction uses LR weights coefficient directly to replace HR weights Coefficient, HR image is predicted in the HR linearity combination corresponding with Sample Storehouse.But, owing to the mapping in LR to HR space is one To many mapping relations, there is non-equidistant in the most this mapping.Directly replace HR space weights with the weights in LR space will draw Enter error.

Summary of the invention

It is an object of the invention to, by a kind of face image super-resolution recovery side based on the constraint of HR-LLE weights Method, is redeveloped into the image that resolution is higher by low-resolution image.Here high-resolution refer to spatial resolution amplify 4 times or More than 4 times.Present invention is generally directed to is facial image.

The present invention is to use techniques below means to realize:

First obtain a large amount of HR face sample and residual error HR face and rebuild weights constraint relative to the average of its neighbour's sample； In process of reconstruction, it is utilized respectively global and local and averagely rebuilds weights constraint to traditional human face super-resolution based on LLE Rebuild weights method for solving and carry out weights constraint.Overall flow figure is as shown in Figure 1.Algorithm for reconstructing is divided into the overall situation to rebuild and local Details compensates two parts.Here the purpose that the overall situation is rebuild is the basic feature that backout criterion face should possess；Local detail is mended The purpose repaid is to rebuild facial image to have the personal characteristics distinguishing other faces；

The method specifically includes following steps:

(1) overall situation HR-LLE weights constraint reestablishing

(1) overall situation is average rebuilds weights constraint

Initially set up paired HR face sample image storehouse and LR face sample image storehouse；Utilize Euclidean distance as measurement Standard finds out the K of each LR face sample successively₁Individual arest neighbors LR face sample and K corresponding thereto₁Individual HR face sample graph Picture；Traditional weights method for solving based on LLE is utilized to calculate LR face sample relative to its K₁Individual LR nearest samples Rebuild weights；Traditional weights method for solving based on LLE is utilized to show that HR face sample is relative to K₁Individual HR nearest samples Reconstruction weights, weight results is as shown in Figure 2；There is difference between LR weights and HR weights, this difference shows weights The overall heaving tendency of coefficient is consistent, but variance is different；The reconstruction weights of LR sample and HR sample are sought l₂Norm, result is such as Shown in accompanying drawing 3.The l rebuilding weights of HR sample₂Norm is to fluctuate in a scope the least, takes a large amount of HR sample weight Build the l of weights₂The meansigma methods of norm averagely rebuilds weights constraint W as the overall situation_g。

(2) overall situation is rebuild

Input LR facial image, utilizes Euclidean distance to find out K in LR face Sample Storehouse₁Individual LR nearest samples.Utilize The reconstruction weights optimization method based on HR-LLE that the present invention proposes show that input LR facial image is relative to its K₁Individual LR is The reconstruction weights of neighbour's sample；Find out K₁The HR sample that individual LR nearest samples is corresponding；Utilize and rebuild weights and K₁Individual HR sample Carry out linear combination and obtain the facial image of overall situation reconstruction.

(2) local detail compensates

(1) local average rebuilds weights constraint

First, HR residual error face Sample Storehouse and corresponding LR residual error face Sample Storehouse are set up；By the LR image profit in sample One group of HR image initially amplified is generated with overall situation HR-LLE weights bounding algorithm；Calculate HR image in Sample Storehouse initial with these Amplify the residual error between HR image, obtain one group of HR residual sample storehouse；Then the initial HR face sample image amplified is carried out Down-sampling, calculates the residual error between subimage and the LR face sample image after down-sampling, obtains one group of LR residual error face sample Image.

Owing to being that piecemeal is carried out when local detail compensates, so asking for the when that local average rebuilding weights constraint also The method using piecemeal.The method for solving averagely rebuilding weights constraint to the overall situation is similar, the reconstruction power of HR residual sample image block The l of value₂Norm is also to fluctuate in a scope the least, therefore takes a large amount of HR residual sample block and rebuilds the l of weights₂Model The meansigma methods of number rebuilds weights constraint W as local average_l。

(2) local detail compensates

The facial image rebuilding the overall situation carries out down-sampling process；Down-sampled images is done with input LR facial image Difference obtains LR residual error facial image；LR residual error facial image is carried out piecemeal process；Utilize Euclidean distance at LR residual error face sample This storehouse is found out the K of each LR residual image block successively₂Individual arest neighbors LR residual sample image block；Find out this K₂Individual arest neighbors LR The HR residual sample block that residual sample image block is corresponding；Utilize the reconstruction weights optimization based on HR-LLE that the present invention proposes Method show that input LR residual error facial image block is relative to K₂The reconstruction weights of individual LR arest neighbors residual sample block；Utilize this reconstruction Weights and K₂The linear combination of individual HR residual sample block obtains the HR residual image block rebuild.

Rebuild HR residual image block successively, finally reconstruct view picture HR residual error facial image；By this HR residual error facial image It is added with the initial facial image that amplifies, obtains final output and amplify facial image.

It is pre-that useful the having the technical effect that of the present invention proposes a kind of HR image reconstruction weights based on the constraint of HR-LLE weights Survey method.The method, when estimating the reconstruction weights based on LLE of target HR image, adds the constraint of HR-LLE weights, makes power Value is at l₂Closer to real HR image reconstruction weights in norm.On this basis, it is proposed that one is based on HR-LLE weights about The face image super-resolution restored method of bundle.Relative to traditional images method for reconstructing, the method can obtain preferable image Restoration result.

The feature of the present invention:

(1) a kind of target HR image reconstruction weights Forecasting Methodology based on the constraint of HR-LLE weights is proposed.The method exists When estimating the reconstruction weights based on LLE of target HR image, add the constraint of HR-LLE weights, make weights at l₂More connect in norm Nearly true HR weights.

(2) a kind of face image super-resolution restored method based on the constraint of HR-LLE weights is proposed.The method is respectively Retrain in terms of global characteristics, local feature two, rebuild high-resolution human face image.Experiment shows, context of methods is desirable Obtain preferable high-resolution human face image reconstruction result.

Accompanying drawing illustrates:

Fig. 1, the inventive method entire block diagram

The reconstruction weights comparison diagram of Fig. 2, HR face image weights and 4 times of down-sampling LR facial images

Fig. 3, face image weights l₂Norm and the weights l of 4 times of down-sampling LR faces₂Norm contrasts

Fig. 4, input low-resolution face image

Fig. 5, tentatively amplify facial image

Fig. 6, final amplification export facial image

Fig. 7, the inventive method and tradition interpolation amplification results contrast

Detailed description of the invention:

Below in conjunction with Figure of description, the embodiment of the present invention is illustrated.

(1) overall situation HR-LLE weights constraint reestablishing

(1) overall situation is average rebuilds weights constraint

Firstly, it is necessary to set up two face image pattern storehouses, the most paired HR facial image Sample Storehouse and corresponding LR people Face image pattern storehouse.This invention select CAS-PEAL facial image database and self-built China second-generation identity card image library have carried out reality Test；Have selected 1470 width front face images altogether, be all normalized to 140 × 160 pixel sizes as the HR face in Sample Storehouse Sample；4 times of down-sampling HR face samples, generate corresponding LR face sample.

Traditional weights method for solving based on LLE is utilized to calculate each LR face sample relative to its K₁Arest neighbors sample This reconstruction weights, as shown in Equation (1).

WhereinFor one of them LR face sample image,It is its K₁Individual nearest samples and It is that it is relative to K₁The reconstruction weights of individual nearest samples.In this invention, we are to K₁Value Test, find that, when it is more than 800, reconstruction effect is not obviously improved but time complexity but increases a lot.This K is taken in bright₁=800。

Traditional weights method for solving based on LLE is utilized to calculate the reconstruction weights of corresponding HR face sample, such as public affairs Shown in formula (2).

WhereinBe withCorresponding HR face sample image,Be with Corresponding K₁Individual arest neighbors HR sample,It is that it is relative to K₁The reconstruction weights of individual arest neighbors HR sample.

Solve optimization problem (1) and (2) and draw LR sample and the reconstruction weights of corresponding HR sample, example results such as accompanying drawing 2 Shown in.Wherein wLR represents the reconstruction weights of LR sample, and wHR represents the reconstruction weights of HR sample；It was found that LR weights and HR There is difference between weights, this difference shows that the overall heaving tendency of weights coefficient is consistent, and variance is different.To LR sample L is sought with the reconstruction weights of HR sample₂Norm, example results is as shown in Figure 3.Wherein star dotted line represents that weights rebuild by LR sample L₂Norm, circular dashed line represents that the l of weights rebuild by HR sample₂Norm；The l of weights rebuild by HR sample₂Norm is the least at one In the range of fluctuate, therefore take a large amount of HR sample rebuild weights l₂The meansigma methods of norm averagely rebuilds weights about as the overall situation Bundle W_g, this invention takes W_g=0.85。

(2) overall situation is rebuild

Input a width LR facial image x(to be not included in Sample Storehouse), size is 35 × 40 pixels, as shown in Figure 4； Euclidean distance is utilized to find out the K of x from LR face Sample Storehouse₁Individual arest neighbors face sampleUtilize Formula (3) draw x relative toReconstruction weightsProfit again The overall situation preliminary amplification facial image y is rebuild with formula (4)_G, as shown in Figure 5.

$\left\{\begin{array}{c}\min \mathrm{\ϵ}={\left|\right|x-\underset{j=1}{\overset{K_1}{\mathrm{\Σ}}}{w}_{G_j}{\stackrel{\→}{l}}_{G_j}\left|\right|}_2+\mathrm{\α}|{\left|\right|W_G\left|\right|}_2-W_g|\\ {\mathrm{\Σ}}_{j=1}^{K_1}{w}_{G_j}=1;\\ w_{G_j}=0,\mathrm{if}({\stackrel{\→}{l}}_{G_j}\∉[{\stackrel{\→}{l}}_{G_1},{\stackrel{\→}{l}}_{G_2},{\stackrel{\→}{l}}_{G_3}{\stackrel{\→}{l}}_{G_4}......{\stackrel{\→}{l}}_{G_{K_1}}\left]\right)\end{array}\right.---\left(3\right)$ $y_G={\mathrm{\Σ}}_{j=1}^{K_1}{w}_{G_j}{\stackrel{\→}{h}}_{G_j}---\left(4\right)$

WhereinIt isHR face sample corresponding in HR face Sample Storehouse.In (3), utilize the average weight of the overall situation Build weights l₂Norm constraint W_gTo rebuilding weights W_GRetraining, the reconstruction weights drawn after so solving optimization problem (3) are with regard to phase The true weights of docking close-target HR facial image.Being a customized parameter, the reconstructed results obtained when taking different value is not With, in this invention, we take=0.01.

(2) locally HR-LLE weights constraint details compensates

(1) local average rebuilds weights constraint

Initially set up HR residual error face sample and corresponding LR residual error face Sample Storehouse.Complete by the LR imagery exploitation in sample Office's HR-LLE weights bounding algorithm generates one group and initially amplifies HR image pattern, calculates HR image in Sample Storehouse and initially puts with these Residual error between big HR sample, obtains one group of HR residual sample storehouse, under then carrying out the initial HR face sample image amplified Sampling, calculates the residual error between subimage and the LR face sample image after down-sampling, obtains LR residual error face sample image.

Owing to being that piecemeal is carried out when local detail compensates, so asking for the when that local average rebuilding weights constraint also The method using piecemeal；HR residual error facial image is divided into the block of pixels of 8 × 8, corresponding LR residual error facial image by this invention It is divided into the block of pixels of 2 × 2；The method for solving averagely rebuilding weights constraint to the overall situation is similar, the reconstruction of HR residual sample image block The l of weights₂Norm is also to fluctuate in a scope the least, takes a large amount of HR residual sample block and rebuilds weights l₂Norm Meansigma methods averagely rebuilds weights constraint W as the overall situation_l, W in this invention_l=0.8。

(2) local detail compensates to generate and finally amplifies face

The overall situation is initially amplified face y_GCarry out 4 times of down-samplings, then down-sampling face done difference with input LR face x, Obtain inputting LR residual error face diffLR.LR residual error face diffLR is carried out piecemeal process (in order to ensure partial reconstruction image Blocking effect between block, all has the overlap of a pixel between image block), block size is 2 × 2 pixels；From LR residual error face Sample Storehouse sequentially finds each residual image blockK₂Individual arest neighbors residual blockThis invention Middle K₂> 600 time, recovery effect is not improved, and therefore takes K₂=600；Formula (5) is utilized to draw LR residual image blockRelatively InReconstruction weights $W_L=[w_{L_1},w_{L_2},w_{L_3},...,w_{L_{K_2}}].$ It is residual that recycling formula (6) rebuilds HR Difference image block

$\left\{\begin{array}{c}\min \mathrm{\ϵ}={\left|\right|{\mathrm{LR}}_{L_i}-\underset{j=1}{\overset{K_2}{\mathrm{\Σ}}}{w}_{L_j}{\stackrel{\→}{l}}_{L_j}\left|\right|}_2+\mathrm{\α}|{\left|\right|W_L\left|\right|}_2-W_l|\\ {\mathrm{\Σ}}_{j=1}^{K_2}{w}_{L_j}=1;\\ w_{L_j}=0,\mathrm{if}({\stackrel{\→}{l}}_{L_j}\∉[{\stackrel{\→}{l}}_{L_1},{\stackrel{\→}{l}}_{L_2},{\stackrel{\→}{l}}_{L_3}......{\stackrel{\→}{l}}_{L_{K_2}}\left]\right)\end{array}\right.---\left(5\right)$

${\mathrm{HR}}_{L_i}={\mathrm{\Σ}}_{j=1}^{K_2}{w}_{L_j}{\stackrel{\→}{h}}_{L_j}---\left(6\right)$

WhereinIt isHR residual error facial image block corresponding in HR residual error face sample image block storehouse.In (5) In, utilize local average to rebuild weights constraint W_lTo rebuilding weights W_LRetrain, draw after so solving optimization problem (5) That rebuilds that weights are just in relatively close proximity to target HR residual error facial image block truly rebuilds weights.

Rebuild HR residual image block successively, finally reconstruct view picture HR residual image diffHR.By HR residual error facial image DiffHR and initial amplification facial image y_GIt is added, obtains final output and amplify facial image y, as shown in Figure 6.

Fig. 7 is the inventive method and tradition interpolation amplification methods and results comparison diagram.

Claims (1)

1. a face image super-resolution restored method based on the constraint of HR-LLE weights, it is characterised in that include walking as follows Rapid:

(1) overall situation HR-LLE weights constraint reestablishing

(1) overall situation is average rebuilds weights constraint

Firstly, it is necessary to set up two face image pattern storehouses, the most paired HR facial image Sample Storehouse and corresponding LR face figure As Sample Storehouse；

Weights method for solving based on LLE is utilized to calculate each LR face sample relative to its K₁The reconstruction power of nearest samples Value, as shown in formula (1)；

WhereinFor one of them LR face sample image,It is its K₁Individual nearest samples and It is that it is relative to K₁The reconstruction weights of individual nearest samples；K₁=800；

Weights method for solving based on LLE is utilized to calculate the reconstruction weights of corresponding HR face sample, as shown in formula (2)；

WhereinBe withCorresponding HR face sample image,Be withRight The K answered₁Individual arest neighbors HR sample,It is that it is relative to K₁The reconstruction weights of individual arest neighbors HR sample；

Solve optimization problem (1) and (2) and draw LR sample and the reconstruction weights of corresponding HR sample, take HR sample and rebuild weights l₂Model The meansigma methods of number averagely rebuilds weights constraint W as the overall situation_g, W_g=0.85；

(2) overall situation is rebuild

Inputting a width LR facial image x, size is 35 × 40 pixels；Euclidean distance is utilized to find out x from LR face Sample Storehouse K₁Individual arest neighbors face sampleUtilize formula (3) draw x relative to Reconstruction weightsRecycling formula (4) rebuilds the overall situation preliminary amplification facial image y_G；

WhereinIt isHR face sample corresponding in HR face Sample Storehouse；In (3), the overall situation is utilized averagely to rebuild weights l₂Norm constraint W_gTo rebuilding weights W_GRetraining, the reconstruction weights drawn after so solving optimization problem (3) are just relatively close to The true weights of target HR facial image；=0.01；

(2) locally HR-LLE weights constraint details compensates

(1) local average rebuilds weights constraint

Initially set up HR residual error face sample and corresponding LR residual error face Sample Storehouse；By the LR imagery exploitation overall situation in sample HR-LLE weights bounding algorithm generates one group and initially amplifies HR image pattern, calculates HR image in Sample Storehouse and initially amplifies with these Residual error between HR sample, obtains one group of HR residual sample storehouse, adopting under then carrying out the initial HR face sample image amplified Sample, calculates the residual error between subimage and the LR face sample image after down-sampling, obtains LR residual error face sample image；

Owing to being that piecemeal is carried out when local detail compensates, so being also adopted by asking for local average reconstruction weights constraint when The method of piecemeal；HR residual error facial image is divided into the block of pixels of 8 × 8, and corresponding LR residual error facial image is divided into the picture of 2 × 2 Element block；Take HR residual sample block and rebuild weights l₂The meansigma methods of norm averagely rebuilds weights constraint W as the overall situation_l, W_l=0.8；

(2) local detail compensates to generate and finally amplifies face

The overall situation is initially amplified face y_GCarry out 4 times of down-samplings, then down-sampling face is done difference with input LR face x, obtain Input LR residual error face diffLR；LR residual error face diffLR is carried out piecemeal process, and block size is 2 × 2 pixels；Residual from LR Dispatch a person face Sample Storehouse sequentially finds each residual image blockK₂Individual arest neighbors residual blockK₂ =600；Formula (5) is utilized to draw LR residual image blockRelative toReconstruction weightsRecycling formula (6) rebuilds HR residual image block

WhereinIt isHR residual error facial image block corresponding in HR residual error face sample image block storehouse；In (5), utilize Local average rebuilds weights constraint W_lTo rebuilding weights W_LRetrain, the reconstruction weights drawn after so solving optimization problem (5) Just be in relatively close proximity to target HR residual error facial image block truly rebuilds weights；

Rebuild HR residual image block successively, finally reconstruct view picture HR residual image diffHR；By HR residual error facial image DiffHR and initial amplification facial image y_GIt is added, is finally exported amplification facial image y.