CN106203269A - A kind of based on can the human face super-resolution processing method of deformation localized mass and system - Google Patents

Abstract

The present invention discloses a kind of based on can the human face super-resolution processing method of deformation localized mass and system, this method is primarily upon the face super-resolution method variable based on partial model, on the basis of face location priori, by introducing SIFT Flow feature, storehouse sample image block is carried out deformation, exptended sample storehouse image block pattern, strengthen the ability to express of existing image block, reconstructed results is made to have higher accuracy, further excavate facial image block and the relation of input facial image block in Sample Storehouse, optimize the result of Super-Resolution for Face Images；The present invention is remarkably improved the visual experience recovering image, is particularly well-suited to the recovery of low quality monitoring environment human face image.

Description

A kind of based on can the human face super-resolution processing method of deformation localized mass and system

Technical field

The present invention relates to image procossing and image-recovery technique field, be specifically related to a kind of based on can the people of deformation localized mass Face super-resolution processing method and system.

Background technology

Human face super-resolution technology is by supplemental training storehouse, learns high-low resolution corresponding relation, and then reaches from Some low-resolution face image estimate the purpose of high-resolution human face image.Human face super-resolution is widely used now In multiple fields, one of the most representational field is exactly that the facial image in monitoring video strengthens.Along with monitoring system Widely available, monitor video plays the most important effect in criminal evidence obtaining and criminal investigation fact-finding process.And face figure As one of positive evidence, in occupation of important position in case analysis and court are collected evidence.But, due to existence conditions Under, with photographic head distance relatively far away from, the monitoring face available pixel captured is considerably less, furthermore truth for target suspect Under due to vile weather (such as: misty rain), illumination (such as: illumination is too strong, the darkest, light and shade uneven), the factor such as device is to capture The serious damage (such as: serious fuzzy and noise) that causes of image, image recovers, amplify and identification suffers from serious Interference.This is accomplished by using human face super-resolution skill upgrading image resolution ratio, returns to high-resolution from low-resolution image Image.

Human face super-resolution technology is the technology solving this difficult problem, and it can be by secondary or several low resolution Facial image reconstruct high-resolution clear face image.Since this technology is proposed in 2000 first by Baker et al. Afterwards, this field just obtains the extensive concern of scholar, and produces a series of outstanding achievement in research, wherein with based on The Super-Resolution for Face Images practised is paid attention to by scholar.In recent years, manifold learning has been increasingly becoming the main flow of human face super-resolution Method.The core concept of this kind of method is: describes the manifold spatial relationship of low-resolution image, searches out each low resolution figure As the local property around data point, then the manifold of low-resolution image is non-linearly mapped to the stream of high-definition picture In shape space, spatially project in high-resolution correspondence, thus synthesize full resolution pricture.Representative having is following several Method: 2004, Chang^[1]In first manifold learning method introducing image super-resolution being reconstructed, it is proposed that a kind of neighborhood is embedding The image super-resolution Reconstruction Method entered.Sung Won Park^[2]A kind of self adaptation manifold based on locality preserving projections is proposed Learning method, analyzes the internal characteristics of face from local submanifold, reconstructs the radio-frequency component of low-resolution image disappearance.2005 Year, Wang^[3]A kind of method that Based PC A (Principal component analysis, principal component analysis) is decomposed is proposed, The linear combination of the main constituent of pending for low resolution image low-resolution spatial is represented, projection coefficient is to corresponding high score Resolution main constituent space obtains final result.The method has preferable robustness to morning, but still at result images Marginal existence ghost, the phenomenon of aliasing.2010, Lan^[5]The image caused for fuzzy and noise serious under monitoring environment The problem that pixel damage is serious, proposes a kind of face super-resolution method based on shape constraining, adds in tradition PCA framework The robustness disturbed, as measuring similarity criterion, is manually added shape when utilizing eye identification shape special by shape constraining Levy a conduct constraint, optimize the reconstructed results of low-quality image.2014, Dong^[4]Side based on local feature conversion is proposed Method, solves this problem further.

But, human face super-resolution technology based on study is wanted to obtain more preferable restoration result, it is necessary to larger Sample Storehouse cover more image block pattern, solve the densest (the limited training storehouse of training sample in manifold space With represent compared with face characteristic information manifold of higher dimension space) problem.But, it is individual heavy and multiple for setting up face training sample database Miscellaneous engineering.Additionally, training storehouse sample is the most, it is the highest that face carries out super-resolution reconstruction computational complexity.Therefore, how to strengthen The ability to express of existing training sample database facial image so that it is provide accurate table when the image block of matching low resolution Reach, become one problem needing solution badly of current face's super-resolution technique research.

In sum, for the problems referred to above, this paper presents based on local can the Super-Resolution for Face Images of deformation model, On the basis of face location priori, by introducing SIFT Flow^[6]Feature carries out deformation at storehouse sample image block, and expansion can Use image block pattern, strengthen the ability to express of existing image block so that reconstructed results has higher accuracy, further digs Face and the relation of input facial image in pick Sample Storehouse, thus realize optimizing the target of Super-Resolution for Face Images result. In CAS-PEAL-R1 face database, to down-sampling four times and have the test sample under noise situations to carry out it is demonstrated experimentally that I The algorithm objective evaluation index that proposes and subjective reconstructed results be superior to the most best algorithm, and noise is had robust Property.

In sum, by introducing SIFT Flow feature, Sample Storehouse image block is carried out deformation, strengthen the figure existed As the ability to express of block, cover more image block pattern, optimize Super-Resolution for Face Images further.

Literary composition relates to following list of references:

[1]H.Chang,D.-Y.Yeung,and Y.Xiong,“Super-resolution through neighbor embedding,”in Proc.IEEE Conf.Comput.Vis.Pattern Recog.,Jul.2004,pp.275–282.

[2]Sung Won Park,Savvides,M."Breaking the Limitation of Manifold Analysis for Super-Resolution of Facial Images",ICASSP,pp:573-576,2007.

[3]Xiaogang Wang and Xiaoou Tang,“Hallucinating face by eigentransformation,”Systems,Man,and Cybernetics,Part C:Applications and Reviews,IEEE Transactions on,vol.35,no.3,pp.425–434,2005.

[4] Dong little Hui, the unreal structure of the noise face [J] of the .2014. data-driven local feature such as Gao Ge, Chen Liang conversion. calculate Machine is applied, and 34 (12): 3576-3579.

[5]C Lan,R Hu,Z Han,A face super-resolution approach using shape semantic mode regularization.IEEE International Conference on Image Processing(ICIP),2021–2024,26-29Sept.2010.

[6]Ce Liu,Jenny Yuen,Antonio Torralba,et al.2011.SIFT Flow:Dense Correspondence across Scenes and its Applications[J],IEEE Transactions on Pattern Analysis and Machine Intelligence,33(5):978-994.

Summary of the invention

The problem existed for prior art, the invention provides a kind of based on can the human face super-resolution of deformation localized mass Processing method and system, be particularly suited for the recovery of facial image in low quality monitor video.

In order to solve above-mentioned technical problem, the present invention adopts the following technical scheme that；

A kind of based on can the human face super-resolution processing method of deformation localized mass, comprise the following steps:

S1: build the training storehouse in the low-resolution face image storehouse comprising high-resolution human face image library and correspondence thereof；

S2: use identical partitioned mode image division in pending low-resolution face image and training storehouse to be handed over for tool The image block of folded part, described image block be the length of side be the square image blocks of psize；

S3: to each piece of pending low-resolution face image, the low resolution at correspondence position is trained in set of blocks and looked into Look for its neighbour's block；

S4: calculate pending low-resolution face image block, the Deformation Field matrix of each piece to neighbour；

S5: according to Deformation Field matrix, calculates each neighbour's block shape to corresponding pending low-resolution face image block Become block；

S6: calculate the weight coefficient between pending low-resolution face image block and its neighbour's deformation block；

S7: weight projected in high resolution space, recovers image block according to reconstructed coefficientsObtain the height of its correspondence Resolution facial image block

S8: splicing high-resolution human face image blockThe resolution that secures satisfactory grades facial image.

Further, in described S1；By high-resolution human face image library middle high-resolution facial image aligned in position, and Carry out the process that degrades, obtain the low-resolution face image storehouse of correspondence, high-resolution human face image library and low resolution face figure As composing training storehouse, storehouse；

Before S2, make pending low-resolution face image identical with image size in training storehouse, and aligned in position.

Further, described aligned in position uses affine transformation method will carry out aligned in position；Concrete five positions include: Two canthus, nose, two corners of the mouths；

Affine transformation method is, divided by sample number, obtains flat by face images phase adduction in high-resolution human face image library All faces；If (x '_i,y′_i) it is ith feature point coordinates, (x on average face_i,y_i) it is on high-resolution human face image to be aligned Corresponding ith feature point coordinates；If affine matrixWherein a, b, c, d, e, f are affine transformation coefficient,Represent ith feature point coordinates (x ' on average face and high-resolution human face image to be aligned_i,y′_i) (x_i,y_iRelation between), uses Method of Direct Liner Transformation to solve affine transformation matrix M；High-resolution human face image to be aligned All coordinate points be multiplied with affine matrix M the coordinate obtained i.e. align after high-resolution human face image coordinate.

Further, in described S3, for pending low-resolution image x_in, it is assumed that the block on the i of position isLow Image in different resolution storehouse be set to X, X upper in position i's so image block is designated as Xⁱ；At XⁱOn K neighbour's block, pass throughAnd Xⁱ The absolute value of each image block difference contrast acquisition one by one, K the low-resolution image block that absolute difference is minimum, as 'sNeighbour, be designated as

Further, in described S4, each image blockDeformation Field matrix [u, v] pending image can be passed through WithBetween be calculated by the following method；

If (x, y) is the coordinate of pixel p on image to p=, if w (p)=(u (p), v (p)) is the flow vector of pixel p point, only Permitting u (p) and v (p) is set of integers；Make s₁And s₂RepresentWithSIFT feature；W (p) is the flow direction of the pixel at p Amount, wherein u (p) and v (p) represents this pixel displacement field in horizontal and vertical directions respectively；Obtain energy function E (w):

$\begin{array}{c}E\left(w\right)=\underset{p}{\mathrm{\Σ}}\min \left(\left|\right|s_1\left(p\right)-s_2\left(p+w\left(p\right)\right)|{|}_1,t\right)+\\ \underset{p}{\mathrm{\Σ}}\mathrm{\η}\left(|u\left(p\right)-u\left(p\right)|\right)+\\ \underset{p}{\mathrm{\Σ}}\min \left(\mathrm{\α}|u\left(p\right)-u\left(p\right)|,d\right)+\min \left(\mathrm{\α}|v\left(p\right)-v\left(p\right)|,d\right)\end{array}$

Wherein, the first row is data item, and expression is to allow SIFT feature mate along flow vector w (p) of pixel p point； Section 2 is displacement item, and expression is when not having other information may utilize, and makes flow vector w (p) the least；Section 3 is Smooth item, makes neighbor similar；In this object function, L1 criterion is used for the first row data item and the third line smooths item In the middle of, calculating coupling and peel off discontinuous with flow vector, t and d is respectively as threshold value；η, α are balance parameters, rule of thumb compose Value；

By minimizing energy function E (w), obtain Deformation Field matrix [u, v].

Further, in described S5, according to Deformation Field matrix, calculate each neighbour's block to corresponding pending low resolution The deformation block of rate facial image block, detailed process is:

By introducing [u, v], allowCoupling low-resolution image blockObtainLow resolution figure in Sample Storehouse As blockMapping relations, this contextual definition is characterized operator sf (), by feature operator sf (), will coupling To Sample Storehouse image block deformation to the most similar to input picture block, then can obtain the expression formula of Deformation Field [u, v]:

$\[u,v\]=sf(x_{in}^i,X_k^i)$

Image block according to high-resolution and low-resolution has similar concordance in manifold space, by deformation matrix [u, V], obtain strain image block R^k(i, j), definition Deform () represents willDeformation process, then

$R_{kL}^i=Deform(X_k^i,\[u,v\])$

For the low-resolution image block in training sample databaseImage block after deformation is expressed asIt is similar to, low High-definition picture block corresponding to image in different resolution block, the high-definition picture block after deformation is expressed as

According to introducing SIFT Flow feature operator sf (), obtain deformation matrix [u, v]；Utilize deformation matrix [u, v] shape Become image block, obtain the strain image block more like with input picture block so that the expression of Sample Storehouse image block is more accurate, increases The ability to express of sample in strong Sample Storehouse.

Further, in described S6, calculate the weight coefficient between pending image block and neighbour's deformation block, concrete mistake Journey is as follows:

It is firstly introduced into Euclidean distance d_k(i, j):

$d_k^i=\left|\right|x_{in}^i-X_k^i|{|}_2$

It it is i.e. input low resolution rate image blockWith the low-resolution image block in training sampleEuclidean distance；So After choose Euclidean distance near K high-definition picture block carry out deformation, the most now strain image block is represented by:

$R_{kL}^i=Deform(X_k^i,\[u,v\]),1\≤k\≤K.$

The low-resolution face image block of inputK closest low resolution strain image is concentrated by sample training BlockOptimum expression weight w represented^*(i, j) be:

$w_i^{*}=\underset{w_i}{\mathrm{argmin}}\left\{\right||x_{in}^i-\underset{k=1}{\overset{K}{\Σ}}{R}_{kL}^i{w}_i|{|}_2^2+\mathrm{\τ}\underset{k=1}{\overset{K}{\Σ}}{\[d_k^i{w}_k^i\]}^2\},s.t.\underset{k=1}{\overset{K}{\Σ}}{w}_k^i=1$

Wherein,It is with the low resolution strain image block in Sample StorehouseThe low-resolution image of linear expression input The weight coefficient of block, w_iIt is that element isVector；τ is balance parameters, rule of thumb assignment.

Further, in described S7, for the target high-resolution image block reconstructedIt can be by arest neighbors K High-resolution strain image blockWith its optimal weightsSynthesis:

$y_{out}^i={\mathrm{\Σ}}_{k=1}^K{R}_{kH}^i{w}_k^i$

Wherein, ForCorresponding high-resolution sample.

A kind of based on can the human face super-resolution processing system of deformation localized mass, including:

Training storehouse builds model, for building the low resolution face figure comprising high-resolution human face image library and correspondence thereof Training storehouse as storehouse；

Piecemeal module, for using identical partitioned mode by image in pending low-resolution face image and training storehouse Be divided into tool overlapping part image block, described image block be the length of side be the square image blocks of psize；

Neighbour's acquisition module, for each piece of pending low-resolution face image, in the low resolution of correspondence position Training set of blocks searches its neighbour's block；

Deformation Field matrix calculus module, is used for calculating pending low-resolution face image block, the shape of each piece to neighbour Variable field matrix；

Deformation module, for according to Deformation Field matrix, calculates each neighbour's block to corresponding pending low resolution face The deformation block of image block；

Weight coefficient acquisition module, for calculating between pending low-resolution face image block and its neighbour's deformation block Weight coefficient；

High-definition picture block generation module, for weight is projected to high resolution space, extensive according to reconstructed coefficients Complex pattern blockObtain the high-resolution human face image block of its correspondence

Concatenation module, for splicing high-resolution human face image block according to position iThe resolution that secures satisfactory grades facial image.

Compared to the prior art, the present invention has the advantages that:

This method is based on learning human face super-resolution technology if it is intended to restore more preferable result, it is necessary to larger Sample Storehouse cover more image block pattern, solve the densest (the limited training storehouse of training sample in manifold space With represent compared with face characteristic information manifold of higher dimension space) problem.But, it is individual heavy and multiple for setting up face training sample database Miscellaneous engineering, additionally, Sample Storehouse image is the most, it is the highest that face carries out super-resolution reconstruction computational complexity.Therefore, how to strengthen The ability to express of existing training sample database facial image, becomes current face's super-resolution research one and needs asking of solution badly Topic.

This method is primarily upon the face super-resolution method variable based on partial model, on the basis of face location priori On, by introducing SIFT Flow feature, storehouse sample image block is carried out deformation, exptended sample storehouse image block pattern, strengthen existing The ability to express of image block so that reconstructed results has higher accuracy, further excavates facial image block in Sample Storehouse With the relation of input facial image block, optimize the result of Super-Resolution for Face Images.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of the embodiment of the present invention；

Fig. 2 is that the facial image of the embodiment of the present invention is based on position piecemeal schematic diagram.

Detailed description of the invention

Storehouse sample image block, on the basis of face location priori, is carried out by the present invention by introducing SIFT Flow feature Deformation, exptended sample storehouse image block pattern, strengthen the ability to express of existing image block so that reconstructed results have higher accurately Property, further excavate facial image block and the relation of input facial image block in Sample Storehouse, utilize the concordance of multiple representation As constraint, strengthen concordance and noise robustness that image block characterizes, promote objective quality and the similarity of restoration result.

Below in conjunction with specific embodiments and the drawings, the present invention will be further described.

The present invention, towards the extremely low quality facial image under monitoring environment, uses double-deck manifold to assume and consistency constraint figure Sign as block.When being embodied as, technical solution of the present invention can use computer software technology to realize automatic operational process.

Seeing Fig. 1, the present invention specifically comprises the following steps that

S1: build the training storehouse in the low-resolution face image storehouse comprising high-resolution human face image library and correspondence thereof；

By high-resolution human face image library Y middle high-resolution facial image aligned in position, high-resolution human face image is entered The capable low-resolution face image processing corresponding that degrades, thus obtain low-resolution face image storehouse X.

Before S2, make pending low-resolution face image identical with image size in training storehouse, and aligned in position.

In being embodied as, first, by eyes and the face aligned in position of high-resolution human face image；Then, to high-resolution Rate facial image carries out down-sampling successively, fuzzy window filters, up-sampling, obtains the low resolution corresponding with high-resolution human face image Rate facial image.

For ease of implement reference, be provided below use affine transformation method realize facial image alignment detailed process:

Described aligned in position uses affine transformation method will carry out aligned in position；Concrete five positions include: two canthus, One nose, two corners of the mouths.

High-resolution human face image is carried out characteristic point mark, and characteristic point is face marginal point, such as canthus, nose, mouth Angle etc.；Then, affine transformation method alignment feature point is used.

Affine transformation method particularly as follows:

Face images phase adduction in high-resolution human face image library Y, divided by sample number, is obtained average face.If (x '_i, y′_i) it is ith feature point coordinates, (x on average face_i,y_i) it is that i-th corresponding on high-resolution human face image to be aligned is special Levy point coordinates.If affine matrixWherein a, b, c, d, e, f are affine transformation coefficient,Represent ith feature point coordinates (x ' on average face and high-resolution human face image to be aligned_i,y′_i) (x_i,y_iRelation between), uses Method of Direct Liner Transformation to solve affine transformation matrix M.High-resolution human face image to be aligned All coordinate points be multiplied with affine matrix M the coordinate obtained i.e. align after high-resolution human face image coordinate.

High-resolution human face image after alignment is done the process that degrades, such as, adopts under high-resolution human face image successively 4 times of sample, fuzzy window filter 23 * 3, up-sampling 4 times, obtain the low-resolution face image corresponding with high-resolution human face image, from And obtain low-resolution face image storehouse X.

Facial image one_to_one corresponding in high-resolution human face image library Y and low-resolution face image storehouse X, constitutes height point Resolution facial image pair.High-resolution human face image library Y and X composing training storehouse, low-resolution face image storehouse.

Make pending low-resolution face image identical with image size in training storehouse, and aligned in position.

The present invention is intended to pending low-resolution face image x_inProcess, estimate the high-resolution human of its correspondence Face image, is designated as high-resolution human face image y to be estimated by the high-resolution human face image estimated_out。

Pending low-resolution face image x_inIt is typically the low-resolution face image obtained at noisy severe environments.Right In the pending low-resolution face image as input, typically will be through pretreatment, including being cut out meeting Uniform provisions Face part, will pending low-resolution face image x_inUp-sample so that it is with facial image size phase in training storehouse With.To pending low-resolution face image x_inCarry out characteristic point mark, finally use the affine transformation method described in S1 to make to treat Process low-resolution face image x_inWith average face aligned in position.So so that facial image and pending low point in training storehouse Resolution facial image x_inIt is in identical level at size, eyebrow height.If pending low-resolution face image x_inDuring collection Insufficient light, then can be to pending low-resolution face image x after aligned in position_inCarry out auto brightness setting contrast, make It is in similar brightness level with low-resolution face image in training storehouse.

S2: use identical partitioned mode image division in pending low-resolution face image, training storehouse to be handed over for tool The square image blocks of folded part；Described image block be the length of side be the square image blocks of psize.

In this step, image each in training storehouse is all divided into N number of square image blocks；Meanwhile, by pending low resolution Rate facial image x_inIt is also divided into N number of image block.Use the corresponding facial image of image block set representations, high-resolution human face to be estimated Image y_outWill be by pending low-resolution face image x_inImage block recover obtain.By pending low resolution face Image x_in, high-resolution human face image y to be estimated_out, low-resolution face image X, training storehouse middle high-resolution face in training storehouse The image block collection of image Y is designated as respectivelyI represents that image block is numbered,Xⁱ、YⁱRepresent pending low-resolution face image x respectively_in, high-resolution human face image y to be estimated_out, training I-th image block in low-resolution face image X, training storehouse middle high-resolution facial image Y in storehouse.

Seeing Fig. 2, the Main Basis that facial image carries out piecemeal is the thought of local manifolds, i.e. facial image is that a class is special Different image, these images have a specific structural meaning, such as on certain position all of fritter be all eyes or certain It is all nose on position, say, that in image, the local fritter of each position is all in a specific local geometric manifold In the middle of.For ensureing this local manifolds, need image is divided into some foursquare image blocks.The size of image block needs conjunction Suitable size, if piecemeal is too big, then can cause ghost phenomena due to small alignment problem；If piecemeal is the least, can obscure, desalinate often The position feature of individual fritter.In addition it is also necessary to the size of overlapping block between selection image block.Because if simply image being divided For the some square tiles without overlapping block, then can be because grid occurs in incompatibility problem between these square block and blocks Effect.And facial image is the most square, then the size Selection of overlapping block should be noted that so that image fills as far as possible The piecemeal divided.

Image block size is designated as psize × psize, and between adjacent image block, the width of overlapping part is designated as D, by image block institute It is expressed as i, i=1,2 in position ... N, then have:

Wherein, height and width is respectively the height and width of facial image.In embodiment, psize takes 12, and D takes 8.

S3: to each piece of pending low-resolution face image, the low resolution at correspondence position is trained in set of blocks and looked into Look for its neighbour's block；

For pending low-resolution image x_in, it is assumed that the block on the i of position isLow-resolution image storehouse is set to X, X On in position i's so image block is designated as Xⁱ。At XⁱOn K neighbour's block, pass throughAnd XⁱEach image block difference Absolute value contrasts acquisition one by one, K the low-resolution image block that absolute difference is minimum, asNeighbour, be designated as

S4: calculate pending low-resolution face image block, the Deformation Field matrix of each piece to neighbour；

In S4, each image blockDeformation Field matrix [u, v] pending image can be passed throughWithBetween by with Lower method is calculated:

We set p=, and (x, y) is the coordinate of pixel p on image, if w (p)=(u (p), v (p)) is the flow direction of pixel p point Amount, we only permit u (p) and v (p) is set of integers.W (p) is the flow vector of the pixel at p, and wherein u (p) and v (p) is respectively Represent this pixel displacement field in horizontal and vertical directions；Allow s₁And s₂RepresentWithSIFT feature.So To energy function E (w):

$\begin{array}{c}E\left(w\right)=\underset{p}{\mathrm{\Σ}}\min \left(\left|\right|s_1\left(p\right)-s_2\left(p+w\left(p\right)\right)|{|}_1,t\right)+\\ \underset{p}{\mathrm{\Σ}}\mathrm{\η}\left(|u\left(p\right)-u\left(p\right)|\right)+\\ \underset{p}{\mathrm{\Σ}}\min \left(\mathrm{\α}|u\left(p\right)-u\left(p\right)|,d\right)+\min \left(\mathrm{\α}|v\left(p\right)-v\left(p\right)|,d\right)\end{array}$

Wherein, the first row is data item, and expression is to allow SIFT feature mate along flow vector w (p) of pixel p point； Section 2 is displacement item, and expression is when not having other information may utilize, and makes flow vector w (p) the least；Section 3 is Smooth item, makes neighbor similar.In this object function, L1 criterion (absolute value addition) be used for the first row data item and The third line smooths in the middle of item, calculates coupling and peels off discontinuous with flow vector, t and d is respectively as threshold value；η, α are balance parameters, Rule of thumb assignment.

By minimizing energy function E (w), we can obtain Deformation Field matrix [u, v].

S5: according to Deformation Field matrix, calculates each neighbour's block shape to corresponding pending low-resolution face image block Become block；

In S5, according to Deformation Field matrix, calculate each neighbour's block to corresponding pending low-resolution face image block Deformation block, detailed process is:

We, by introducing [u, v], allowCoupling low-resolution image blockObtainLow resolution in Sample Storehouse Rate image blockMapping relations, this contextual definition is characterized operator sf (), by feature operator sf (), I Can will match to Sample Storehouse image block deformation to the most similar to input picture block, then can obtain the expression formula of Deformation Field [u, v]:

$\[u,v\]=sf(x_{in}^i,X_k^i)$

The deformation matrix [u, v] that we obtain, has similar according to the image block of high-resolution and low-resolution in manifold space Concordance, then we then can pass through deformation matrix [u, v], obtains strain image block R^k(i, j), we define Deform () represents willDeformation process, then

$R_{kL}^i=Deform(X_k^i,\[u,v\])$

For the low-resolution image block in training sample databaseImage block after its deformation is represented bySimilar , high-definition picture block corresponding to low-resolution image block, the high-definition picture block after its deformation is represented by

Therefore, introduce SIFT Flow feature operator sf () herein, obtain deformation matrix [u, v].Utilize deformation matrix [u, v] strain image block, can obtain the strain image block more like with input picture block so that the expression of Sample Storehouse image block More accurate, strengthen the ability to express of sample in Sample Storehouse.

S6: calculate the weight coefficient between pending image block and neighbour's deformation block；Particularly low resolution face figure The weight coefficient of each neighbour's image block of each image block of picture

In step S6, calculating the weight coefficient between pending image block and neighbour's deformation block, detailed process is as follows:

It is firstly introduced into Euclidean distance d_k(i, j):

$d_k^i=\left|\right|x_{in}^i-X_k^i|{|}_2$

It it is i.e. input low resolution rate image blockWith the low-resolution image block in training sampleEuclidean distance.So After choose Euclidean distance near K high-definition picture block carry out deformation, the most now strain image block is represented by:

$R_{kL}^i=Deform(X_k^i,\[u,v\]),1\≤k\≤K.$

The low-resolution face image block of inputK closest low resolution strain image is concentrated by sample training BlockOptimum expression weight w represented^*(i, j) be:

$w_i^{*}=\underset{w_i}{\mathrm{argmin}}\left\{\right||x_{in}^i-\underset{k=1}{\overset{K}{\Σ}}{R}_{kL}^i{w}_i|{|}_2^2+\mathrm{\τ}\underset{k=1}{\overset{K}{\Σ}}{\[d_k^i{w}_k^i\]}^2\},s.t.\underset{k=1}{\overset{K}{\Σ}}{w}_k^i=1$

Wherein,It is with the low resolution strain image block in Sample StorehouseThe low-resolution image of linear expression input The weight coefficient of block；w_iIt is that element isVector；τ is balance parameters, rule of thumb assignment.

S7: weight projected in high resolution space, recovers image block according to reconstructed coefficientsObtain the height of its correspondence Resolution facial image block

In step S7, and weight is projected in high resolution space, recover image block according to reconstructed coefficientsObtain it Corresponding high-resolution human face image blockDetailed process is:

For the target high-resolution image block reconstructedIt can be by K high-resolution strain image block of arest neighborsWith its optimal weightsSynthesis:

$y_{out}^i={\mathrm{\Σ}}_{k=1}^K{R}_{kH}^i{w}_k^i$

Wherein, ForCorresponding high-resolution sample..

S8: splicing high-resolution human face image blockThe resolution that secures satisfactory grades facial image.

For checking the technology of the present invention effect, China face database CAS-PEAL is used to verify.Therefrom select 1040 Individual face sample, resolution is 112*96, with affine transformation method alignment face.40 width image down samplings are chosen from face sample 4 times (resolution is 24*28) is afterwards plus conduct test image after the Gaussian noise of 0.015.Using face sample residual image as Training storehouse, be respectively adopted tradition local face face super-resolution method (method 1). data-driven local feature conversion noise Face unreal structure method (method 2), robustness human face super-resolution processing method based on profile priori (method 3) obtain subjective figure Picture.

Knowable to experimental result, although method 1～3 has promoted in resolution than interpolation method, but occur in that tighter Weight error is the lowest with the similarity of original image.Result in method 2 is owing to being overall situation face framework, and method based on the overall situation is past Toward the short slab having on detail recovery, so being slightly poorer than the inventive method in this respect.The matter of the recovered image of the inventive method Measure compared to method 1～3 and bicubic interpolation method be all significantly increased.

Table 1 illustrates the objective quality that each image is corresponding, including PSNR (Y-PSNR) and SSIM value (structural similarity Criterion).From table 1 it follows that the inventive method is on the objective quality recovering image, also has and more significantly stably carry Rise.

Table 1 recovers the contrast of image objective quality

	PSNR value	SSIM value
			Method 1	20.6077	0.6006
Method 2	21.8865	0.6711
			Method 3	21.6226	0.6569
The inventive method	22.7622	0.7316

The large scale MARG that the inventive method is passed through to automatically extract from original low-resolution facial image is with original The characteristics of image of yardstick is combined, and recovers low quality facial image.Experimental result is from subjective quality to objective quality The introducing all demonstrating effectiveness of the invention, i.e. MARG effectively reduces the critical noisy shadow to super-resolution rebuilding Ringing, the feature automatically extracted avoids the counter productive (such as problems such as result instability, inaccuracy) that manual intervention brings, Thus improve human face super-resolution result.

Specific embodiment described herein is only to present invention spirit explanation for example.Technology neck belonging to the present invention Described specific embodiment can be made various amendment or supplements or use similar mode to replace by the technical staff in territory Generation, but without departing from the spirit of the present invention or surmount scope defined in appended claims.

Claims (9)

1. one kind based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that comprise the following steps:

S1: build the training storehouse in the low-resolution face image storehouse comprising high-resolution human face image library and correspondence thereof；

S2: using identical partitioned mode is tool overlap by image division in pending low-resolution face image and training storehouse Point image block, described image block be the length of side be the square image blocks of psize；

S3: to each piece of pending low-resolution face image, the low resolution at correspondence position is trained in set of blocks and searched it Neighbour's block；

S4: calculate pending low-resolution face image block, the Deformation Field matrix of each piece to neighbour；

S5: according to Deformation Field matrix, calculates each neighbour's block deformation block to corresponding pending low-resolution face image block；

S6: calculate the weight coefficient between pending low-resolution face image block and its neighbour's deformation block；

S7: weight projected in high resolution space, recovers image block according to reconstructed coefficientsObtain the high-resolution of its correspondence Rate facial image block

S8: splicing high-resolution human face image blockThe resolution that secures satisfactory grades facial image.

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

In described S1；By high-resolution human face image library middle high-resolution facial image aligned in position, and carry out the process that degrades, Obtain the low-resolution face image storehouse of correspondence, high-resolution human face image library and low-resolution face image storehouse composing training Storehouse；

Before S2, make pending low-resolution face image identical with image size in training storehouse, and aligned in position.

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

Described aligned in position uses affine transformation method will carry out aligned in position；Concrete five positions include: two canthus, Nose, two corners of the mouths；

Affine transformation method is, divided by sample number, face images phase adduction in high-resolution human face image library is obtained average face； If (x'_i,y'_i) it is ith feature point coordinates, (x on average face_i,y_i) it is corresponding on high-resolution human face image to be aligned Ith feature point coordinates；If affine matrixWherein a, b, c, d, e, f are affine transformation coefficient,Represent ith feature point coordinates (x' on average face and high-resolution human face image to be aligned_i,y'_i) (x_i,y_iRelation between), uses Method of Direct Liner Transformation to solve affine transformation matrix M；High-resolution human face image to be aligned All coordinate points be multiplied with affine matrix M the coordinate obtained i.e. align after high-resolution human face image coordinate.

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

In described S3, for pending low-resolution image x_in, it is assumed that the block on the i of position isLow-resolution image storehouse Be set to X, X upper in position i's so image block is designated as Xⁱ；At XⁱOn K neighbour's block, pass throughAnd XⁱEach image The absolute value of block difference contrasts acquisition one by one, K the low-resolution image block that absolute difference is minimum, asNeighbour, It is designated as

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

In described S4, each image blockDeformation Field matrix [u, v] pending image can be passed throughWithBetween pass through Following methods is calculated；

If (x, y) is the coordinate of pixel p on image to p=, if w (p)=(u (p), v (p)) is the flow vector of pixel p point, only permits U (p) and v (p) is set of integers；Make s₁And s₂RepresentWithSIFT feature；W (p) is the flow vector of the pixel at p, its Middle u (p) and v (p) represent this pixel displacement field in horizontal and vertical directions respectively；Obtain energy function E (w):

$\begin{array}{c}E\left(w\right)=\underset{p}{\mathrm{\Σ}}\min \left(\left|\right|s_1\left(p\right)-s_2\left(p+w\left(p\right)\right)|{|}_1,t\right)+\\ \underset{p}{\mathrm{\Σ}}\mathrm{\η}\left(|u\left(p\right)-u\left(p\right)|\right)+\\ \underset{p}{\mathrm{\Σ}}\min \left(\mathrm{\α}|u\left(p\right)-u\left(p\right)|,d\right)+\min \left(\mathrm{\α}|v\left(p\right)-v\left(p\right)|,d\right)\end{array}$

Wherein, the first row is data item, and expression is to allow SIFT feature mate along flow vector w (p) of pixel p point；Second Item is displacement item, and expression is when not having other information may utilize, and makes flow vector w (p) the least；Section 3 is smooth , make neighbor similar；In this object function, L1 criterion is used for the first row data item and the third line smooths item and works as In, calculating coupling and peel off discontinuous with flow vector, t and d is respectively as threshold value；η, α are balance parameters, rule of thumb assignment；

By minimizing energy function E (w), obtain Deformation Field matrix [u, v].

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

In described S5, according to Deformation Field matrix, calculate each neighbour's block to corresponding pending low-resolution face image block Deformation block, detailed process is:

By introducing [u, v], allowCoupling low-resolution image blockObtainLow-resolution image block in Sample StorehouseMapping relations, this contextual definition is characterized operator sf (), by feature operator sf (), will match to sample This storehouse image block deformation to the most similar to input picture block, then can obtain the expression formula of Deformation Field [u, v]:

$\[u,v\]=sf(x_{in}^i,X_k^i)$

Image block according to high-resolution and low-resolution has similar concordance in manifold space, by deformation matrix [u, v], To strain image block R^k(i, j), definition Deform () represents willDeformation process, then

$R_{kL}^i=Deform(X_k^i,\[u,v\])$

For the low-resolution image block in training sample databaseImage block after deformation is expressed asIt is similar to, low resolution High-definition picture block corresponding to rate image block, the high-definition picture block after deformation is expressed as

According to introducing SIFT Flow feature operator sf (), obtain deformation matrix [u, v]；Utilize deformation matrix [u, v] deformation map As block, obtain the strain image block more like with input picture block so that the expression of Sample Storehouse image block is more accurate, strengthens sample The ability to express of sample in this storehouse.

It is a kind of based on can the human face super-resolution processing method of deformation localized mass, it is characterised in that:

In described S6, calculating the weight coefficient between pending image block and neighbour's deformation block, detailed process is as follows:

It is firstly introduced into Euclidean distance d_k(i, j):

$d_k^i=\left|\right|x_{in}^i-X_k^i|{|}_2$

It it is i.e. input low resolution rate image blockWith the low-resolution image block in training sampleEuclidean distance；Then select K the high-definition picture block taking Euclidean distance near carries out deformation, and the most now strain image block is represented by:

$R_{kL}^i=Deform(X_k^i,\[u,v\]),1\≤k\≤K.$

The low-resolution face image block of inputK closest low resolution strain image block is concentrated by sample trainingOptimum expression weight w represented^*(i, j) be:

$w_i^{*}=\underset{w_i}{\mathrm{argmin}}\left\{\right||x_{in}^i-\underset{k=1}{\overset{K}{\Σ}}{R}_{kL}^i{w}_i|{|}_2^2+\mathrm{\τ}\underset{k=1}{\overset{K}{\Σ}}{\[d_k^i{w}_k^i\]}^2\},s.t.\underset{k=1}{\overset{K}{\Σ}}{w}_k^i=1$

Wherein,It is with the low resolution strain image block in Sample StorehouseThe power of the low-resolution image block of linear expression input Weight coefficient, w_iIt is that element isVector；τ is balance parameters, rule of thumb assignment.

A kind of based on can the human face super-resolution processing method of deformation localized mass and system, it is special Levy and be:

In described S7,

For the target high-resolution image block reconstructedIt can be by K high-resolution strain image block of arest neighborsWith Its optimal weightsSynthesis:

$y_{out}^i={\mathrm{\Σ}}_{k=1}^K{R}_{kH}^i{w}_k^i$

Wherein, ForCorresponding high-resolution sample.

9. one kind based on can the human face super-resolution processing system of deformation localized mass, it is characterised in that including:

Training storehouse builds model, for building the low-resolution face image storehouse comprising high-resolution human face image library and correspondence thereof Training storehouse；

Piecemeal module, for using identical partitioned mode by image division in pending low-resolution face image and training storehouse For have overlapping part image block, described image block be the length of side be the square image blocks of psize；

Neighbour's acquisition module, for each piece of pending low-resolution face image, the low resolution at correspondence position is trained Set of blocks is searched its neighbour's block；

Deformation Field matrix calculus module, is used for calculating pending low-resolution face image block, the Deformation Field of each piece to neighbour Matrix；

Deformation module, for according to Deformation Field matrix, calculates each neighbour's block to corresponding pending low-resolution face image The deformation block of block；

Weight coefficient acquisition module, for calculating the weight between pending low-resolution face image block and its neighbour's deformation block Coefficient；

High-definition picture block generation module, for weight is projected to high resolution space, recovers figure according to reconstructed coefficients As blockObtain the high-resolution human face image block of its correspondence

Concatenation module, for splicing high-resolution human face image block according to position iThe resolution that secures satisfactory grades facial image.