CN102354397A - Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs - Google Patents
Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs Download PDFInfo
- Publication number
- CN102354397A CN102354397A CN2011102787713A CN201110278771A CN102354397A CN 102354397 A CN102354397 A CN 102354397A CN 2011102787713 A CN2011102787713 A CN 2011102787713A CN 201110278771 A CN201110278771 A CN 201110278771A CN 102354397 A CN102354397 A CN 102354397A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- training
- similarity
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a method for reconstructing human facial image super-resolution based on the similarity of facial characteristic organs. The method comprises the following steps of: 1, establishing a high-resolution front human facial image library and a high-resolution characteristic organ image library by utilizing a gray scale projection method according to a preset ideal high-resolution human facial image; 2, extracting a low-resolution characteristic organ image from a low-resolution target human facial image; 3, performing bicubic interpolation on the low-resolution target human facial image and the low-resolution characteristic organ image to acquire a training image set of the low-resolution image; 4, constructing characteristic space corresponding to the training image set by the training image set to reconstruct projection vectors of a corresponding high-resolution integral human facial image and a corresponding high-resolution organ image; and 5, fusing the high-resolution integral human facial image and the high-resolution characteristic organ image into a high-resolution target human facial image. The method has the characteristics of less preprocessing time, high retrieval accuracy of training images, high trueness of the acquired human facial images and the like.
Description
Technical field
The present invention relates to a kind of face image super-resolution reconstruction method based on facial characteristics organ similarity.
Background technology
The purpose of image super-resolution rebuilding technology is the resolution of given low-resolution image is effectively promoted through digital image processing techniques, thereby obtains high-quality high-definition picture.Face image super-resolution rebuilding (claiming illusion face technology again) promptly reconstructs the high-resolution human face image that comprises enough effective informations through given low resolution front face image; This technology is widely used in safety-security areas such as video monitoring, criminal investigation at present, and there is very high requirement in above field of while to the quality of the high-resolution human face image that this technology recovers.The current image super-resolution rebuilding technology that generally adopts is divided three classes basically: based on interpolation, based on reconstruct with based on study.The estimation that is inserted into a pixel according to the original pixels and the specific interpolation formula completion that are inserted in K neighborhood a little based on the method for interpolation.The arithmetic speed of the method depends on the size of radius of neighbourhood K and the complexity of interpolation formula.Than other method, the method arithmetic speed is the fastest, but effect is also least desirable, therefore is not suitable for facial image.Based on image after the method utilization degeneration of reconstruct and the structure of the similarity between original low-resolution image cost function equation; Utilize the regularization term of characteristics of image prior imformation equationof structure simultaneously; Adopt the method for iteration to ask for this regularization equation optimum solution at last; Obtain optimum high-definition picture; But the effect of the method depends on the characteristics of image prior imformation that is adopted; Final high-definition picture edge effect of rebuilding is better, but operand is big and detailed information is deficient.Method based on study mainly is to remedy the required detailed information of original low-resolution image by the detailed information that one group of high resolving power training image provides; The reconstructed image edge effect that the method obtains is general; But image detail is abundant; Have visual effect preferably, so face image super-resolution rebuilding generally adopts the method based on study.
At present, researchist and technician have proposed multiple face image super-resolution reconstruction method.Method one: based on the face image super-resolution rebuilding algorithm of multiple dimensioned and multi-direction characteristic; Employing can be controlled the space distribution of the low level local feature of pyramid structure study facial image, predicts the characteristic matching between best high low-resolution image in conjunction with pyramid hierarchical structure and local Optimum Matching algorithm.Method two: based on the method for vector quantization, train the grid relational model between the high low resolution facial image, accomplish estimation through this model and training image then to the high-resolution human face image.Method three: based on the method for Markov random field; This method is thought between the image block or is satisfied a kind of nonparametric priori relational model between the image pixel; This model can utilize Markov random field model to replace; Can from training image, obtain high-frequency information by this model and compensate the required high-frequency information of low resolution target facial image, but the method operand is very big.Method four: based on the method for tensor, utilize the layered characteristic tensor to characterize people's face, obtain the global characteristics tensor, combine the local feature tensor to accomplish face image super-resolution rebuilding jointly again through training.Method five: based on the method for sparse expression; This method thinks that the high-definition picture piece can adopt the sparse expression of one group of standard signal unit to describe; Under the not serious situation of deteriroation of image quality, can recover the sparse expression of corresponding high-definition picture smoothly based on the principle of compressed sensing through low-resolution image.Method six: based on the method for proper subspace; Through multivariate statistics technology (MulitiVariate Statistical Technique); Pivot analysis (Principle Component Analysis; PCA); Multiple linear analyzes (MultiLinear Analysis) and nonnegative matrix is decomposed (Non-negative Matrix Factorization; NMF) etc. method with image transitions to proper subspace; Low resolution target facial image can be expressed through the linear combination of low resolution training image; Keep combination coefficient and the low resolution training image is replaced with corresponding high resolving power training image; The output that obtains is high resolving power target facial image; The method adaptability is strong, but training image accurately need be registrated to target image.
Face image super-resolution reconstruction method based on study mainly contains following two defectives: one of which, need complicated pretreatment operation.All there is this defective in above listed any method, mainly depends on the degree of functioning of training image, the i.e. integral body of training image and target image and local similar degree based on the face image super-resolution rebuilding process of study.Preprocessing process is meant to be that a width of cloth target image is set up effective image training set; Generally need the rapid complex operations of multistep; Comprise operations such as image retrieval, proportional zoom, image registration and brightness normalization; Simple pre-service is difficult to obtain desirable effect, high-precision preprocess method very time-consuming again (like optical flow method etc.).If pre-service is not enough, the effect of the high resolving power target image that reconstructs can receive very big influence, especially to have the greatest impact based on the method for proper subspace is suffered; Two, unsuitable little target image super-resolution rebuilding.Outside the eliminating method six; The effect of training image all is the high-frequency information compensation in the above method; Promptly concentrate the retrieval image similar or the high-frequency information of topography's piece with target image at training image; And the medium and low frequency information of training plan image set has only played the effect that helps retrieval; Do not participate in the image super-resolution rebuilding process of essence, caused the waste of resource.When the target image size hour, target image itself does not comprise effective high-frequency information basically, the high-frequency information that only relies on the low-frequency image similarity retrieval to obtain can not play effective compensating action, sometimes even can cause reverse effect.
Summary of the invention
The present invention is directed to the proposition of above problem, and develop a kind of face image super-resolution reconstruction method based on facial characteristics organ similarity.Concrete technical scheme is following:
A kind of face image super-resolution reconstruction method based on facial characteristics organ similarity is characterized in that comprising following method:
Step 1 is according to given desirable high-resolution human face image; Utilize the Gray Projection method to locate the human eye pupil position fast; The facial image all with cutting according to the interpupillary distance convergent-divergent; Make pupil position and face contour size basically identical between all images, the image behind the registration has been formed high resolving power front face image storehouse; Simultaneously all images in the high resolving power front face image storehouse is utilized once more the centre coordinate of the legal plane of Gray Projection portion characteristic organ, extract organic image according to fixed size and centre coordinate then, set up high-resolution features organic image storehouse;
Step 2 a pair width of cloth is treated the low resolution target facial image registration pupil of both eyes position of super-resolution rebuilding; Calculate the size of low resolution target facial image pupil coordinate and characteristic organic image according to the image super-resolution amplification coefficient, then according to the legal plane of Gray Projection portion's characteristic organ and extract low resolution characteristic organic image;
Step 3 is at first carried out bicubic interpolation to low resolution target facial image and low resolution characteristic organic image; Obtain initial target facial image and initial characteristics organic image, with all images in the high resolving power front face image storehouse as the candidate's training image that is directed against the initial target facial image; And repeatedly calculate the degree of approximation between initial target facial image and corresponding candidate's training image with it through the method for pivot analysis; A part of candidate's training image that similarity is low is rejected; Candidate's training image that the residue degree of approximation is high is formed the training plan image set to low resolution target facial image, and is identical with it to the training plan image set construction method of low resolution characteristic organic image;
Step 4 makes up pairing separately feature space to the training plan image set of low resolution target facial image and low resolution characteristic organic image; Projection vector through low resolution target facial image and low resolution characteristic organic image; Reconstruct the projection vector of whole facial image of corresponding high resolving power and high resolving power organic image, the projection vector that will in feature space, rebuild returns to pixel space and has promptly obtained whole facial image of high resolving power and high-resolution features organic image;
Step 5 is fused to final high resolving power target facial image with whole facial image of high resolving power and the high-resolution features organic image that step 4 obtains; In the low resolution target facial image preprocessing process of step 2, located the centre coordinate of facial characteristics organ, final high resolving power target facial image is to rebuild back characteristic organic image and the whole facial image weighted sum at this position pixel value at the pixel value of organ site; Wherein, characteristic organic image center is to the weights Gaussian distributed on border.
Step 3 is said repeatedly calculates the similarity between initial target image and corresponding candidate's training image with it through the method for pivot analysis; The concrete steps that a part of image that similarity is low is rejected are following: 1. the similarity measurement of the first half number of times carries out in gradient field; Promptly at first calculate the gradient image of all images; Similarity between the compute gradient image then, thus the similarity of edge contour between candidate's training image and initial target image guaranteed; 2. the similarity measurement of back one demidegree is accomplished in the gray scale territory, the gray scale similarity between promptly direct calculated candidate training image and initial target image.
The described image similarity measuring method of step 3 concrete steps are following: 1. the method by pivot analysis makes up all candidate's training image characteristic of correspondence spaces; Calculate all candidate's training images and the projection vector of initial target image in this feature space; 2. the Euclidean distance between the projection vector of the projection vector of initial target image and candidate's training image is as the basis for estimation of similarity between them; Candidate's training image that similarity is little is rejected; Remaining candidate re-constructs feature space; And with identical method calculating similarity; Once more that similarity is little candidate's training image is rejected; Number until residue candidate training image meets the demands, and finally obtains the training plan image set.
It is conspicuous comparing advantage of the present invention with prior art, is specially:
1, the image pre-service only needs to have practiced thrift pretreatment time greatly, image fault and the quality degradation of also having avoided many ratios registration and brightness processed to cause according to pupil position and face contour simple scalability and cutting image;
2, the multistep pca method retrieval similar image that combines with the gray scale territory through gradient field; Structure training plan image set; Both removed the interference of illumination change; Guaranteed the similarity of image outline again; Brought into play advantage to greatest extent, further improved the precision of result for retrieval based on the image search method of pivot analysis;
3, the facial characteristics organic image carries out super-resolution rebuilding separately, makes that the final facial image validity of rebuilding is higher, and the key position detailed information is abundanter;
4, the present invention more is applicable to the small-sized image super-resolution rebuilding; Remarkable based on the method for proper subspace effect when the small-sized image super-resolution rebuilding; And characteristic organic image itself is simple in structure; Similarity is high between image, has therefore further improved based on the reconstruction effect of proper subspace method to the small size facial image.
Description of drawings
Fig. 1 is the process flow diagram of the method for the invention;
Fig. 2 is the structure process flow diagram in high resolving power front face image storehouse and characteristic organic image storehouse;
Fig. 3 makes up process flow diagram for the training plan image set;
Fig. 4 is the face image super-resolution rebuilding process flow diagram based on pivot analysis;
Fig. 5 is a face image super-resolution rebuilding comparison diagram as a result.
Embodiment
Like Fig. 1 to its basic thought of face image super-resolution reconstruction method based on facial characteristics organ similarity shown in Figure 4 be: utilize the whole facial image of low resolution and facial characteristics organ utilized respectively based on the method for proper subspace and carry out super-resolution rebuilding, the two reconstructed results is merged obtain final high-definition picture.Than previous method; In the improvement that all has aspect algorithm complex, operand, image effect and the robustness in various degree; The whole facial image of high-resolution after the reconstruction has guaranteed the consistance of image border profile information and original object image; And the facial characteristics organic image after rebuilding has promoted the visual effect of facial image organ sites; Reduce overcritical to the training image registration accuracy simultaneously, reduced pretreatment time greatly.Concrete grammar is at first to set up high resolving power front face image storehouse; Utilize the Gray Projection method to locate the human eye pupil position fast; The facial image all with cutting according to the interpupillary distance convergent-divergent; Make pupil position and face contour size basically identical between all images, the image behind the registration has been formed high resolving power front face image storehouse.Secondly all images in the image library is utilized once more the centre coordinate of six facial characteristics organs such as the legal position of Gray Projection left eye, right eye, left eyebrow, right eyebrow, nose and lip; Extract organic image according to fixed size and centre coordinate then, set up high-resolution features organic image storehouse.When a given width of cloth is treated the low resolution target facial image of super-resolution rebuilding; At first it is carried out simple registration and extracts the feature organic image; Utilize the method for multistep pivot analysis to retrieve similar whole facial image and feature organic image then respectively for it in gradient field and gray scale territory; Result for retrieval is facial image and feature organic image training plan image set as a whole; On the basis of its training plan image set, adopt respectively for the general image of target people face and feature organic image and to carry out super-resolution rebuilding based on the method for proper subspace; Obtain whole man's face high-definition picture and facial characteristics organ high-definition picture; The two is merged according to previous locating information, promptly obtain final high-resolution human face image.
Combine accompanying drawing that the present invention is done further describes at present:
Fig. 1 describes is the overall flow of face image super-resolution rebuilding algorithm based on facial characteristics organ similarity proposed by the invention.
Step 1 is according to given desirable high-resolution human face image; Utilize the Gray Projection method to locate the human eye pupil position fast; The facial image all with cutting according to the interpupillary distance convergent-divergent; Make pupil position and face contour size basically identical between all images, the image behind the registration has been formed high resolving power front face image storehouse; Simultaneously all images in the high resolving power front face image storehouse is utilized once more the centre coordinate of the legal plane of Gray Projection portion characteristic organ, extract organic image according to fixed size and centre coordinate then, set up high-resolution features organic image storehouse; High resolving power front face image storehouse is based on the source of the required training image of face image super-resolution rebuilding of study, and the facial image in this image library has desirable high resolving power, and profile is consistent, measure-alike between image.The construction method of this image library is: given desirable high-resolution human face image; At first utilize the legal position of Gray Projection pupil position; According to the pupil position registering images; On this basis according to the face contour cutting image; After this processing; All images consistent size, pupil position are identical, and all have desirable high resolving power.The present invention adopts Gray Projection method location and extracts the facial characteristics organ, and its principle is: for front face image arbitrarily, organ is to satisfy certain statistical law at the distributing position of face.To the lower jaw bottom, people's face is divided into three parts from the forehead middle part from top to bottom, eyes are positioned at 1/3rd places basically, and nose is positioned at 1/2nd places, and face is positioned at 1/3rd places down, and the center of nose and mouth also is positioned on the median vertical line of people's face simultaneously.Can confirm the approximate location of each facial characteristics organ according to this statistical law at facial image.The characteristic of concrete implementation method is: at first according to the approximate location target setting zone of organ at face; Be generally rectangular window; Calculate the Gray Projection curve of the interior image of window then in level and vertical direction; Trough on each curve is the central point of character pair organ; Position like eyebrow; The eye pupil position, the center of lip etc.According to the characteristic organ centre coordinate that obtains, be partitioned into the organic image of fixed size then.Image in the high resolving power front face image storehouse is extracted the facial characteristics organ, structure left eye, right eye, left eyebrow, right eyebrow, nose, lip organ storehouse.
Step 2 a pair width of cloth is treated the low resolution target facial image of super-resolution rebuilding, and what " low resolution target facial image " referred to is exactly that needs adopt said method to carry out the original input picture of super-resolution rebuilding; Registration pupil of both eyes position; Calculate the size of low resolution target facial image pupil coordinate and characteristic organic image according to the image super-resolution amplification coefficient; Then according to the legal plane of Gray Projection portion's characteristic organ and extract low resolution characteristic organic image, " low resolution characteristic organic image " is the organic image that extracts from " low resolution target facial image ";
The structure of step 3 training plan image set is the process of the similar image of searched targets people face and characteristic organ in facial image database and organic image storehouse.The present invention proposes the multistep pca method retrieval similar image that gradient field and gray scale territory combine, the number of the image through constantly dwindling the construction feature space progressively improves the image similarity measuring reliability.At first low resolution target facial image and low resolution characteristic organic image are carried out bicubic interpolation; Obtain initial target facial image and initial characteristics organic image; " initial target facial image " and " initial characteristics organic image " obtains through interpolation; Be high resolving power; But poor definition, with all images in the high resolving power front face image storehouse as to candidate's training image of initial target facial image; And repeatedly (the supposition number of times is K) calculated the degree of approximation between initial target facial image and corresponding candidate's training image with it through the method for pivot analysis; A part of candidate's training image that similarity is low is rejected; Candidate's training image that the residue degree of approximation is high is formed the training plan image set to low resolution target facial image, and is identical with it to the training plan image set construction method of low resolution characteristic organic image; " low resolution characteristic organic image " and " low resolution target facial image " extracts the high image conduct training image separately of the degree of approximation respectively from " high resolving power organic image storehouse " and " high resolving power front face image storehouse ".Said many times through principal component analysis method to calculate the initial target image (image refers to the initial target of the initial target face image feature organ or initial image) and corresponding training image similarity between the candidate will be low similarity Excluding the specific part of the image as follows: ① first half of the number of times (before K / 2 times) similarity measure the gradient region, ie the first calculation of all image gradient image, and then compute the gradient similarity between the images, thus ensuring candidate training image and the initial target image similarity between the edge contour; ② number of the latter half (after K / 2 times) of the similarity measure the complete gray area, the direct calculation of the initial target candidates for the training image and the image similarity between the gray degrees; wherein the image similarity measure specific steps are as follows: ① through principal component analysis method to build all the candidate training image corresponding feature space, computing all candidate training images and the initial target image (initial target image refers to the procedure described The initial target face image feature organ or initial image) at the projection of the feature vector space, ② the initial target image projection vectors and the candidate training image projection Euclidean distance between the vectors as the basis to judge the similarity between them, the similarity of a candidate training image removing small remaining candidate feature space to reconstruct and similarity is calculated in the same way, again a candidate training image similarity small removed until the number of remaining candidate training image to meet the requirements, the final Get training image set.
Step 4 is the most sufficient to the utilization of training image information in all super resolution ratio reconstruction methods based on study based on the method for proper subspace, is therefore quoted by the present invention.Though proper subspace method implementation is more; But all be based upon under the maximum posteriori probability framework; Therefore the present invention is that example is done and briefly introduced with the method based on pivot analysis only; Training plan image set to low resolution target facial image and low resolution feature organic image makes up pairing separately feature space; Projection vector by low resolution target facial image and low resolution feature organic image; Reconstruct the projection vector of whole facial image of corresponding high-resolution and high-resolution organic image, the projection vector that will in feature space, rebuild returns to pixel space and has promptly obtained whole facial image of high-resolution and high-resolution features organic image;
Step 5 is fused to final high resolving power target facial image with whole facial image of high resolving power and the high-resolution features organic image that step 4 obtains; In the low resolution target facial image preprocessing process of step 2, located the centre coordinate of facial characteristics organ, final high resolving power target facial image is to rebuild back characteristic organic image and the whole facial image weighted sum at this position pixel value at the pixel value of organ site; Wherein, characteristic organic image center is to the weights Gaussian distributed on border.
The foundation in high resolving power front face image storehouse and organic image storehouse mainly is based on the image preprocessing process of Gray Projection method.As shown in Figure 2; Given any high-resolution human face image at first utilizes pupil of both eyes position, the legal position of Gray Projection, and according to the pupil position registering images; Press the fixed size cutting image according to face contour then, the gained result deposits in the high resolving power front face image storehouse; Next utilize organ centers such as the left eyebrow in the legal position of Gray Projection, right eyebrow, nose, lip once more, and extract all characteristic organic images that comprise eyes by fixed size, the gained result deposits in the corresponding organic image storehouse.
To the training plan image set of low resolution target facial image and basic identical to the structure principle of the training plan image set of low resolution characteristic organic image, difference only is that the latter need at first locate and extract the facial characteristics organ of low resolution target facial image.The building process of training plan image set is that example is described its characteristic with whole facial image as shown in Figure 3: supposition low resolution target facial image I
LAnd high resolving power front face image storehouse
Wherein i is the index position of image in the storehouse, as
Be the i width of cloth image in the high resolving power front face image storehouse; Wherein N is a total number of images order in the storehouse.Choose m (m N) width of cloth image people's face training plan image set as a whole for low resolution target facial image, m is the training image number, generally confirms the concrete numerical value of m according to actual conditions.Setting initial people's face training plan image set does
M ' is current candidate's training image number, and its initial value is total number of images order N in the high resolving power front face image storehouse, with target image I
LAdopt bicubic interpolation to be reconstructed into desirable high resolving power and obtain initial target image I
I, calculate I then
IWith
Gradient image I
GIWith
The similarity between the gradient field computed image at first, characteristic is: utilize the method for pivot analysis to make up
(m≤m '≤N) characteristic of correspondence space Ω
l(l＜m '), wherein l is the dimension of feature space, generally gets l less than m ', with the I of vectorization
GIAnd
Interior image all projects to Ω
l, obtain projection vector X separately
GIAnd
Calculate X
GIWith
The Euclidean distance of interior all projection vectors:
The larger
corresponding candidate training image
from the current candidate training image focused remove, delete images in this procedure may be repeated the number of times to be adjusted to obtain new candidate training set
where m 'value according to the number of deleted images accordingly changes, repeat the process several times, to delete the image with low similarity.As m ' during less than predetermined threshold value T, generally T is set to N/2, changes into gray scale territory computed image similarity, it is characterized in that: utilize the method for pivot analysis to make up current candidate's training plan image set
Characteristic of correspondence space Ω
L '(l '＜m '), wherein l ' is the dimension of feature space, with the initial target image I of vectorization
IAnd current candidate's training plan image set
Interior image all projects to Ω
L ', obtain projection vector X separately
IAnd
Calculate X
IWith
The Euclidean distance of interior all projection vectors:
Similarly a larger
corresponding
and concentrated to delete from the training images, deleting images based on the number of repetitions of this procedure to adjust, to get new training set
Repeat the process until m '= m.This completes the whole face image training set
build, features organ build an image with the same training set, not repeat them.
Characteristic based on pivot analysis technical construction characteristics of image subspace is: all images vectorization in the training plan image set is expressed this training plan image set of expression matrix of therefore available m row
Wherein m is the training image number, calculates the covariance matrix of this matrix, calculates the eigenwert of covariance matrix then
Keep l bigger eigenwert
Wherein i refers to the index position of element in set, as
(l＜m) and characteristic of correspondence thereof are vectorial to refer to i element in the set
Proper vector structural attitude space Ω thus, all training images and all can be expressed by its projection vector in Ω through the initial target image that bicubic interpolation obtains so are for the image I of arbitrary width of cloth vectorization
H, the projection vector in feature space Ω is X=B
T(I
H-μ), B=[b wherein
1..., b
l] be the eigenvectors matrix of previous reservation
μ is the training plan image set
The pixel average of interior all images.
Shown in Figure 4 is basic flow sheet based on the image super-resolution rebuilding method of pivot analysis.According to the theoretical derivation of maximum posteriori probability (MAP), suppose low resolution target image I
L, the high resolving power projection vector X that it is rebuild in feature space Ω
*For:
X
*＝(B
TA
TAB+λΛ
-1)
-1B
TA
T(I
L-Aμ)
A is a sample matrix, can be understood as a recognized function form, which functions as the high-resolution images will be quantized to a resolution of the target image sampling resolution; λ is a proportionality factor (values in between 0.02-0.5 ),
feature space constructed by the process described previously retained l eigenvalues
posed diagonal.The reconstruction of the projection vector back to the pixel space, restoring the expression is:
that was the final high-resolution image reconstruction
The purpose of image co-registration is that the characteristic organic image that will rebuild is fused in the whole facial image of reconstruction.In the construction process of organic image training set, low resolution target facial image has been carried out the organ location, according to these elements of a fix and super-resolution coefficient, the characteristic organic image accurately can be fused in the whole facial image of reconstruction.In order to reduce algorithm complex, the present invention adopts the image fusion technology based on intensity-weighted:
I
*(x，y)＝β(x，y)G(x，y)+(1-β(x，y))I(x，y)
Wherein ((x y) is the whole facial image of rebuilding, I to I to G for x, y) the high-resolution features organic image for rebuilding
*(x, y) for merging the back facial image, (x y) is the weights coefficient of pixel fusion to β, and span is between 0-1, and for guaranteeing the visual effect of fused images, (x, y) obey with the organic image central spot is peaked Gaussian distribution to β.
For the effect of verifying that the present invention proposes based on the face image super-resolution reconstruction method of facial characteristics organ; And the advantage of outstanding method involved in the present invention; We adopt the small size front face image to experimentize as the low resolution target image; The image size is 16 * 16, and amplification coefficient is 4.Experimental situation is Intelcore2CPU, and the PC of Windows7 system of 3GHz dominant frequency adopts matlab (R2010b) software to carry out emulation.All the experiment facial image derives from the Asian of Chinese Academy of Sciences face image data base CAS-PEAL-R1(document 1:WenGao; Etal.TheCAS-PEALLarge-ScaleChineseFaceDatabaseandBaselin eEvaluations.IEEETransactionsonSystemMan; AndCybernetics(PartA); 2008; (38): 149-161.); Comprise man's front face image 600 width of cloth altogether; Ms's front face image 400 width of cloth; All images is gray level image; Deposit high-resolution front face image storehouse in through intercepting facial zone behind the simple pupil registration, its resolution ratio is 64 * 64.
Can know according to experimental result in the past; When small size low resolution target facial image is reconstructed into high-resolution; General reconstructing method can't be competent at; And based on two-step approach (the document 2:C.Liu of pivot analysis; H.Shum; And C.Zhang; A Two-Step Approach to Hallucinating Faces:Global Parametric Model and Local Nonparametric Model; In Proc.Of CRPR; 2001; 1:192-198.) in the face image super-resolution rebuilding technology based on study, gain universal acceptance always; But this method requires very high for the facial characteristics point registration between target image and training image; Require training image and target image that very high similarity is arranged simultaneously; Usually we can't obtain sufficiently high registration accuracy, also can't obtain enough perfectly training image, so in the significant zone of image personal characteristics; Like organ; Profiles etc. are located, and can produce more fuzzy and degradation effect.Simultaneously with method involved in the present invention with contrast based on the method for bicubic interpolation with based on the two-step approach of pivot analysis; Reduced parameter is the Y-PSNR of image, and the Y-PSNR comparison diagram behind the super-resolution rebuilding of front face image as shown in Figure 5.Adopt the image Y-PSNR value contrast box figure after super-resolution rebuilding is carried out in the present invention and two kinds of control methodss; Box figure has described and has adopted every kind of method that five width of cloth man images and five width of cloth woman images are carried out the statistics of ten Y-PSNRs behind the super-resolution rebuilding, is followed successively by minimum value, quartile value, intermediate value, the 3rd quartile value and maximal value from top to bottom.
Simultaneously; Experiment is found when employing is carried out image super-resolution rebuilding based on the two-step approach of pivot analysis; The computation process of local feature facial image is very consuming time in second step; When low resolution target image size hour; This local feature image is not made the contribution of being expected to improving final high resolving power target image quality, and high-resolution human face image effect aspect details that institute of the present invention design method reconstructs is remarkable.The image reconstruction time average of method involved in the present invention is about 30 seconds, and the method that document [2] relates to is rebuild identical image and then needed about 400 seconds.
The above; Only be the preferable embodiment of the present invention; But protection scope of the present invention is not limited thereto; Any technician who is familiar with the present technique field is in the technical scope that the present invention discloses; Be equal to replacement or change according to technical scheme of the present invention and inventive concept thereof, all should be encompassed within protection scope of the present invention.
Claims (3)
1. face image super-resolution reconstruction method based on facial characteristics organ similarity is characterized in that comprising following method:
Step 1 is according to given desirable high-resolution human face image; Utilize the Gray Projection method to locate the human eye pupil position fast; The facial image all with cutting according to the interpupillary distance convergent-divergent; Make pupil position and face contour size basically identical between all images, the image behind the registration has been formed high resolving power front face image storehouse; Simultaneously all images in the high resolving power front face image storehouse is utilized once more the centre coordinate of the legal plane of Gray Projection portion characteristic organ, extract organic image according to fixed size and centre coordinate then, set up high-resolution features organic image storehouse;
Step 2 a pair width of cloth is treated the low resolution target facial image registration pupil of both eyes position of super-resolution rebuilding; Calculate the size of low resolution target facial image pupil coordinate and characteristic organic image according to the image super-resolution amplification coefficient, then according to the legal plane of Gray Projection portion's characteristic organ and extract low resolution characteristic organic image;
Step 3 is at first carried out bicubic interpolation to low resolution target facial image and low resolution characteristic organic image; Obtain initial target facial image and initial characteristics organic image, with all images in the high resolving power front face image storehouse as the candidate's training image that is directed against the initial target facial image; And repeatedly calculate the degree of approximation between initial target facial image and corresponding candidate's training image with it through the method for pivot analysis; A part of candidate's training image that similarity is low is rejected; Candidate's training image that the residue degree of approximation is high is formed the training plan image set to low resolution target facial image, and is identical with it to the training plan image set construction method of low resolution characteristic organic image;
Step 4 makes up pairing separately feature space to the training plan image set of low resolution target facial image and low resolution characteristic organic image; Projection vector through low resolution target facial image and low resolution characteristic organic image; Reconstruct the projection vector of whole facial image of corresponding high resolving power and high resolving power organic image, the projection vector that will in feature space, rebuild returns to pixel space and has promptly obtained whole facial image of high resolving power and high-resolution features organic image;
Step 5 is fused to final high resolving power target facial image with whole facial image of high resolving power and the high-resolution features organic image that step 4 obtains; In the low resolution target facial image preprocessing process of step 2, located the centre coordinate of facial characteristics organ, final high resolving power target facial image is to rebuild back characteristic organic image and the whole facial image weighted sum at this position pixel value at the pixel value of organ site; Wherein, characteristic organic image center is to the weights Gaussian distributed on border.
(2) according to claim 1, wherein the organ based on the similarity of the facial features of face image super-resolution reconstruction method, wherein said step three times through the main element method to calculate the initial target image and corresponding candidate training similarity between the images, the image will be removed as part of a low degree of similarity of the specific steps are as follows: ① the first half of the number of similarity measures within the gradient, ie the first calculation of all image gradient image, and then compute the gradient similarity between the images , thus ensuring the candidate training image and the initial target image similarity between the edge contour; ② the latter half of the number of complete similarity measure the gray area, the direct calculation of the initial target candidates for the training image and the image similarity between the gray-scale.
3. a kind of face image super-resolution reconstruction method according to claim 1 and 2 based on facial characteristics organ similitude; It is characterized in that the described image similarity measuring method of step 3 concrete steps are following: 1. the method by pivot analysis makes up all candidate's training image characteristic of correspondence spaces; Calculate all candidate's training images and the projection vector of initial target image in this feature space; 2. the Euclidean distance between the projection vector of the projection vector of initial target image and candidate's training image is as the basis for estimation of similarity between them; Candidate's training image that similarity is little is rejected; Remaining candidate re-constructs feature space; And with identical method calculating similarity; Once more that similarity is little candidate's training image is rejected; Number until residue candidate training image meets the demands, and finally obtains the training plan image set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110278771 CN102354397B (en) | 2011-09-19 | 2011-09-19 | Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110278771 CN102354397B (en) | 2011-09-19 | 2011-09-19 | Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102354397A true CN102354397A (en) | 2012-02-15 |
| CN102354397B CN102354397B (en) | 2013-05-15 |
Family
ID=45577958
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110278771 Expired - Fee Related CN102354397B (en) | 2011-09-19 | 2011-09-19 | Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102354397B (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629373A (en) * | 2012-02-27 | 2012-08-08 | 天津大学 | Super-resolution image acquisition method based on sparse representation theory |
| CN102800069A (en) * | 2012-05-22 | 2012-11-28 | 湖南大学 | Image super-resolution method for combining soft decision self-adaptation interpolation and bicubic interpolation |
| CN102902966A (en) * | 2012-10-12 | 2013-01-30 | 大连理工大学 | Super-resolution face recognition method based on deep belief networks |
| CN103020897A (en) * | 2012-09-28 | 2013-04-03 | 香港应用科技研究院有限公司 | Device for reconstructing based on super-resolution of multi-block single-frame image, system and method thereof |
| CN103454225A (en) * | 2013-07-05 | 2013-12-18 | 中南大学 | MPCA (Multiway Principal Component Analysis)-based measurement method for area of local region of copper flotation froth image |
| CN103914807A (en) * | 2012-12-31 | 2014-07-09 | 北京大学 | Non-locality image super-resolution method and system for zoom scale compensation |
| CN106033529A (en) * | 2014-09-12 | 2016-10-19 | 宏达国际电子股份有限公司 | Image processing method and electronic apparatus |
| CN107895345A (en) * | 2017-11-29 | 2018-04-10 | 浙江大华技术股份有限公司 | A kind of method and apparatus for improving facial image resolution ratio |
| CN108121957A (en) * | 2017-12-19 | 2018-06-05 | 北京麒麟合盛网络技术有限公司 | The method for pushing and device of U.S. face material |
| WO2018099405A1 (en) * | 2016-11-30 | 2018-06-07 | 京东方科技集团股份有限公司 | Human face resolution re-establishing method and re-establishing system, and readable medium |
| CN108776983A (en) * | 2018-05-31 | 2018-11-09 | 北京市商汤科技开发有限公司 | Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network |
| CN109284729A (en) * | 2018-10-08 | 2019-01-29 | 北京影谱科技股份有限公司 | Method, apparatus and medium based on video acquisition human face recognition model training data |
| CN109948555A (en) * | 2019-03-21 | 2019-06-28 | 于建岗 | Human face super-resolution recognition methods based on video flowing |
| CN109961451A (en) * | 2019-03-22 | 2019-07-02 | 西北工业大学 | A kind of material grains tissue segmentation methods based on marginal information |
| CN110136055A (en) * | 2018-02-02 | 2019-08-16 | 腾讯科技（深圳）有限公司 | Super-resolution method and device, storage medium, the electronic device of image |
| CN110188598A (en) * | 2019-04-13 | 2019-08-30 | 大连理工大学 | A kind of real-time hand Attitude estimation method based on MobileNet-v2 |
| CN110503606A (en) * | 2019-08-29 | 2019-11-26 | 广州大学 | A method of improving face clarity |
| CN110956599A (en) * | 2019-11-20 | 2020-04-03 | 腾讯科技（深圳）有限公司 | Picture processing method and device, storage medium and electronic device |
| CN110991310A (en) * | 2019-11-27 | 2020-04-10 | 北京金山云网络技术有限公司 | Portrait detection method, portrait detection device, electronic equipment and computer readable medium |
| CN111353943A (en) * | 2018-12-20 | 2020-06-30 | 杭州海康威视数字技术股份有限公司 | Face image recovery method and device and readable storage medium |
| WO2021115403A1 (en) * | 2019-12-13 | 2021-06-17 | 深圳市中兴微电子技术有限公司 | Image processing method and apparatus |
| CN114549307A (en) * | 2022-01-28 | 2022-05-27 | 电子科技大学 | High-precision point cloud color reconstruction method based on low-resolution image |
| CN115311477A (en) * | 2022-08-09 | 2022-11-08 | 北京惠朗时代科技有限公司 | Simulated trademark accurate detection method and system based on super-resolution reconstruction |
| WO2024042970A1 (en) * | 2022-08-26 | 2024-02-29 | ソニーグループ株式会社 | Information processing device, information processing method, and computer-readable non-transitory storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101299235A (en) * | 2008-06-18 | 2008-11-05 | 中山大学 | Method for reconstructing human face super resolution based on core principle component analysis |
| CN101710386A (en) * | 2009-12-25 | 2010-05-19 | 西安交通大学 | Super-resolution face recognition method based on relevant characteristic and non-liner mapping |
-
2011
- 2011-09-19 CN CN 201110278771 patent/CN102354397B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101299235A (en) * | 2008-06-18 | 2008-11-05 | 中山大学 | Method for reconstructing human face super resolution based on core principle component analysis |
| CN101710386A (en) * | 2009-12-25 | 2010-05-19 | 西安交通大学 | Super-resolution face recognition method based on relevant characteristic and non-liner mapping |
Non-Patent Citations (4)
| Title |
|---|
| JIANCHAO YANG ET AL: "Image Super-Resolution Via Sparse Representation", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》, vol. 19, no. 11, 30 November 2010 (2010-11-30), XP011328631, DOI: doi:10.1109/TIP.2010.2050625 * |
| T. CELIK ET AL: "Region-Based Super-Resolution Aided Facial Feature Extraction from Low-Resolution Video Sequences", 《IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING》, 23 March 2005 (2005-03-23) * |
| ZHIFEI WANG ET AL: "Feature-Based Super-Resolution for Face Recognition", 《IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO》, 26 June 2008 (2008-06-26) * |
| 黄丽 等: "基于多尺度和多方向特征的人脸超分辨率算法", 《计算机辅助设计与图形学学报》, vol. 16, no. 7, 31 July 2004 (2004-07-31) * |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629373A (en) * | 2012-02-27 | 2012-08-08 | 天津大学 | Super-resolution image acquisition method based on sparse representation theory |
| CN102629373B (en) * | 2012-02-27 | 2014-05-28 | 天津大学 | Super-resolution image acquisition method based on sparse representation theory |
| CN102800069A (en) * | 2012-05-22 | 2012-11-28 | 湖南大学 | Image super-resolution method for combining soft decision self-adaptation interpolation and bicubic interpolation |
| CN103020897B (en) * | 2012-09-28 | 2016-04-20 | 香港应用科技研究院有限公司 | Based on device, the system and method for the super-resolution rebuilding of the single-frame images of multi-tiling |
| CN103020897A (en) * | 2012-09-28 | 2013-04-03 | 香港应用科技研究院有限公司 | Device for reconstructing based on super-resolution of multi-block single-frame image, system and method thereof |
| CN102902966A (en) * | 2012-10-12 | 2013-01-30 | 大连理工大学 | Super-resolution face recognition method based on deep belief networks |
| CN103914807A (en) * | 2012-12-31 | 2014-07-09 | 北京大学 | Non-locality image super-resolution method and system for zoom scale compensation |
| CN103454225B (en) * | 2013-07-05 | 2016-04-06 | 中南大学 | Based on the copper floatation foam image regional area area measurement method of MPCA |
| CN103454225A (en) * | 2013-07-05 | 2013-12-18 | 中南大学 | MPCA (Multiway Principal Component Analysis)-based measurement method for area of local region of copper flotation froth image |
| CN106033529A (en) * | 2014-09-12 | 2016-10-19 | 宏达国际电子股份有限公司 | Image processing method and electronic apparatus |
| WO2018099405A1 (en) * | 2016-11-30 | 2018-06-07 | 京东方科技集团股份有限公司 | Human face resolution re-establishing method and re-establishing system, and readable medium |
| US11436702B2 (en) | 2017-11-29 | 2022-09-06 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for super-resolusion image reconstruction |
| CN107895345B (en) * | 2017-11-29 | 2020-05-26 | 浙江大华技术股份有限公司 | Method and device for improving resolution of face image |
| CN107895345A (en) * | 2017-11-29 | 2018-04-10 | 浙江大华技术股份有限公司 | A kind of method and apparatus for improving facial image resolution ratio |
| CN108121957A (en) * | 2017-12-19 | 2018-06-05 | 北京麒麟合盛网络技术有限公司 | The method for pushing and device of U.S. face material |
| CN108121957B (en) * | 2017-12-19 | 2021-09-03 | 麒麟合盛网络技术股份有限公司 | Method and device for pushing beauty material |
| CN110136055B (en) * | 2018-02-02 | 2023-07-14 | 腾讯科技（深圳）有限公司 | Super resolution method and device for image, storage medium and electronic device |
| CN110136055A (en) * | 2018-02-02 | 2019-08-16 | 腾讯科技（深圳）有限公司 | Super-resolution method and device, storage medium, the electronic device of image |
| CN108776983A (en) * | 2018-05-31 | 2018-11-09 | 北京市商汤科技开发有限公司 | Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network |
| CN109284729A (en) * | 2018-10-08 | 2019-01-29 | 北京影谱科技股份有限公司 | Method, apparatus and medium based on video acquisition human face recognition model training data |
| CN111353943B (en) * | 2018-12-20 | 2023-12-26 | 杭州海康威视数字技术股份有限公司 | Face image recovery method and device and readable storage medium |
| CN111353943A (en) * | 2018-12-20 | 2020-06-30 | 杭州海康威视数字技术股份有限公司 | Face image recovery method and device and readable storage medium |
| CN109948555A (en) * | 2019-03-21 | 2019-06-28 | 于建岗 | Human face super-resolution recognition methods based on video flowing |
| CN109948555B (en) * | 2019-03-21 | 2020-11-06 | 于建岗 | Face super-resolution identification method based on video stream |
| CN109961451A (en) * | 2019-03-22 | 2019-07-02 | 西北工业大学 | A kind of material grains tissue segmentation methods based on marginal information |
| CN110188598A (en) * | 2019-04-13 | 2019-08-30 | 大连理工大学 | A kind of real-time hand Attitude estimation method based on MobileNet-v2 |
| CN110188598B (en) * | 2019-04-13 | 2022-07-05 | 大连理工大学 | Real-time hand posture estimation method based on MobileNet-v2 |
| CN110503606B (en) * | 2019-08-29 | 2023-06-20 | 广州大学 | Method for improving face definition |
| CN110503606A (en) * | 2019-08-29 | 2019-11-26 | 广州大学 | A method of improving face clarity |
| CN110956599A (en) * | 2019-11-20 | 2020-04-03 | 腾讯科技（深圳）有限公司 | Picture processing method and device, storage medium and electronic device |
| CN110991310A (en) * | 2019-11-27 | 2020-04-10 | 北京金山云网络技术有限公司 | Portrait detection method, portrait detection device, electronic equipment and computer readable medium |
| CN110991310B (en) * | 2019-11-27 | 2023-08-22 | 北京金山云网络技术有限公司 | Portrait detection method, device, electronic equipment and computer readable medium |
| WO2021115403A1 (en) * | 2019-12-13 | 2021-06-17 | 深圳市中兴微电子技术有限公司 | Image processing method and apparatus |
| CN114549307A (en) * | 2022-01-28 | 2022-05-27 | 电子科技大学 | High-precision point cloud color reconstruction method based on low-resolution image |
| CN115311477A (en) * | 2022-08-09 | 2022-11-08 | 北京惠朗时代科技有限公司 | Simulated trademark accurate detection method and system based on super-resolution reconstruction |
| CN115311477B (en) * | 2022-08-09 | 2024-01-16 | 北京惠朗时代科技有限公司 | Super-resolution reconstruction-based simulated trademark accurate detection method and system |
| WO2024042970A1 (en) * | 2022-08-26 | 2024-02-29 | ソニーグループ株式会社 | Information processing device, information processing method, and computer-readable non-transitory storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102354397B (en) | 2013-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102354397B (en) | Method for reconstructing human facial image super-resolution based on similarity of facial characteristic organs | |
| CN110458939B (en) | Indoor scene modeling method based on visual angle generation | |
| CN101520894B (en) | Method for extracting significant object based on region significance | |
| CN109035172B (en) | Non-local mean ultrasonic image denoising method based on deep learning | |
| CN110599528A (en) | Unsupervised three-dimensional medical image registration method and system based on neural network | |
| CN106228528B (en) | A kind of multi-focus image fusing method based on decision diagram and rarefaction representation | |
| CN111798462A (en) | Automatic delineation method for nasopharyngeal carcinoma radiotherapy target area based on CT image | |
| CN110826389B (en) | Gait recognition method based on attention 3D frequency convolution neural network | |
| CN109977968B (en) | SAR change detection method based on deep learning classification comparison | |
| CN105913431A (en) | Multi-atlas dividing method for low-resolution medical image | |
| CN110648331B (en) | Detection method for medical image segmentation, medical image segmentation method and device | |
| CN113762147B (en) | Facial expression migration method and device, electronic equipment and storage medium | |
| CN106157249A (en) | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN114283285A (en) | Cross consistency self-training remote sensing image semantic segmentation network training method and device | |
| CN115147600A (en) | GBM multi-mode MR image segmentation method based on classifier weight converter | |
| CN104408731A (en) | Region graph and statistic similarity coding-based SAR (synthetic aperture radar) image segmentation method | |
| Luo et al. | Bi-GANs-ST for perceptual image super-resolution | |
| CN107392211A (en) | The well-marked target detection method of the sparse cognition of view-based access control model | |
| CN104732230A (en) | Pathology image local-feature extracting method based on cell nucleus statistical information | |
| CN114529551A (en) | Knowledge distillation method for CT image segmentation | |
| CN105069767A (en) | Image super-resolution reconstruction method based on representational learning and neighbor constraint embedding | |
| CN104123719B (en) | Method for carrying out infrared image segmentation by virtue of active outline | |
| Liu et al. | Fine-grained MRI reconstruction using attentive selection generative adversarial networks | |
| Yang et al. | Automatic brain tumor segmentation using cascaded FCN with DenseCRF and K-means |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130515 Termination date: 20150919 |
|
| EXPY | Termination of patent right or utility model |