CN101216889A - A face image super-resolution method with the amalgamation of global characteristics and local details information - Google Patents

A face image super-resolution method with the amalgamation of global characteristics and local details information Download PDF

Info

Publication number
CN101216889A
CN101216889A CNA2008100591307A CN200810059130A CN101216889A CN 101216889 A CN101216889 A CN 101216889A CN A2008100591307 A CNA2008100591307 A CN A2008100591307A CN 200810059130 A CN200810059130 A CN 200810059130A CN 101216889 A CN101216889 A CN 101216889A
Authority
CN
China
Prior art keywords
image
resolution
residual
sample
low resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA2008100591307A
Other languages
Chinese (zh)
Inventor
庄越挺
张剑
肖俊
吴飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CNA2008100591307A priority Critical patent/CN101216889A/en
Publication of CN101216889A publication Critical patent/CN101216889A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention discloses a face mage super-resolution method which fuses global features and local detail information. The invention can synthesize a high-resolution face image according to a low-resolution face image based on a sample image. Firstly, a local maintaining mapping algorithm and a radial basic function return algorithm are combined together to get a global high-resolution face image; then a neighborhood reconstruction method is adopted to synthesize a high-resolution face residual image block and consequently form a high-resolution face residual image by combination; finally, the high-resolution face residual image is overlapped to the high-resolution face image to obtain a final super-resolution effect. The technology provided by the invention can synthesize the clearer high-resolution face image, improve the recognition of the face image and have important application significances on video monitoring, face recognition and other aspects.

Description

The face image super-resolution method of a kind of amalgamation of global characteristics and local detail information
Technical field
The present invention relates to Digital Image Processing, relate in particular to the face image super-resolution method of a kind of amalgamation of global characteristics and local detail information.
Background technology
The human face super-resolution technology is the special image super-resolution technology of a class, present image super-resolution technology can be divided into two classes substantially, promptly based on the image super-resolution of rebuilding with based on the image super-resolution of learning, in general the latter is more more effective than the former.Occurred some representational image super-resolution technology based on study in recent years, the main thought of these methods is based on a sample image storehouse that comprises paired high resolving power and low-resolution image and carries out image super-resolution.People such as Freeman propose a kind of method based on sample, they are by the relation between markov network (Markov Network) study low-resolution image and the corresponding high-definition picture, and utilize the relation of learning that other low-resolution image is carried out super-resolution, this work is published on the IEEE international computer visual conference in 1999 (IEEE InternationalConference on Computer Vision, (1999) 1182~1189).People such as Hertzmann go up in the ACM of calendar year 2001 graphics meeting (ACM SIGGRAPH 2001) and propose a kind of general local feature conversion method, are called " image is analogized " (Image Analogies).They adopt multiple dimensioned autoregression (Multi-scaleAuto-regression) method study height-low-resolution image between local similar, and carry out image super-resolution on this basis.These methods are more suitable for handling the super-resolution problem of general pattern, because they do not consider the specific properties of facial image.Baker and Kanade announce " illusion of people's face " (Face Hallucination) technology that proposes first at IEEE in 2000 in moving face and gesture recognition international conference, they as priori, and adopt the Bayesian inference means to obtain the high-resolution human face image from the low resolution facial image space distribution of facial image gradient information.This method depends on very complicated probability model.Ce Liu (IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition.Kauai Marriott, Hawaii (2001) 192-198) in calendar year 2001 IEEE computer vision and pattern-recognition international conference has announced that a kind of two step people's face fantasy approacheses have reached same purpose.People such as Wang based on Feature Conversion (Eigen-transformation) algorithm development a kind of technology of human face super-resolution efficiently.They are based on principal component analysis (PCA) (PCA), approach the input facial image and find the solution combination coefficient with the linear combination of low resolution sample facial image.Keep these combination coefficients, and the low resolution facial image is replaced with high resolving power sample facial image, can synthesize the high-resolution human face image by linear combination.But this method is only considered the global image feature, and ignored local detail information, cause synthetic image unintelligible at regional area, lack minutia, (IEEE Trans.on Systems, Man, and Cybernetics on the IEEE transactions that this work appears at 2005, Part-C.2005,35 (3): 425~434).
In video surveillance applications, owing to video resolution is not high, people's face is crossed reasons such as far away apart from camera lens, the image resolution ratio of people face part is too low, and identification is too poor, and people's identification is caused difficulty.The human face super-resolution technology can go out the high-resolution human face image according to the facial image reasonable " derivation " of low resolution, increases the identification of people's face in the image, will be used widely in recognition of face and field of video monitoring.
Summary of the invention
The purpose of this invention is to provide the face image super-resolution method of a kind of amalgamation of global characteristics and local detail information, comprise the steps:
1), adopt the local mapping algorithm that keeps to set up converting vector according to the sample data of low resolution and high-resolution human face image;
2) adopt the local intrinsic characteristics that keeps mapping algorithm to extract low resolution sample facial image;
3) adopt radial basis function to return between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image, to set up related;
4) the low resolution facial image with input projects to and obtains intrinsic characteristics on the converting vector, with the input of this intrinsic characteristics as radial basis function, obtains the high-resolution human face image of the overall situation;
5) set up high resolving power and low resolution sample residual image block matrix according to sample image;
6) calculate the weight of rebuilding the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, high resolving power residual image piece is synthesized in weighting;
7) combination high resolving power residual image piece also carries out smoothly obtaining high-resolution residual error facial image with the linear smoothing operator;
8) the overall high-resolution human face image addition that high-resolution residual error facial image that step 7) is obtained and step 4) obtain obtains final super-resolution result.
The local maintenance of described employing mapping algorithm is set up converting vector and is comprised the steps:
1) establishing P is n panel height resolution sample facial image, P=p 1..., p n, dimension is m; Q is corresponding n width of cloth low resolution sample facial image, Q=q 1..., q n, dimension is d, adopts principal component analysis (PCA) that low resolution sample facial image Q is carried out dimensionality reduction, obtains proper vector U and characteristic coefficient V;
2) in the subspace that proper vector U represents, calculate any two width of cloth facial image q iAnd q jBetween distance, and be that every width of cloth image q selects k arest neighbors in sample space, structure reflects the neighborhood figure of data set local topology;
3) if facial image q iBe facial image q jOne of k neighbour or facial image q jBe facial image q iOne of k neighbour, weight is set to W Ij=‖ q i-q j2, otherwise W Ij=0;
4) formula of calculating converting vector is: QLQ Ta 1=λ QDQ Ta 1D wherein Ii=∑ jW Ji, and L=D-W is Laplce's matrix, establishing and finding the solution the eigenwert that obtains is λ i(l=1 ... L), use S = [ a 1 1 , . . . , a h 1 ] Expression and preceding h minimum eigenwert characteristic of correspondence vector, the converting vector that then original higher-dimension facial image is mapped in the low-dimensional stream shape space is expressed as A=US.
The described intrinsic characteristics that adopts local maintenance mapping algorithm extraction low resolution sample facial image: by formula y Tr=A TQ calculates the intrinsic characteristics y of sample low resolution facial image Tr, wherein Q is a low resolution sample facial image, A adopts the local converting vector that keeps mapping algorithm to set up.
Described employing radial basis function returns that to set up correlating method between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image as follows: the citation form of radial basis function is p j = Σ i = 1 n w j k ( y i tr , y j tr ) , Wherein k ( y i tr , y j tr ) = exp ( | | y i tr - y j tr | | 2 / 2 σ 2 ) , And constant σ can be calculated by following formula σ 2 = ( ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) - ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) ) n / nbs , Wherein n represents the number of sample image, and nbs represents the number of k arest neighbors, and the radial basis function of matrix form is expressed as P=WK, and wherein correlation coefficient is W= w 1 , . . . , w n , K = k ( y 1 tr , y 1 tr ) . . . k ( y 1 tr , y n tr ) . . . . . . . . . k ( y n tr , y 1 tr ) . . . k ( y n tr , y n tr ) , Adopt y I=1 ... n TrAnd P=p 1... p nTrain radial basis function and obtain correlation coefficient W.
The overall high-resolution human face image of described generation comprises the steps:
1) with the low resolution facial image q that imports InProject on the converting vector A, obtain q InCoordinate y in low-dimensional stream shape space In, y In=A Tq In
2) with y InAs the input data, according to k (y i Tr, y j Tr) compute matrix K, and calculate overall high-resolution human face image p according to P=WK Out
Describedly set up high resolving power and low resolution sample residual image block matrix method is as follows: establish I according to sample image lBe sample low resolution facial image, and I hBe corresponding overall high-resolution human face image, then residual error people's face R of low resolution lBe R l=I l-D (I h), wherein D () is the down-sampling function; High-resolution residual error people's face R hBe R h=I o-I h, I wherein oBe the original high resolution facial image, make I again lFor low resolution sample residual facial image, make I hBe high resolving power sample residual facial image, they are divided into the image fritter of similar number, each all satisfies one-to-one relationship to height-low-resolution image fritter; Definition P t l(i j) is I lIn low resolution residual error fritter, the center is v Ij l, P t h(i j) is I hIn high resolving power residual error fritter, the center is v Ij h, with P t l(i j) is fixed as n l* n lSize, n lBe odd number, with P t h(i j) is fixed as n h* n hSize, n lAnd n hBetween satisfy n h=λ n l, wherein λ is a zoom factor; I lOverlapping dimension between the middle low resolution fritter is made as (n l-1)/2, and I hOverlapping dimension between the middle high-resolution fritter is made as (odd (n h)-1)/2, wherein the effect of function odd (x) is the odd number that finds the maximum that is not more than x; In case determine v Ij l, P t l(i, j) be known, and P t h(i, position j) determines that also the coordinate of fritter central point can be calculated as follows:
v ij l = ( Σ a = 1 i k a , Σ b = 1 j k b ) v ij h = ( λ Σ a = 1 i k a - 1 , λ Σ b = 1 j k b - 1 ) = λv ij l - ( 1,1 )
k aAnd k bFor:
k a = ( n l + 1 ) / 2 mod ( a , 2 ) = 1 ( n l - 1 ) / 2 mod ( a , 2 ) = 0 k b = ( n l + 1 ) / 2 mod ( b , 2 ) = 1 ( n l - 1 ) / 2 mod ( b , 2 ) = 0
Wherein mod () is a modulo operation;
Every width of cloth residual error facial image is exactly a little block matrix of image, and the image fritter that is arranged in the capable j row of matrix i is expressed as P t(i, j).
The weight that calculating is rebuild the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, the synthetic high resolving power residual image piece of weighting comprises the steps:
1) the low resolution residual error facial image R that the overall high-resolution human face image of subduction behind the down-sampling obtains importing on the low-resolution image of input In l, with R In lBe divided into overlapped residual error fritter P t In(i, j), i=1 ..., r; J=1 ..., c, wherein r and c are respectively the line number and the columns of image block matrix, the initial value of i and j is 1;
2) if i>r or j>c, algorithm stops;
Otherwise, for current P t In(i, j), at P T (m) l(i finds its k neighbour based on Euclidean distance in j), m=1 ..., n, these k neighbor table are shown P T (k) l(i, j), k=1 ..., K, K≤n;
3) be P T (k) l(i j) calculates weight; This be by Σ k = 1 K w k = 1 Minimize P under the constraint t In(i, reconstruction error j) realizes that objective function is ϵ = | | P t in ( i , j ) - Σ k = 1 K w k P t ( k ) l ( i , j ) | | 2 , Definition C is C = P t in ( i , j ) · ones ( 1 , K ) - [ P t ( 1 ) l ( i , j ) , . . . , P t ( K ) l ( i , j ) ] , Wherein ones is that K element is 1 row vector, and then local covariance matrix G can be expressed as G=C TC is rebuild the weight w=(G of the low resolution residual image piece of input by the immediate low resolution sample residual image block of k -1Ones (K, 1))/(ones (K, 1) TG -1Ones (K, 1)), wherein w is the weight vectors of K dimension;
4) calculate high-resolution residual image piece based on w P t out ( i , j ) : P t out ( i , j ) = Σ k = 1 K w ( k ) P t ( k ) h ( i , j ) ;
5) if j<c then makes j=j+1; Otherwise make i=i+1 and j=1, change step 2).
Described generation high resolving power residual error facial image comprises the steps:
1) the residual image fritter is stacked up obtains initial high resolving power residual error facial image;
2) suppose R hBe the simple superposition of all residual image pieces, definition smoothing operator SMO is to R on each location of pixels h(x y) carries out smoothly, and SMO is defined as:
Figure S2008100591307D00053
1≤p≤r wherein, 1≤q≤c, r and c are respectively the line number and the columns of image block matrix, for R h(x, smooth operation y) is linear operation R h(x, y)=R h(x, y) SMO (x, y).
The beneficial effect that the present invention has:
It is 1) local that to keep mapping algorithm be linear, and as principal component analysis (PCA) explicit provide one group of converting vector.This shows that the local mapping algorithm that keeps can pass through a linear projection outer data of treatment samples notebook data collection easily, is better than principal component analytical method on generalization.
2) this method keeps mapping and radial basis function to organically combine the part.The local essential characteristic that keeps mapping to catch sample image, radial basis function return between image intrinsic characteristics and original image, set up related.Compare with linear principal component analysis (PCA), the overall high-resolution human face image that this algorithm synthesizes is more near real human face, and counting yield is very high.
Search strategy when 3) subordinate phase is sought kNN is " position is relevant ", and this just means that a sample data that only needs travel through on the same position gets final product when on search in the sample image piece and the ad-hoc location during the immediate data of input picture piece.This can not cause the decline of quality of human face image because appear at the feature that the facial image piece of same position roughly reflects the same position of people's face, and kNN also most probable appear in these sample block.In addition, the kNN algorithm is to be applied on the low-resolution image piece.These characteristics have greatly reduced the computation complexity of kNN search.
Description of drawings
Fig. 1 is that the part that has different parameters among the present invention keeps the overall high-resolution human face synoptic diagram of mapping algorithm generation;
Fig. 2 is the one-to-one relationship between middle high-resolution of the present invention and low-resolution image residual error fritter;
Fig. 3 is the overlapped situation synoptic diagram of four residual error fritters adjacent among the present invention;
Fig. 4 is the overlapped situation synoptic diagram of residual error fritter on the entire image among the present invention;
Fig. 5 is the final super-resolution efect synoptic diagram that the stack of overall high-resolution human face and high resolving power residual error facial image forms among the present invention;
Fig. 6 (a) is an original low-resolution image;
Fig. 6 (b) is the high-definition picture that obtains according to the direct interpolation of original low-resolution image;
Fig. 6 (c) is the high-definition picture that adopts the method for the invention to obtain;
Fig. 6 (d) is real high-definition picture;
Fig. 7 (a) is the low resolution original image of actual photographed in the stadium;
From left to right be followed successively by the high-resolution human face image that the original facial image of low resolution, interpolation obtain, the high-resolution human face image that adopts the method for the invention to obtain among Fig. 7 (b);
Fig. 8 (a) is the low resolution original image of actual photographed in the laboratory;
From left to right be followed successively by the high-resolution human face image that the original facial image of low resolution, interpolation obtain, the high-resolution human face image that adopts the method for the invention to obtain among Fig. 8 (b);
Embodiment
The face image super-resolution method of amalgamation of global characteristics and local detail information is implemented as follows:
1) adopts the local intrinsic characteristics that keeps mapping algorithm to extract sample low resolution facial image.We verify the method for the invention on Asian's face image data base.Facial image in the PF01 database is from 56 male sex and 51 women totally 107 volunteers, and everyone has 17 width of cloth to have the image (1 width of cloth front face, 4 width of cloth comprise illumination variation, 8 width of cloth comprise attitude to be changed, remaining 4 width of cloth have expression shape change) of different external appearance characteristics.In all volunteers, 24 male sex and 8 women have worn glasses.Most of volunteer's age is between 20 to 30 years old.Owing to the objective of the invention is under even illumination condition, to carry out the super-resolution of front face image, removal has the facial image that illumination and attitude change, and constructs a new data set that comprises 321 width of cloth images (for everyone keeps the image that 1 width of cloth direct picture and 2 width of cloth have typical expression shape change).All sample images have passed through preliminary registration, and underwriter's pupil roughly is positioned at the same position of image.On this basis, further the manual setting image resolution ratio is 96 * 128.If P=p 1..., p 60Be 60 panel height resolution sample facial images, dimension is 12288; Q=q 1..., q 60Be 60 corresponding width of cloth low resolution sample facial images, dimension is 768, adopts principal component analysis (PCA) that Q is carried out dimensionality reduction, obtains proper vector U and characteristic coefficient V; In the subspace that U represents, calculate any two width of cloth facial image q iAnd q jBetween distance, and be that every width of cloth image q selects 30 arest neighbors in sample space, the neighborhood figure of structure reflection data set local topology; Weight is set: if q iBe q jOne of k neighbour or q jBe q iOne of k neighbour, weight is set to W Ij=‖ q i-q j2, otherwise W Ij=0; Calculate converting vector: the eigenvalue problem of separating following popularization: QLQ Ta 1=λ QDQ Ta 1, D wherein Ii=∑ jW Ji, and L=D-W is Laplce's matrix, establishing and finding the solution the eigenwert that obtains is λ l(l=1 ... L), use S = [ a 1 1 , . . . , a 50 1 ] Expression and minimum preceding 50 eigenwert characteristic of correspondence vectors, the converting vector that then original higher-dimension facial image is mapped in the low-dimensional stream shape space can be expressed as A=US.The intrinsic characteristics y of sample low resolution facial image TrCan be calculated as y Tr=A TQ.
2) adopt the intrinsic characteristics of low resolution sample image and corresponding high resolving power sample image training radial basis function.The citation form of RBF is p j = Σ i = 1 n w j k ( y i tr , y j tr ) , Wherein k ( y i tr , y j tr ) = exp ( | | y i tr - y j tr | | 2 / 2 σ 2 ) , And constant σ can be calculated by following formula σ 2 = ( ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) - ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) ) n / nbs , Wherein n represents the number of sample image, is 60 here, and nbs represents the number of defined arest neighbors in the step 2, is 30 here.The RBF of matrix form is expressed as P=WK, wherein W = [ w 1 , . . . , w n ] , K = k ( y 1 tr , y 1 tr ) . . . k ( y 1 tr , y n tr ) . . . . . . . . . k ( y n tr , y 1 tr ) . . . k ( y n tr , y n tr ) , Adopt y i Tr(i=1 ... 60) and P=[p 1... p 60] train RBF and obtain W.
3) the low resolution facial image with input is projected to converting vector.The dimension of low resolution facial image is 768,50 converting vectors have been extracted altogether, then converting vector constitutes one 768 * 50 matrix, the low resolution facial image of input is one 1 * 768 vector, obtain one 1 * 50 vector after this vector and the converting vector matrix operation, be the intrinsic characteristics of input low resolution facial image, top computing can be expressed as y In=A Tq In, wherein A is a converting vector, q InBe the low resolution facial image of input, y InIt is intrinsic characteristics.
4) utilize radial basis function to return and obtain overall high-resolution human face image.If overall high-resolution human face image p OutDimension be N, p OutCan be according to y InObtain with the W computing that obtains previously, formula is p Out=y InW, wherein W is 50 * 12288 matrixes.Represent respectively that with k and h every width of cloth facial image keeps the neighbour's number in the algorithm and the number of converting vector in the part.We have tested this algorithm on a sample data collection that comprises 75 width of cloth images, synthetic overall high-resolution human face as shown in Figure 1 under different k and h.First the row in image be real high-resolution human face, for second and third, four lines, the neighborhood size is appointed as 5,15 and 30 respectively.To each k, calculate the converting vector of from 10 to 70 different numbers respectively, thereby form people's face matrix of one 3 * 7.Fig. 1 shows that different k will make synthetic facial image that very big difference is arranged when converting vector seldom the time; And when the converting vector number increases gradually, algorithm will converge to the value of an optimization, and the result who obtains under different k reaches unanimity.This explanation is compared with the neighborhood size, and the number of converting vector is The key factor more for the LPH algorithm.Work as k=30, obtain optimal result during h=70.
5) synthesize the sample residual image by image being carried out down-sampling and difference, and further residual image is divided into the image fritter.If I lBe sample low resolution facial image, and I hBe corresponding overall high-resolution human face image, then residual error people's face R of low resolution lBe R l=I l-D (I h), wherein D () is the down-sampling function, with image from 96 * 128 down-samplings to 24 * 32; High-resolution residual error people's face R hBe R h=I o-I h, I wherein oBe the original high resolution facial image, make I again lFor low resolution sample residual facial image, make I hBe high resolving power sample residual facial image, they are divided into the image fritter of similar number, each all satisfies one-to-one relationship to height-low-resolution image fritter, and the one-to-one relationship between high resolving power and low-resolution image fritter as shown in Figure 2; Definition P t l(i j) is I lIn low resolution residual error fritter, the center is v Ij l, P t h(i j) is I hIn high resolving power residual error fritter, the center is v Ij h, with P t l(i j) is fixed as n l* n lSize, n lBe made as 3, with P h t(i j) is fixed as n h* n hSize, n hBe made as 12, n lAnd n hBetween satisfy n h=λ n l, wherein λ is a zoom factor, is 4 here; I lOverlapping dimension between the middle low resolution fritter is made as (n l-1)/2=1, and I hOverlapping dimension between the middle high-resolution fritter is made as (odd (n h)-1)/and 2=5, wherein the effect of function odd (x) is the odd number that finds the maximum that is not more than x; In case determine v Ij l, P t l(i, j) be known, and P t h(i, position j) can uniquely determine that also the coordinate of fritter central point can be calculated as follows:
v ij l = ( Σ a = 1 i k a , Σ b = 1 j k b ) v ij h = ( λ Σ a = 1 i k a - 1 , λ Σ b = 1 j k b - 1 ) = λv ij l - ( 1,1 )
k aAnd k bFor:
k a = ( n l + 1 ) / 2 = 2 mod ( a , 2 ) = 1 ( n l - 1 ) / 2 = 2 mod ( a , 2 ) = 0 k b = ( n l + 1 ) / 2 = 2 mod ( b , 2 ) = 1 ( n l - 1 ) / 2 = 1 mod ( b , 2 ) = 0
Wherein mod () is a modulo operation;
6) low-resolution image according to input extracts residual image, and this residual image is divided into fritter, and searches for 30 neighbours in low resolution sample image fritter.The residual image that extracts input picture is to finish by following method: with the D () in the 5th step synthetic high resolving power global image is carried out down-sampling, again with the image behind the low-resolution image subduction down-sampling of input.Regard every width of cloth residual error facial image as a little block matrix of image, the image fritter that is arranged in the capable j row of matrix i is expressed as P t(i, j), each fritter comprises n * n pixel, and each fritter and it is of a size of (n-1)/2 in four adjacent isles overlapping regions up and down, and the little block structure of overlapped image is as shown in Figure 3.If P T (k) l(i j) is the low resolution residual error fritter P of input t In(i, j) 30 arest neighbors in sample set, P T (k) l(i j) can directly calculate by Euclidean distance.
7) calculate by 30 arest neighbors P T (k) l(i j) rebuilds P t In(i, weight w j) 1... w 30One-to-one relationship according to low resolution residual image piece and high resolving power residual image interblock replaces with corresponding high resolving power residual image piece with 30 immediate low resolution residual image pieces, with the synthetic corresponding high resolving power residual image piece P of same weight t Out(i, j).Be P T (k) l(i, j) calculate weight be by Σ k = 1 30 w k = 1 Minimize P under the constraint t In(i, reconstruction error j) realizes that objective function is ϵ = | | P t in ( i , j ) - Σ k = 1 30 w k P t ( k ) l ( i , j ) | | 2 , This is a least square problem based on constraint, and definition C is C = P t in ( i , j ) · ones ( 1,30 ) - [ P t ( 1 ) l ( i , j ) , . . . , P t ( 30 ) l ( i , j ) ] , Wherein ones is that 30 elements are 1 row vector, and then local covariance matrix G can be expressed as G=C TC, the separating of least square problem based on constraint is w=(G above -1Ones (30,1))/(ones (30,1) TG -1Ones (30,1)), wherein w is the weight vectors of 30 dimensions.Calculate high-resolution residual image piece P based on w t Out(i, j): P t out ( i , j ) = Σ k = 1 30 w ( k ) P t ( k ) h ( i , j ) ;
8) overlapped high resolving power residual image piece is integrated, form the high resolving power residual image of view picture, and adopt smoothing operator that image is carried out smoothly, form final super-resolution efect with overall high-resolution human face image overlay again.
Synthetic residual error fritter overlaps each other, and has brought redundant image high-frequency characteristic, and this can make synthetic people's face seem more sharp-pointed.We propose a linear smoothing operator according to residual error fritter structure and address this problem.Suppose R hBe the simple superposition of all residual image pieces, (x y) goes up definition smoothing operator SMO to R at each location of pixels h(x y) carries out smoothly, and SMO is defined as:
1≤p≤r wherein, 1≤q≤c, r and c are respectively the line number and the columns of image block matrix, obviously SMO and R hHas identical dimension.For R h(x, smooth operation y) is a simple linear operation R h(x, y)=R h(x, y) SMO (x, y).According to this linear smoothing operator, for the overlapping location of pixels of several adjacent isles, smoothly the pixel value after is the algebraic mean of these fritter respective pixel values.Overlapped residual error fritter can be described by Fig. 4.According to image block matrix, R hThe zone of Oxford gray is covered simultaneously by 4 residual error fritters, and light gray areas is then covered by 2 residual error fritters.
High resolving power residual error people face after level and smooth and the overall high-resolution human face image overlay that had before obtained can be obtained final super-resolution result, as shown in Figure 5.
In order to verify method of the present invention, we provide two groups of super-resolution examples.Promptly carry out super-resolution at positive unscreened facial image and actual photographed image in the database respectively.
Embodiment 1
Positive nothing is blocked the super-resolution embodiment of facial image in the database:
Purpose is the high-resolution human face that generates correspondence according to the low resolution front face image of the neutral expression of a width of cloth.Select 75 not have wearing spectacles in database among 107 volunteers, as the experimental data collection, wherein 60 width of cloth images are used for synthetic sample set with their front face image, and 15 width of cloth images are used as test data.At first with 60 96 * 128 high-resolution human face image down sampling to 24 * 32, with these 60 high-low resolution facial images to as sample data.
The territory size is more crucial near the number of converting vector in part maintenance mapping algorithm, the neighborhood size is fixed as 30 here, and the number with converting vector is set at 50 simultaneously.In the kNN of residual error fritter composition algorithm search, the number of neighbour's fritter is set at 30 equally.Carry out residual block when synthetic, the sample residual image of height-low resolution is divided into same number of residual error fritter respectively, the low resolution tile size is 3 * 3, and the high resolving power tile size is 12 * 12.In this group experiment, use 50 converting vectors just can obtain very desirable overall facial image.The residual error facial image has contained the image high-frequency information, is used for compensating the minutia of overall people's face, the not only clear but also very approaching real image of final result, and super-resolution result is as shown in Figure 6.Among Fig. 6, (a) be the low resolution facial image of input, (b) the super-resolution result for obtaining by simple bicubic interpolation is the super-resolution result who obtains by the method for the invention (c), (d) is real high-resolution human face image.
Embodiment 2
The super-resolution embodiment of actual photographed image:
For further verifying the effect of the method for the invention, we carry out super-resolution on the real scene shooting image.Still adopt 60 described in the example 1 high-the low resolution facial image is to as sample data, it is local that to keep the converting vector number in the mapping algorithm be 50, the neighborhood size is 30.Carry out the image residual block when synthetic, the low resolution tile size is 3 * 3, and the high resolving power tile size is 12 * 12, and the number of neighbour's fritter is set at 30 equally.
Fig. 7 (a) is the low resolution facial image that a width of cloth is taken with mobile phone in the stadium, manual extraction human face region and carry out super-resolution on this image with diverse ways, and the result is shown in Fig. 7 (b).The result that three width of cloth images among Fig. 7 (b) from left to right are respectively the result of original low-resolution image, cubic B-spline interpolation, obtain with the method for the invention.
Fig. 8 (a) is the image of a width of cloth in indoor random shooting, and human face region wherein is very little.The super-resolution result who adopts above two kinds of methods shown in Fig. 8 (b), the result that three width of cloth images among Fig. 8 (b) from left to right are respectively the result of original low-resolution image, cubic B-spline interpolation, obtain with the method for the invention.

Claims (8)

1. the face image super-resolution method of amalgamation of global characteristics and local detail information is characterized in that comprising the steps;
1), adopt the local mapping algorithm that keeps to set up converting vector according to the sample data of low resolution and high-resolution human face image;
2) adopt the local intrinsic characteristics that keeps mapping algorithm to extract low resolution sample facial image;
3) adopt radial basis function to return between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image, to set up related;
4) the low resolution facial image with input projects to and obtains intrinsic characteristics on the converting vector, with the input of this intrinsic characteristics as radial basis function, obtains the high-resolution human face image of the overall situation;
5) set up high resolving power and low resolution sample residual image block matrix according to sample image;
6) calculate the weight of rebuilding the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, high resolving power residual image piece is synthesized in weighting;
7) combination high resolving power residual image piece also carries out smoothly obtaining high-resolution residual error facial image with the linear smoothing operator;
8) the overall high-resolution human face image addition that high-resolution residual error facial image that step 7) is obtained and step 4) obtain obtains final super-resolution result.
2. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that described employing part keeps mapping algorithm to set up converting vector and comprises the steps:
1) establishing P is n panel height resolution sample facial image, P=p 1..., p n, dimension is m; Q is corresponding n width of cloth low resolution sample facial image, Q=q 1..., q n, dimension is d, adopts principal component analysis (PCA) that low resolution sample facial image Q is carried out dimensionality reduction, obtains proper vector U and characteristic coefficient V;
2) in the subspace that proper vector U represents, calculate any two width of cloth facial image q iAnd q jBetween distance, and be that every width of cloth image q selects k arest neighbors in sample space, structure reflects the neighborhood figure of data set local topology;
3) if facial image q iBe facial image q jOne of k neighbour or facial image q jBe facial image q iOne of k neighbour, weight is set to W Ij=‖ q i-q j2, otherwise W Ij=0;
4) formula of calculating converting vector is: QLQ Ta 1=λ QDQ Ta 1
D wherein Ii=∑ jW Ji, and L=D-W is Laplce's matrix, establishing and finding the solution the eigenwert that obtains is λ l(l=1 ... L), use S = [ a 1 1 , . . . , a h 1 ] Expression and preceding h minimum eigenwert characteristic of correspondence vector, the converting vector that then original higher-dimension facial image is mapped in the low-dimensional stream shape space is expressed as A=US.
3. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and 2 and local detail information is characterized in that the described local intrinsic characteristics that keeps mapping algorithm to extract low resolution sample facial image that adopts: by formula y Tr=A TQ calculates the intrinsic characteristics y of sample low resolution facial image Tr, wherein Q is a low resolution sample facial image, A adopts the local converting vector that keeps mapping algorithm to set up.
4. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information, it is characterized in that described employing radial basis function returns that to set up correlating method between the intrinsic characteristics of low resolution facial image and corresponding high-resolution human face image as follows: the citation form of radial basis function is p j = Σ i = 1 n w j k ( y i tr , y j tr ) , Wherein k ( y i tr , y j tr ) = exp ( | | y i tr - y j tr | | 2 / 2 σ 2 ) , And constant σ can be calculated by following formula σ 2 = ( ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) - ma x i = 1 , . . . , n j = 1 , . . . , n K ( i , j ) ) n / nbs , Wherein n represents the number of sample image, and nbs represents the number of k arest neighbors, and the radial basis function of matrix form is expressed as P=WK, and wherein correlation coefficient is W = w 1 , . . . , w n , K = k ( y 1 tr , y 1 tr ) . . . k ( y 1 tr , y n tr ) . . . . . . . . . k ( y n tr , y 1 tr ) . . . k ( y n tr , y n tr ) , Adopt y I=1 ... n TrAnd P=p 1... p nTrain radial basis function and obtain correlation coefficient W.
5. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that the overall high-resolution human face image of described generation comprises the steps:
1) with the low resolution facial image q that imports InProject on the converting vector A, obtain q InCoordinate y in low-dimensional stream shape space In, y In=A Tq In
2) with y InAs the input data, according to k (y i Tr, y j Tr) compute matrix K, and calculate overall high-resolution human face image p according to P=WK Out
6. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that describedly setting up high resolving power and low resolution sample residual image block matrix method is as follows according to sample image: establish I lBe sample low resolution facial image, and I hBe corresponding overall high-resolution human face image, then residual error people's face R of low resolution lBe R l=I l-D (I h), wherein D () is the down-sampling function; High-resolution residual error people's face R hBe R h=I o-I h, I wherein oBe the original high resolution facial image, make I again lFor low resolution sample residual facial image, make I hBe high resolving power sample residual facial image, they are divided into the image fritter of similar number, each all satisfies one-to-one relationship to height-low-resolution image fritter; Definition P t l(i j) is I lIn low resolution residual error fritter, the center is v Ij l, P t h(i j) is I hIn high resolving power residual error fritter, the center is v Ij h, with P t l(i j) is fixed as n l* n lSize, n lBe odd number, with P t h(i j) is fixed as n h* n hSize, n lAnd n hBetween satisfy n h=λ n l, wherein λ is a zoom factor; I lOverlapping dimension between the middle low resolution fritter is made as (n l-1)/2, and I hOverlapping dimension between the middle high-resolution fritter is made as (odd (n h)-1)/2, wherein the effect of function odd (x) is the odd number that finds the maximum that is not more than x; In case determine v Ij l, P t l(i, j) be known, and P t h(i, position j) determines that also the coordinate of fritter central point can be calculated as follows:
v ij l = ( Σ a = 1 i k a , Σ b = 1 j k b ) V ij h = ( λ Σ a = 1 i k a - 1 , λ Σ b = 1 j k b - 1 ) = λv ij l - ( 1,1 )
k aAnd k bFor:
k a = ( n l + 1 ) / 2 mod ( a , 2 ) = 1 ( n l - 1 ) / 2 mod ( a , 2 ) = 0
Wherein mod () is a modulo operation;
Every width of cloth residual error facial image is exactly a little block matrix of image, and the image fritter that is arranged in the capable j row of matrix i is expressed as P t(i, j).
7. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information, it is characterized in that calculating the weight of rebuilding the low resolution residual image piece of input by k immediate low resolution sample residual image block, then low resolution residual image piece is replaced with high resolving power sample residual image block, the synthetic high resolving power residual image piece of weighting comprises the steps:
1) the low resolution residual error facial image R that the overall high-resolution human face image of subduction behind the down-sampling obtains importing on the low-resolution image of input In l, with R In lBe divided into overlapped residual error fritter P t In(i, j), i=1 ..., r; J=1 ..., c, wherein r and c are respectively the line number and the columns of image block matrix, the initial value of i and j is 1;
2) if i>r or j>c, algorithm stops;
Otherwise, for current P t In(i, j), at P T (m) l(i finds its k neighbour based on Euclidean distance in j), m=1 ..., n, these k neighbor table are shown P T (k) l(i, j), k=1 ..., K, K≤n;
3) be P T (k) l(i j) calculates weight; This be by Σ k = 1 K w k = 1 Minimize P under the constraint t In(i, reconstruction error j) realizes that objective function is ϵ = | | P t in ( i , j ) - Σ k = 1 K w k P t ( k ) l ( i , j ) | | 2 , Definition C is C = P t in ( i , j ) · ones ( 1 , K ) - [ P t ( 1 ) l ( i , j ) , . . . , P t ( K ) l ( i , j ) ] , Wherein ones is that K element is 1 row vector, and then local covariance matrix G can be expressed as G=C TC is rebuild the weight w=(G of the low resolution residual image piece of input by the immediate low resolution sample residual image block of k -1Ones (K, 1))/(ones (K, 1) TG -1Ones (K, 1)), wherein w is the weight vectors of K dimension;
4) calculate high-resolution residual image piece based on w P t out ( i , j ) : P t out ( i , j ) = Σ k = 1 K w ( k ) P t ( k ) h ( i , j ) ;
5) if j<c then makes j=j+1; Otherwise make i=i+1 and j=1, change step 2).
8. the face image super-resolution method of a kind of amalgamation of global characteristics according to claim 1 and local detail information is characterized in that described generation high resolving power residual error facial image comprises the steps:
1) the residual image fritter is stacked up obtains initial high resolving power residual error facial image;
2) suppose R hBe the simple superposition of all residual image pieces, definition smoothing operator SMO is to R on each location of pixels h(x y) carries out smoothly, and SMO is defined as:
1≤p≤r wherein, 1≤q≤c, r and c are respectively the line number and the columns of image block matrix, for R h(x, smooth operation y) is linear operation R h(x, y)=R h(x, y) SMO (x, y).
CNA2008100591307A 2008-01-14 2008-01-14 A face image super-resolution method with the amalgamation of global characteristics and local details information Pending CN101216889A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNA2008100591307A CN101216889A (en) 2008-01-14 2008-01-14 A face image super-resolution method with the amalgamation of global characteristics and local details information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNA2008100591307A CN101216889A (en) 2008-01-14 2008-01-14 A face image super-resolution method with the amalgamation of global characteristics and local details information

Publications (1)

Publication Number Publication Date
CN101216889A true CN101216889A (en) 2008-07-09

Family

ID=39623320

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA2008100591307A Pending CN101216889A (en) 2008-01-14 2008-01-14 A face image super-resolution method with the amalgamation of global characteristics and local details information

Country Status (1)

Country Link
CN (1) CN101216889A (en)

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101799919A (en) * 2010-04-08 2010-08-11 西安交通大学 Front face image super-resolution rebuilding method based on PCA alignment
CN102110303A (en) * 2011-03-10 2011-06-29 西安电子科技大学 Method for compounding face fake portrait\fake photo based on support vector return
CN102298775A (en) * 2010-06-24 2011-12-28 财团法人工业技术研究院 Super-resolution method and system for human face based on sample
CN102298703A (en) * 2011-04-20 2011-12-28 中科院成都信息技术有限公司 Classification method based on projection residual errors
CN102306374A (en) * 2011-08-30 2012-01-04 西安交通大学 Method for rebuilding super-resolution human face image by position block nonlinear mapping
CN101739676B (en) * 2009-12-04 2012-02-22 清华大学 Method for manufacturing face effigy with ultra-low resolution
CN101515328B (en) * 2008-12-18 2012-05-09 东华大学 Local projection preserving method for identification of statistical noncorrelation
CN101719266B (en) * 2009-12-25 2012-05-23 西安交通大学 Affine transformation-based frontal face image super-resolution reconstruction method
CN101609503B (en) * 2009-07-16 2012-07-25 复旦大学 Face super-resolution image processing method based on double-manifold alignment
CN102693416A (en) * 2012-05-09 2012-09-26 浙江捷尚视觉科技有限公司 Highly noise-resistant single face automatic super-resolution method
CN102693419A (en) * 2012-05-24 2012-09-26 武汉大学 Super-resolution face recognition method based on multi-manifold discrimination and analysis
CN102867191A (en) * 2012-09-04 2013-01-09 广东群兴玩具股份有限公司 Dimension reducing method based on manifold sub-space study
CN102880878A (en) * 2011-07-15 2013-01-16 富士通株式会社 Single-image-based super-resolution analytical method and system
US8488913B2 (en) 2010-06-14 2013-07-16 Industrial Technology Research Institute Method and system for example-based face hallucination
CN103236041A (en) * 2013-04-25 2013-08-07 北京交通大学 Image super resolution reconstruction method on basis of Contourlet transformation
CN103489174A (en) * 2013-10-08 2014-01-01 武汉大学 Human face super-resolution method based on residual keeping
CN103514432A (en) * 2012-06-25 2014-01-15 诺基亚公司 Method, device and computer program product for extracting facial features
CN103824272A (en) * 2014-03-03 2014-05-28 武汉大学 Face super-resolution reconstruction method based on K-neighboring re-recognition
CN104318219A (en) * 2014-10-31 2015-01-28 上海交通大学 Face recognition method based on combination of local features and global features
CN104471589A (en) * 2012-07-11 2015-03-25 皇家飞利浦有限公司 Patient interface identification system
CN104966090A (en) * 2015-07-21 2015-10-07 公安部第三研究所 Visual word generation and evaluation system and method for realizing image comprehension
CN106204451A (en) * 2016-07-08 2016-12-07 西安电子科技大学 The Image Super-resolution Reconstruction method embedded based on the fixing neighborhood of constraint
CN107111786A (en) * 2014-11-13 2017-08-29 瑞穗情报综研株式会社 System, method and program for information of forecasting
WO2017161710A1 (en) * 2016-03-21 2017-09-28 京东方科技集团股份有限公司 Resolving method and system based on deep learning
CN107590775A (en) * 2017-09-21 2018-01-16 大连理工大学 A kind of image super-resolution amplification method using regression tree field
CN107644455A (en) * 2017-10-12 2018-01-30 北京旷视科技有限公司 Face image synthesis method and apparatus
CN107845127A (en) * 2017-12-02 2018-03-27 天津浩宝丰科技有限公司 A kind of human face cartoon animation image design method
CN107886568A (en) * 2017-12-09 2018-04-06 东方梦幻文化产业投资有限公司 A kind of method and system that human face expression is rebuild using 3D Avatar
CN108200334A (en) * 2017-12-28 2018-06-22 广东欧珀移动通信有限公司 Image capturing method, device, storage medium and electronic equipment
CN108446601A (en) * 2018-02-27 2018-08-24 东南大学 A kind of face identification method based on sound Fusion Features
CN108765291A (en) * 2018-05-29 2018-11-06 天津大学 Super resolution ratio reconstruction method based on dense neural network and two-parameter loss function
CN108846817A (en) * 2018-06-22 2018-11-20 Oppo(重庆)智能科技有限公司 Image processing method, device and mobile terminal
CN110210336A (en) * 2019-05-16 2019-09-06 赣南师范大学 A kind of low resolution single sample face recognition method
WO2019174522A1 (en) * 2018-03-16 2019-09-19 华为技术有限公司 Image generating method and device
CN110503606A (en) * 2019-08-29 2019-11-26 广州大学 A method of improving face clarity
CN110580482A (en) * 2017-11-30 2019-12-17 腾讯科技(深圳)有限公司 Image classification model training, image classification and personalized recommendation method and device
CN110910310A (en) * 2019-10-25 2020-03-24 南京大学 Face image reconstruction method based on identity information
CN111260558A (en) * 2020-01-22 2020-06-09 武汉大学 Image super-resolution network model with variable magnification
CN111931692A (en) * 2020-08-31 2020-11-13 青岛聚看云科技有限公司 Display device and image recognition method
CN112150615A (en) * 2020-09-24 2020-12-29 四川川大智胜软件股份有限公司 Face image generation method and device based on three-dimensional face model and storage medium
CN112581362A (en) * 2019-09-27 2021-03-30 美敦力公司 Image processing method and device for adjusting image details
WO2022006693A1 (en) * 2020-07-06 2022-01-13 Polycom Communications Technology (Beijing) Co. Ltd. Videoconferencing systems with facial image rectification
US11308636B2 (en) 2018-04-28 2022-04-19 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method, apparatus, and computer-readable storage medium for obtaining a target image
CN117952833A (en) * 2023-10-30 2024-04-30 中国科学院长春光学精密机械与物理研究所 Hyper-spectral image super-resolution reconstruction system and method based on three-branch network

Cited By (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101515328B (en) * 2008-12-18 2012-05-09 东华大学 Local projection preserving method for identification of statistical noncorrelation
CN101609503B (en) * 2009-07-16 2012-07-25 复旦大学 Face super-resolution image processing method based on double-manifold alignment
CN101739676B (en) * 2009-12-04 2012-02-22 清华大学 Method for manufacturing face effigy with ultra-low resolution
CN101719266B (en) * 2009-12-25 2012-05-23 西安交通大学 Affine transformation-based frontal face image super-resolution reconstruction method
CN101799919A (en) * 2010-04-08 2010-08-11 西安交通大学 Front face image super-resolution rebuilding method based on PCA alignment
US8488913B2 (en) 2010-06-14 2013-07-16 Industrial Technology Research Institute Method and system for example-based face hallucination
CN102298775A (en) * 2010-06-24 2011-12-28 财团法人工业技术研究院 Super-resolution method and system for human face based on sample
CN102110303B (en) * 2011-03-10 2012-07-04 西安电子科技大学 Method for compounding face fake portrait\fake photo based on support vector return
CN102110303A (en) * 2011-03-10 2011-06-29 西安电子科技大学 Method for compounding face fake portrait\fake photo based on support vector return
CN102298703A (en) * 2011-04-20 2011-12-28 中科院成都信息技术有限公司 Classification method based on projection residual errors
CN102298703B (en) * 2011-04-20 2015-06-17 中科院成都信息技术股份有限公司 Classification method based on projection residual errors
CN102880878A (en) * 2011-07-15 2013-01-16 富士通株式会社 Single-image-based super-resolution analytical method and system
CN102880878B (en) * 2011-07-15 2015-05-06 富士通株式会社 Single-image-based super-resolution analytical method and system
CN102306374A (en) * 2011-08-30 2012-01-04 西安交通大学 Method for rebuilding super-resolution human face image by position block nonlinear mapping
CN102306374B (en) * 2011-08-30 2013-02-06 西安交通大学 Method for rebuilding super-resolution human face image by position block nonlinear mapping
CN102693416A (en) * 2012-05-09 2012-09-26 浙江捷尚视觉科技有限公司 Highly noise-resistant single face automatic super-resolution method
CN102693416B (en) * 2012-05-09 2014-03-12 浙江捷尚视觉科技有限公司 Highly noise-resistant single face automatic super-resolution method
CN102693419B (en) * 2012-05-24 2014-02-26 武汉大学 Super-resolution face recognition method based on multi-manifold discrimination and analysis
CN102693419A (en) * 2012-05-24 2012-09-26 武汉大学 Super-resolution face recognition method based on multi-manifold discrimination and analysis
CN103514432A (en) * 2012-06-25 2014-01-15 诺基亚公司 Method, device and computer program product for extracting facial features
US9710698B2 (en) 2012-06-25 2017-07-18 Nokia Technologies Oy Method, apparatus and computer program product for human-face features extraction
CN104471589A (en) * 2012-07-11 2015-03-25 皇家飞利浦有限公司 Patient interface identification system
CN102867191A (en) * 2012-09-04 2013-01-09 广东群兴玩具股份有限公司 Dimension reducing method based on manifold sub-space study
CN103236041B (en) * 2013-04-25 2015-11-18 北京交通大学 A kind of image super-resolution rebuilding method based on contourlet transformation
CN103236041A (en) * 2013-04-25 2013-08-07 北京交通大学 Image super resolution reconstruction method on basis of Contourlet transformation
CN103489174A (en) * 2013-10-08 2014-01-01 武汉大学 Human face super-resolution method based on residual keeping
CN103824272B (en) * 2014-03-03 2016-08-17 武汉大学 The face super-resolution reconstruction method heavily identified based on k nearest neighbor
CN103824272A (en) * 2014-03-03 2014-05-28 武汉大学 Face super-resolution reconstruction method based on K-neighboring re-recognition
CN104318219A (en) * 2014-10-31 2015-01-28 上海交通大学 Face recognition method based on combination of local features and global features
CN104318219B (en) * 2014-10-31 2017-10-31 上海交通大学 The face identification method combined based on local feature and global characteristics
CN107111786A (en) * 2014-11-13 2017-08-29 瑞穗情报综研株式会社 System, method and program for information of forecasting
CN107111786B (en) * 2014-11-13 2021-02-12 瑞穗情报综研株式会社 System, method, and computer program product for predicting information
CN104966090A (en) * 2015-07-21 2015-10-07 公安部第三研究所 Visual word generation and evaluation system and method for realizing image comprehension
CN104966090B (en) * 2015-07-21 2018-08-14 公安部第三研究所 Realize the system and method that the vision word towards image understanding is generated and evaluated
WO2017161710A1 (en) * 2016-03-21 2017-09-28 京东方科技集团股份有限公司 Resolving method and system based on deep learning
US10769758B2 (en) 2016-03-21 2020-09-08 Boe Technology Group Co., Ltd. Resolving method and system based on deep learning
CN106204451A (en) * 2016-07-08 2016-12-07 西安电子科技大学 The Image Super-resolution Reconstruction method embedded based on the fixing neighborhood of constraint
CN106204451B (en) * 2016-07-08 2019-04-23 西安电子科技大学 Based on the Image Super-resolution Reconstruction method for constraining fixed neighborhood insertion
CN107590775A (en) * 2017-09-21 2018-01-16 大连理工大学 A kind of image super-resolution amplification method using regression tree field
CN107590775B (en) * 2017-09-21 2020-04-07 大连理工大学 Image super-resolution amplification method using regression tree field
CN107644455B (en) * 2017-10-12 2022-02-22 北京旷视科技有限公司 Face image synthesis method and device
CN107644455A (en) * 2017-10-12 2018-01-30 北京旷视科技有限公司 Face image synthesis method and apparatus
CN110580482B (en) * 2017-11-30 2022-04-08 腾讯科技(深圳)有限公司 Image classification model training, image classification and personalized recommendation method and device
CN110580482A (en) * 2017-11-30 2019-12-17 腾讯科技(深圳)有限公司 Image classification model training, image classification and personalized recommendation method and device
CN107845127A (en) * 2017-12-02 2018-03-27 天津浩宝丰科技有限公司 A kind of human face cartoon animation image design method
CN107886568A (en) * 2017-12-09 2018-04-06 东方梦幻文化产业投资有限公司 A kind of method and system that human face expression is rebuild using 3D Avatar
CN107886568B (en) * 2017-12-09 2020-03-03 东方梦幻文化产业投资有限公司 Method and system for reconstructing facial expression by using 3D Avatar
CN108200334A (en) * 2017-12-28 2018-06-22 广东欧珀移动通信有限公司 Image capturing method, device, storage medium and electronic equipment
CN108446601A (en) * 2018-02-27 2018-08-24 东南大学 A kind of face identification method based on sound Fusion Features
CN108446601B (en) * 2018-02-27 2021-07-13 东南大学 Face recognition method based on dynamic and static feature fusion
CN110276720A (en) * 2018-03-16 2019-09-24 华为技术有限公司 A kind of image generating method and device
WO2019174522A1 (en) * 2018-03-16 2019-09-19 华为技术有限公司 Image generating method and device
CN110276720B (en) * 2018-03-16 2021-02-12 华为技术有限公司 Image generation method and device
US11308636B2 (en) 2018-04-28 2022-04-19 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method, apparatus, and computer-readable storage medium for obtaining a target image
CN108765291A (en) * 2018-05-29 2018-11-06 天津大学 Super resolution ratio reconstruction method based on dense neural network and two-parameter loss function
CN108846817A (en) * 2018-06-22 2018-11-20 Oppo(重庆)智能科技有限公司 Image processing method, device and mobile terminal
CN108846817B (en) * 2018-06-22 2021-01-12 Oppo(重庆)智能科技有限公司 Image processing method and device and mobile terminal
CN110210336B (en) * 2019-05-16 2021-03-16 赣南师范大学 Low-resolution single-sample face recognition method
CN110210336A (en) * 2019-05-16 2019-09-06 赣南师范大学 A kind of low resolution single sample face recognition method
CN110503606B (en) * 2019-08-29 2023-06-20 广州大学 Method for improving face definition
CN110503606A (en) * 2019-08-29 2019-11-26 广州大学 A method of improving face clarity
CN112581362A (en) * 2019-09-27 2021-03-30 美敦力公司 Image processing method and device for adjusting image details
CN110910310B (en) * 2019-10-25 2021-04-30 南京大学 Face image reconstruction method based on identity information
CN110910310A (en) * 2019-10-25 2020-03-24 南京大学 Face image reconstruction method based on identity information
CN111260558A (en) * 2020-01-22 2020-06-09 武汉大学 Image super-resolution network model with variable magnification
WO2022006693A1 (en) * 2020-07-06 2022-01-13 Polycom Communications Technology (Beijing) Co. Ltd. Videoconferencing systems with facial image rectification
CN111931692A (en) * 2020-08-31 2020-11-13 青岛聚看云科技有限公司 Display device and image recognition method
CN112150615A (en) * 2020-09-24 2020-12-29 四川川大智胜软件股份有限公司 Face image generation method and device based on three-dimensional face model and storage medium
CN117952833A (en) * 2023-10-30 2024-04-30 中国科学院长春光学精密机械与物理研究所 Hyper-spectral image super-resolution reconstruction system and method based on three-branch network

Similar Documents

Publication Publication Date Title
CN101216889A (en) A face image super-resolution method with the amalgamation of global characteristics and local details information
CN110443842B (en) Depth map prediction method based on visual angle fusion
Zhang et al. Residual networks for light field image super-resolution
CN110458939B (en) Indoor scene modeling method based on visual angle generation
CN107204010B (en) A kind of monocular image depth estimation method and system
CN105741252B (en) Video image grade reconstruction method based on rarefaction representation and dictionary learning
CN111696137B (en) Target tracking method based on multilayer feature mixing and attention mechanism
Nie et al. Skeleton-bridged point completion: From global inference to local adjustment
CN108549844A (en) A kind of more people's Attitude estimation methods based on multi-layer fractal network and joint relatives' pattern
Sheng et al. Cross-view recurrence-based self-supervised super-resolution of light field
Wu et al. Perceptual metric-guided human image generation
CN103824272A (en) Face super-resolution reconstruction method based on K-neighboring re-recognition
CN101877143A (en) Three-dimensional scene reconstruction method of two-dimensional image group
CN115423734B (en) Infrared and visible light image fusion method based on multi-scale attention mechanism
CN115482382A (en) Image semantic segmentation method based on Transformer architecture
CN114004847B (en) Medical image segmentation method based on graph reversible neural network
CN111241963B (en) First person view video interactive behavior identification method based on interactive modeling
CN112232134A (en) Human body posture estimation method based on hourglass network and attention mechanism
Shi et al. Exploiting multi-scale parallel self-attention and local variation via dual-branch transformer-CNN structure for face super-resolution
CN109447175A (en) In conjunction with the pedestrian of deep learning and metric learning recognition methods again
CN113128424A (en) Attention mechanism-based graph convolution neural network action identification method
CN115841697A (en) Motion recognition method based on skeleton and image data fusion
CN111488951B (en) Method for generating countermeasure metric learning model for RGB-D image classification
Li et al. Maskformer with improved encoder-decoder module for semantic segmentation of fine-resolution remote sensing images
Liu et al. Multi-scale residual pyramid attention network for monocular depth estimation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20080709