CN102136065A - Face super-resolution method based on convex optimization - Google Patents
Face super-resolution method based on convex optimization Download PDFInfo
- Publication number
- CN102136065A CN102136065A CN 201110075606 CN201110075606A CN102136065A CN 102136065 A CN102136065 A CN 102136065A CN 201110075606 CN201110075606 CN 201110075606 CN 201110075606 A CN201110075606 A CN 201110075606A CN 102136065 A CN102136065 A CN 102136065A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- facial image
- low
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a face super-resolution method based on convex optimization for mainly solving the problem of low quality of a super-resolution face image acquired by using the conventional method. The method is implemented by the following steps of: (1) partitioning a high-resolution face image training set, a low-resolution face image training set and a testing low-resolution face image into image blocks; (2) solving the reconstruction coefficient of each testing low-resolution face image block between adjacent image blocks at the corresponding positions of the low-resolution face image training set by using a convex optimization method for each testing low-resolution face image block; and (3) reconstructing a super-resolution face image block by using the reconstruction coefficient of the low-resolution image block and synthesizing an entire super-solution face image. By adopting the method, the quality of the super-resolution face image is enhanced, the complexity of an algorithm is lowered and higher generality is achieved. The method is suitable for video meetings, public safety and face recognition.
Description
Technical field:
The invention belongs to technical field of image processing, relate to the human face super-resolution method, can be used for video conference, news broadcast, recognition of face and video monitoring.
Background technology:
People's face is generally believed it is the object that researching value is arranged most in image processing field, be widely used in real society, as video conference, news broadcast, recognition of face, video monitoring etc. all mainly based on people's face.Because people's face is a kind of very complicated, the pattern of multidimensional is subjected to the influence of aspects such as position, direction, illumination condition and facial expression easily, add people to people's face be familiar with and responsive, therefore for the research of people's face difficulty relatively.But obtain clear, high-resolution facial image is a requisite step of people's face disposal system, and the facial image that obtains in the video equipment is very little often, the image of this low resolution is difficult to directly use in engineering, thereby becomes one of biggest obstacle of recognition of face.
The human face super-resolution technology is a kind of technology of obtaining corresponding high-resolution human face image from the low resolution facial image.For facial image, because everyone image all is made up of organs such as eyebrow, eyes, noses, each face organ presents specific textural characteristics.And after remarkable face alignment, it is roughly the same can being similar to the residing position of thinking in every pictures of each organ.Because the feature of this structure, Baker and Kanade have proposed the notion of human face super-resolution (Hallucinating Faces) for the first time in 2000, make reconstructing human face super resolution from the super-resolution technique of image, separate, as a relatively independent research field, in the method for Baker and Kanade proposition, they select the level of gaussian pyramid of facial image and the derivative and the laplacian pyramid of vertical direction, as the feature space of facial image, but there is bigger noise in the super-resolution facial image that obtains at some position.In order to address this problem, mainly contain following several existing algorithm.
Document [1]: people such as W.T.Freeman had proposed the image super-resolution method based on example in 2002, (W.T.Freeman, T.R.Jones, and E.C.Pasztor.Example-based super-resolution.IEEE Computer Graphics and Applications, Vol.22, lssue2,2002) this method utilizes markov network to come in the learning training storehouse details with the corresponding high resolving power rate of low-resolution image zones of different image, the relation that obtains with study is predicted the detailed information of input low-resolution image again, but the learning method of this markov network has still been lost a lot of image informations.
Document [2]: people such as Hong Chang in 2004 have proposed neighborhood and have embedded algorithm (H.Chang, D.-Y.Yeung, and Y.Xiong.Super-resolutionthrongh neighbor embedding.CVPR, 2004.), in this algorithm, suppose that the high-resolution and low-resolution image has similar popular structure, the weights in low resolution rate space are applied to high resolution space, reconstruct high-definition picture.But this hypothesis also is false, and algorithm has instability, and the super-resolution quality of human face image of acquisition is not ideal enough.
Document [3]: 2010, people such as Xiang Ma have proposed the human face super-resolution method (X.Ma of position-based piece, J.Zhang, C.Qi, Hullucinating face by position-patch.Pattern Recognition, 2010), he utilizes protruding each image block of least-squares algorithm reconstruct, reconstruction accuracy is not high enough, and the super-resolution quality of human face image of acquisition is also not ideal enough.
Summary of the invention
The objective of the invention is to overcome the deficiency of above-mentioned prior art, proposed a kind of human face super-resolution method, to improve the quality of super-resolution facial image based on protruding optimization.
For achieving the above object, the present invention includes following steps:
(1) input high-resolution human face image training set { H
m, m=1,2 ..., M blurs and down-sampling high-resolution facial image, obtains the facial image training set { L of low resolution
m, m=1,2 ..., M, the facial image training set with high-resolution and low-resolution is divided into overlapped image block respectively then
With
M=1,2 ..., M, M represent the number of facial image in the facial image training set of high-resolution and low-resolution, and N represents the number of divided image piece, and (i j) is illustrated in the positional information of the image block that is positioned at the capable j row of i in the facial image;
(2) input test low resolution facial image I, according to the mode of the divided block identical with low resolution facial image training set, I is expressed as overlapped image block test low resolution facial image
(3) for each the image block I that tests the low resolution facial image in the step (2)
k(i, j), k=1,2 ..., N solves its reconstruction coefficients x for the image block of corresponding same position in the low resolution facial image training set by following protruding majorized function
k:
min||x
k||
1?subject?to?I
k(i，j)＝A
kx
k
I wherein
k(i, j) k image block of expression test low resolution facial image, (i, j) k the position of image block in facial image of expression, A
kRepresent a matrix, its each row are by the image block L of corresponding same position in the low resolution facial image training set
Mk(i j) forms, m=1, and 2 ..., M, k=1,2 ..., N;
(4) utilize the reconstruction coefficients x that obtains in the step (3)
k, with the image block H of corresponding same position in the high-resolution human face image training set
Mk(i, j), m=1,2 ..., super-resolution facial image piece S is synthesized in the M weighting
k:
K=1,2 ..., N;
(5) with all super-resolution facial image pieces that synthesizes, form whole facial image according to its position in facial image, obtain the facial image of super-resolution.
The present invention has been owing to used majorized function to find the solution the reconstruction coefficients of everyone face image block, thereby compares with existing method and to have the following advantages:
1) saves operation time, reduced the complexity of computing;
2) improve reconstruction accuracy, promoted the quality of super-resolution image.
Description of drawings
Fig. 1 is a process flow diagram of the present invention;
Fig. 2 is the synoptic diagram that the present invention and existing method compare the PSNR value;
Fig. 3 is the present invention and existing method visual effect figure relatively.
Embodiment
Below with reference to Fig. 1 the present invention is elaborated:
Step 1: import high-resolution and low-resolution facial image training set, and be expressed as the form of image block.
The facial image of input is the front face picture of CMU PIE face database, CMU PIE face database is made up of 1428 front faces, always have 68 different people, everyone has 21 pictures that obtain under different illumination conditions, wherein the size of each pictures is 100*100, and all passes through standardization;
In order to guarantee the validity of algorithm, the present invention selects 30 people's 630 pictures as high-resolution human face image training set { H at random
m, m=1,2 ..., M, remaining is divided into 10 groups according to the method at random as test pattern.Blur and down-sampling for the every pictures in the high-resolution human face image training set, the image set that obtains is as low resolution facial image training set { L
m, m=1,2 ..., M, M represent the number of facial image in the facial image training set of high-resolution and low-resolution, M=630 in the present embodiment, but be not limited to 630, the size of low resolution facial image is 25 * 25;
Facial image training set for the high-resolution and low-resolution of importing is divided into overlapped image block by the following method respectively
With
M=1,2 ..., M, N represent the number of divided image piece, (i j) is illustrated in the positional information of the image block that is positioned at the capable j of i row in the facial image:
The enlargement factor of supposing facial image is a, low resolution facial image divided image block size is n * n, the size of the image block of corresponding high-resolution human face image division is under a condition doubly of low resolution facial image piece, the size of low-resolution image piece overlapping region is set to n/3, high-definition picture piece overlapping region size is set to an/3, n=3 in the present embodiment, a=4.
Step 2: input test low resolution facial image, and be expressed as the form of image block.
Input test low resolution facial image I carries out the partitioned image piece according to the partitioned image block mode identical with image in the described low resolution facial image of step 1 training set, and is expressed as overlapped image block form
N represents the number of divided image piece, and (i j) is illustrated in the positional information of the image block that is positioned at the capable j of i row in the facial image.
Step 3: utilize protruding optimization method, find the solution the reconstruction coefficients of each test low resolution facial image piece.
Each image block I for test low resolution facial image in the step (2)
k(i, j), k=1,2 ..., N solves its reconstruction coefficients x for the image block of corresponding same position in the low resolution facial image training set by following protruding majorized function
k(image block is represented with the column vector form):
min||x
k||
1?subject?to
I wherein
k(i, j) k image block of expression test low resolution facial image, A
kRepresent a matrix, each row of this matrix are by the image block L of corresponding same position in the low resolution facial image training set
Mk(i j) forms, m=1, and 2 ..., M, k=1,2 ..., N, ε represent fault-tolerant parameter, ε＞0.Adopt basic tracing algorithm to find the solution this majorized function in the present embodiment, but be not limited to this algorithm, for example orthogonal matching pursuit algorithm, gradient project algorithms and Lasso algorithm etc.;
Step 4: reconstruct super-resolution facial image piece.
Utilize the reconstruction coefficients x that obtains in the step 3
k, with the image block H of corresponding same position in the high-resolution human face image training set
Mk(i, j) the synthetic super-resolution facial image piece of weighting:
K=1,2 ..., N; M=1,2 ..., M.
Step 5: synthetic whole super-resolution facial image.
With the resulting all super-resolution facial image piece S of step 4
k, k=1,2 ..., N is stitched together according to the position of each image block in facial image, and for the lap of image block by averaging as its pixel value, thereby obtain whole super-resolution facial image.
Effect of the present invention further specifies by following emulation:
1, simulated conditions and content:
Use CMU PIE face database to carry out the human face super-resolution experiment, this database is made up of 1428 front faces, always have 68 different people, everyone comprises 21 pictures, all in the different time, different illumination conditions is taken and is obtained, and people's face figure of shooting has different facial expressions, wherein the size of each pictures is 100*100, and all passes through standardization.In experiment, 30 people's of picked at random of the present invention 630 pictures are as the training picture, and remaining is divided into 10 groups according to the method at random as the test picture.
Software platform is MATLAB7.1
2, simulation result:
The present invention experimentizes on CMU PIE face database, carry out emulation relatively, in order to verify validity of the present invention, this experiment compares the PSNR value and the SSIM value of test sample book, and PSNR represents Y-PSNR, and it is worth, and high image quality is good more more, SSIM represents structural similarity, its value is high more approaching more with true picture, and all experimental results are averaged, and experimental result is as shown in table 1.
Table 1 contrasts in the experimental result of PIE face database with this method and existing method
| Method | The Bicubic interpolation | Document [1] | Document [2] | Document [3] | The inventive method |
| ?PSNR | ?24.5388 | 26.0954 | 26.3785 | 28.1613 | 28.2437 |
| ?SSIM | ?0.7278 | 0.7544 | 0.7444 | 0.8146 | 0.8178 |
Document in the table 1 [1] is meant W.T.Freeman, T.R.Jones, and and E.C.Pasztor.Example-based super-resolution.IEEE Computer Graphics and Applications, Vol.22, lssue 2,2002; Document [2] is meant H.Chang, D.-Y.Yeung, and Y.Xiong.Super-resolutionthrongh neighbor embedding.CVPR, 2004; Document [3] is meant X.Ma, J.Zhang, C.Qi, Hullucinating face by position-patch.Pattern Recognition, 2010.
As can be seen from Table 1, the inventive method is better than the result of document [1], [2] and [3] on the whole.
For this experiment, add up the experimental result of the test facial image under the CMU PIE database different illumination conditions, compare the PSNR value of the present invention and existing method, its result is as shown in Figure 2.The present invention is better than existing method for the super-resolution facial image effect that obtains under the different illumination conditions as can be seen from Figure 2.
For this experiment, choose the super-resolution facial image that a part of the present invention and existing method obtain, the visual effect of comparison the present invention and existing method, its result as shown in Figure 3, wherein, Fig. 3 (a) represents input test low-resolution image (25 * 25); Fig. 3 (b) represents bi-cubic interpolation result (100 * 100); Fig. 3 (c) represents document [1] methods and results (100 * 100); Fig. 3 (d) represents document [2] methods and results (100 * 100); Fig. 3 (e) represents document [3] methods and results (100 * 100); Fig. 3 (f) represents result of the present invention (100 * 100); Fig. 3 (g) represents true picture (100 * 100).As can be seen from Figure 3, the present invention is better than existing method on visual effect.
Claims (3)
1. the human face super-resolution method based on protruding optimization comprises the steps:
(1) input high-resolution human face image training set { H
m, m=1,2 ..., M blurs and down-sampling high-resolution facial image, obtains the facial image training set { L of low resolution
m, m=1,2 ..., M, the facial image training set with high-resolution and low-resolution is divided into overlapped image block respectively then
With
M=1,2 ..., M, M represent the number of facial image in the facial image training set of high-resolution and low-resolution, and N represents the number of divided image piece, and (i j) is illustrated in the positional information of the image block that is positioned at the capable j row of i in the facial image;
(2) input test low resolution facial image I, according to the mode of the divided block identical with low resolution facial image training set, I is expressed as overlapped image block test low resolution facial image
(3) for each the image block I that tests the low resolution facial image in the step (2)
k(i, j), k=1,2 ..., N solves its reconstruction coefficients x for the image block of corresponding same position in the low resolution facial image training set by following protruding majorized function
k:
min||x
k||
1?subject?to?I
k(i，j)＝A
kx
k
I wherein
k(i, j) k image block of expression test low resolution facial image, (i, the j) positional information of k image block of expression in facial image, A
kRepresent a matrix, its each row are by the image block L of corresponding same position in the low resolution facial image training set
Mk(i j) forms, m=1, and 2 ..., M, k=1,2 ..., N;
(4) utilize the reconstruction coefficients x that obtains in the step (3)
k, with the image block H of corresponding same position in the high-resolution human face image training set
Mk(i, j), m=1,2 ..., super-resolution facial image piece S is synthesized in the M weighting
k:
K=1,2 ..., N;
(5) with all super-resolution facial image pieces that synthesizes, form whole facial image according to its position in facial image, obtain the facial image of super-resolution.
2. human face super-resolution method according to claim 1, wherein the described facial image training set with high-resolution and low-resolution of step (1) is divided into overlapped image block respectively
With
Be the hypothesis facial image enlargement factor be a, low resolution facial image divided image block size is n * n, the size of the image block of corresponding high-resolution human face image division is under a condition doubly of low resolution facial image piece, the size of low-resolution image piece overlapping region is set to n/3, and high-definition picture piece overlapping region size is set to an/3.
3. human face super-resolution method according to claim 1, wherein step (5) is described with all test high-resolution human face image blocks that synthesizes, form whole facial image according to its position in facial image, be that test high-resolution human face image blocks that all are synthetic are stitched together according to its position in facial image, and for the lap of image block by averaging as its pixel value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110075606.8A CN102136065B (en) | 2011-03-28 | 2011-03-28 | Face super-resolution method based on convex optimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110075606.8A CN102136065B (en) | 2011-03-28 | 2011-03-28 | Face super-resolution method based on convex optimization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102136065A true CN102136065A (en) | 2011-07-27 |
| CN102136065B CN102136065B (en) | 2014-04-02 |
Family
ID=44295847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110075606.8A Expired - Fee Related CN102136065B (en) | 2011-03-28 | 2011-03-28 | Face super-resolution method based on convex optimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102136065B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065292A (en) * | 2012-12-26 | 2013-04-24 | 武汉大学 | Face super resolution rebuilding method based on principal component sparse expression |
| CN103093444A (en) * | 2013-01-17 | 2013-05-08 | 西安电子科技大学 | Image super-resolution reconstruction method based on self-similarity and structural information constraint |
| CN103312941A (en) * | 2013-06-19 | 2013-09-18 | 清华大学 | Joint de-noising and super-resolution method and joint de-noising and super-resolution system on basis of convex optimization theories for videos |
| CN103325104A (en) * | 2013-06-09 | 2013-09-25 | 中山大学 | Facial image super-resolution reestablishing method based on iterative sparse representation |
| CN105335930A (en) * | 2015-10-28 | 2016-02-17 | 武汉大学 | Edge data driven robustness-based face super-resolution processing method and system |
| CN105701515A (en) * | 2016-01-18 | 2016-06-22 | 武汉大学 | Face super-resolution processing method and system based on double-layer manifold constraint |
| CN105701770A (en) * | 2016-01-18 | 2016-06-22 | 武汉大学 | Face super-resolution processing method and system based on context linear model |
| CN107680043A (en) * | 2017-09-29 | 2018-02-09 | 杭州电子科技大学 | Single image super-resolution output intent based on graph model |
| CN110738601A (en) * | 2019-10-23 | 2020-01-31 | 智慧视通(杭州)科技发展有限公司 | low-resolution face image super-resolution reconstruction method based on three-dimensional face model |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477684A (en) * | 2008-12-11 | 2009-07-08 | 西安交通大学 | Process for reconstructing human face image super-resolution by position image block |
-
2011
- 2011-03-28 CN CN201110075606.8A patent/CN102136065B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477684A (en) * | 2008-12-11 | 2009-07-08 | 西安交通大学 | Process for reconstructing human face image super-resolution by position image block |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065292B (en) * | 2012-12-26 | 2015-07-15 | 武汉大学 | Face super resolution rebuilding method based on principal component sparse expression |
| CN103065292A (en) * | 2012-12-26 | 2013-04-24 | 武汉大学 | Face super resolution rebuilding method based on principal component sparse expression |
| CN103093444A (en) * | 2013-01-17 | 2013-05-08 | 西安电子科技大学 | Image super-resolution reconstruction method based on self-similarity and structural information constraint |
| CN103093444B (en) * | 2013-01-17 | 2015-05-20 | 西安电子科技大学 | Image super-resolution reconstruction method based on self-similarity and structural information constraint |
| CN103325104A (en) * | 2013-06-09 | 2013-09-25 | 中山大学 | Facial image super-resolution reestablishing method based on iterative sparse representation |
| CN103325104B (en) * | 2013-06-09 | 2016-05-11 | 中山大学 | Based on the face image super-resolution reconstruction method of iteration sparse expression |
| CN103312941B (en) * | 2013-06-19 | 2016-12-07 | 清华大学 | Joint Video denoising based on convex optimum theory and super-resolution method and system |
| CN103312941A (en) * | 2013-06-19 | 2013-09-18 | 清华大学 | Joint de-noising and super-resolution method and joint de-noising and super-resolution system on basis of convex optimization theories for videos |
| CN105335930B (en) * | 2015-10-28 | 2018-05-29 | 武汉大学 | The robustness human face super-resolution processing method and system of edge data driving |
| CN105335930A (en) * | 2015-10-28 | 2016-02-17 | 武汉大学 | Edge data driven robustness-based face super-resolution processing method and system |
| CN105701770A (en) * | 2016-01-18 | 2016-06-22 | 武汉大学 | Face super-resolution processing method and system based on context linear model |
| CN105701515A (en) * | 2016-01-18 | 2016-06-22 | 武汉大学 | Face super-resolution processing method and system based on double-layer manifold constraint |
| CN105701770B (en) * | 2016-01-18 | 2018-12-14 | 武汉大学 | A kind of human face super-resolution processing method and system based on context linear model |
| CN105701515B (en) * | 2016-01-18 | 2019-01-04 | 武汉大学 | A kind of human face super-resolution processing method and system based on the constraint of the double-deck manifold |
| CN107680043A (en) * | 2017-09-29 | 2018-02-09 | 杭州电子科技大学 | Single image super-resolution output intent based on graph model |
| CN107680043B (en) * | 2017-09-29 | 2020-09-22 | 杭州电子科技大学 | Single image super-resolution output method based on graph model |
| CN110738601A (en) * | 2019-10-23 | 2020-01-31 | 智慧视通(杭州)科技发展有限公司 | low-resolution face image super-resolution reconstruction method based on three-dimensional face model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102136065B (en) | 2014-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102136065B (en) | Face super-resolution method based on convex optimization | |
| Liang et al. | Details or artifacts: A locally discriminative learning approach to realistic image super-resolution | |
| Zhang et al. | Scene-adaptive remote sensing image super-resolution using a multiscale attention network | |
| Kim et al. | Progressive face super-resolution via attention to facial landmark | |
| Cai et al. | FCSR-GAN: Joint face completion and super-resolution via multi-task learning | |
| Zhang et al. | Image super-resolution based on structure-modulated sparse representation | |
| Ma et al. | Hallucinating face by position-patch | |
| Du et al. | Fully convolutional measurement network for compressive sensing image reconstruction | |
| Li et al. | Single image superresolution via directional group sparsity and directional features | |
| Yang et al. | Reference-guided deep super-resolution via manifold localized external compensation | |
| CN102902961A (en) | Face super-resolution processing method based on K neighbor sparse coding average value constraint | |
| Sun et al. | Super resolution reconstruction of images based on interpolation and full convolutional neural network and application in medical fields | |
| Zhang et al. | Color-guided depth image recovery with adaptive data fidelity and transferred graph Laplacian regularization | |
| Li et al. | Frequency separation network for image super-resolution | |
| Yang et al. | Multi-sensor image super-resolution with fuzzy cluster by using multi-scale and multi-view sparse coding for infrared image | |
| Zhu et al. | Image Super‐Resolution Based on Sparse Representation via Direction and Edge Dictionaries | |
| Jia et al. | Dual-complementary convolution network for remote-sensing image denoising | |
| Liu et al. | Arbitrary-scale super-resolution via deep learning: A comprehensive survey | |
| Yu et al. | Scale-aware frequency attention network for super-resolution | |
| Wang et al. | Super resolution for compressed screen content video | |
| Polasek et al. | Vision UFormer: Long-range monocular absolute depth estimation | |
| Rrmoku et al. | Data augmentation techniques for expanding the dataset in the task of image processing | |
| Hua et al. | Image super resolution using fractal coding and residual network | |
| Amiri et al. | A fast video super resolution for facial image | |
| Deka et al. | Joint sparse representation-based single image super-resolution for remote sensing applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140402 Termination date: 20200328 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |