CN101888557A - Cartoon image and video up-sampling method based on gradient domain transformation - Google Patents

Cartoon image and video up-sampling method based on gradient domain transformation Download PDF

Info

Publication number
CN101888557A
CN101888557A CN 201010199109 CN201010199109A CN101888557A CN 101888557 A CN101888557 A CN 101888557A CN 201010199109 CN201010199109 CN 201010199109 CN 201010199109 A CN201010199109 A CN 201010199109A CN 101888557 A CN101888557 A CN 101888557A
Authority
CN
China
Prior art keywords
image
gradient
video
domain transformation
carry out
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN 201010199109
Other languages
Chinese (zh)
Inventor
沈建冰
颜星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN 201010199109 priority Critical patent/CN101888557A/en
Publication of CN101888557A publication Critical patent/CN101888557A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention belongs to the field of image and video processing, and in particular relates to an image and video up-sampling method based on gradient domain transformation, which is particularly suitable for processing cartoon images and videos. The method comprises the following steps of: firstly carrying out up-sampling and amplification on a low-resolution image, and then obtaining the gradient of a high-resolution image after gradient transformation to be used as a constraint reconstructed high-resolution image according to a defined gradient domain transformation method. The method provided by the invention can rapidly and effectively carry out up-sampling on the cartoon images or videos, the computation is simple, the image after reconstruction has natural color, fuzzy edges caused by interpolation become clear, details are restored to a certain degree, and the visual effect is better.

Description

Cartoon image and video up-sampling method based on gradient domain transformation
Technical field
The invention belongs to image and field of video processing, be specifically related to a kind of image and video up-sampling method, be particularly suited for handling cartoon image and video based on gradient domain transformation.
Background technology
The super-resolution technique of image and video up-sampling technology (hereinafter to be referred as the up-sampling technology) similar image and video is image and the video amplification of expectation with low resolution, and keeps edge clear, does not have phenomenons such as sawtooth, has reached the purpose of improving visual effect.The up-sampling technology is widely used in fields such as video communication, object identification, HDTV (High-Definition Television), image compression and transmission.
At present employed top sampling method mainly is an interpolation algorithm in the business software, and commonly used have methods such as arest neighbors interpolation, bilinear interpolation, bicubic interpolation.The characteristics of these methods are can handle in real time and realization easily, but can not keep fully through the radio-frequency component of the image after the up-sampling, and it is fuzzyyer to show as the edge, and certain sawtooth (mosaic) phenomenon is arranged, and influences visual effect.
Super-resolution technique in the document can be divided three classes usually:
1, based on the method for image sequence, the multiple image that typically uses same scene comes the super-resolution image of re-construct.Its shortcoming is to be not easy to use, and amplification multiple is limited in can't obtaining more high-resolution image about 2 usually.
2, based on the super-resolution method of sample, its basic thought is the data set that makes up two associations, one is low resolution, another one is high-resolution, according to this two data training practice high low-resolution images between corresponding relation, and estimate the pairing high-definition picture of low-resolution image of input thus.Its shortcoming is to calculate comparatively complexity, and reconstructed results is relevant with training set, and the details that is recovered also is false, just from visually having improved the effect of image.
3, the method for edge sharpening, its basic thought are to recover the radio-frequency component of image, and just the fuzzy edge of expectation becomes clear, thereby improves visual effect.
The method that the present invention proposes belongs to the 3rd class methods recited above, at cartoon image, and the image that promptly lines are obvious, texture is single, local color is similar, details is less.
Summary of the invention
The invention discloses and a kind ofly carry out conversion, the gradient after the conversion is rebuild the image and the video up-sampling method of high-definition picture/frame of video as constraint by gradient to low-resolution image/frame of video.Core of the present invention has been to propose a kind of new algorithm of gradient domain transformation fast, and uses energy-optimised method to rebuild high-resolution image, makes that fuzzy edge becomes clear in image and the video, thereby improves the visual effect of image and video.
Cartoon image and video up-sampling method based on gradient domain transformation may further comprise the steps:
Step 1: for each the two field picture I in the video of input l, carry out image according to the multiplication factor of appointment and amplify;
If image I lBe gray level image, obtain initial pictures after then amplifying
Figure BSA00000165316600021
If image I lBe coloured image, the image after then will amplifying is transformed into the YCbCr color space by rgb color space, and to get the Y passage be initial pictures
Figure BSA00000165316600022
And preserve the data of Cb passage and Cr passage;
Step 2: each two field picture correspondence that step 1 is obtained Carry out gradient domain transformation:
(1) asks
Figure BSA00000165316600024
Gradient
Figure BSA00000165316600025
And the mould of this gradient
Figure BSA00000165316600026
And it is right
Figure BSA00000165316600027
Carry out normalization;
(2) ask
Figure BSA00000165316600028
Laplce's gradient
Figure BSA00000165316600029
And it is right
Figure BSA000001653166000210
Carry out normalization;
(3) ask for G according to following formula T:
G ^ T ( x , y ) = G l u - | ▿ 2 I l u | - - - ( 1 )
Figure BSA000001653166000212
Wherein
Figure BSA000001653166000213
Be Laplacian, (x y) is the pixel coordinate;
(4) to G TCarry out normalization, then according to the gradient behind the following formula acquisition gradient domain transformation
Figure BSA000001653166000214
▿ I T = γ · ▿ I l u · G T G l u ,
Wherein γ is the regulatory factor that presets, and its value can be regulated, and is made as 2～6 usually, is used for regulating under the situation that iterations is fixed the readability of image border;
Step 3: to each frame, according to the gradient behind the gradient domain transformation in the step 2
Figure BSA000001653166000216
Rebuild high-definition picture;
Step 4: if the I of present frame lBe gray level image, then execution in step (5);
If the I of present frame lBe coloured image, then step (3) rebuild the high-definition picture that obtains and reconfigure, be converted to the RGB image then as Cb and the Cr channel data preserved in Y passage and the step 1;
Step 5: the image that step 4 obtains is each frame of processed video, and it is formed video data.
When utilizing this method to handle single image, regard it as only have a frame video, the operation of carrying out step 1-5 gets final product.
Contrast prior art, beneficial effect of the present invention are, calculate simply, realize easily, can expand to the video up-sampling field naturally, are particularly useful for cartoon image, the image that promptly lines are obvious, texture is single, local color is similar, details is less.Method provided by the invention can be carried out up-sampling with cartoon image or video fast and effectively, and the nature of the image color after rebuilding, and interpolation causes the ground fuzzy edge to become clear, and details has obtained recovery to a certain degree, and visual effect is better.
Description of drawings
Fig. 1 is a flow chart of the present invention;
Fig. 2 is the schematic diagram of gradient domain transformation proposed by the invention; (a) low-resolution image of process bicubic interpolation; (b) gradient of figure (210); (c) Laplce's gradient; (d) gradient after the conversion; (e) reconstructed results;
Fig. 3 is to use gradient descent algorithm to find the solution the flow chart of energy function;
Fig. 4 (a) is the frame of video of the low resolution of input;
Fig. 4 (b) is to use the arest neighbors interpolation algorithm to amplify result after 4 times respectively along image x and y direction;
Fig. 4 (c) is to use the bicubic interpolation algorithm to amplify 4 times of resulting results afterwards;
The result that Fig. 4 (d) is to use the method among the present invention to obtain.
Embodiment
Below in conjunction with drawings and Examples technical solution of the present invention is further explained.
As shown in Figure 1, cartoon image and the concrete technology implementation scheme of video up-sampling method based on gradient domain transformation of the present invention is as follows:
Step 1: for each the two field picture I in the video of the low resolution of importing l, carry out image according to the multiplication factor (being the up-sampling factor in the accompanying drawing 1) of appointment and amplify, use traditional bicubic interpolation method to amplify in the present embodiment, if image I lBe gray level image, promptly obtain initial pictures after then amplifying
Figure BSA00000165316600031
(Fig. 2 (a)) is if image I lBe coloured image, the image after then will amplifying is transformed into the YCbCr color space by rgb color space, and getting the Y passage is initial pictures
Figure BSA00000165316600032
And preserve the data of Cb passage and Cr passage.
Step 2: Fig. 2 has schematically provided the gradient domain transformation method that defines among the present invention.
(1) at first calculates each two field picture correspondence Gradient
Figure BSA00000165316600034
And the mould of gradient
Figure BSA00000165316600035
See Fig. 2 (b), computational methods are
(2) calculate then
Figure BSA00000165316600037
Laplce's gradient
Figure BSA00000165316600038
Its absolute value is shown in Fig. 2 (c);
(3) ask for G according to following formula T:
G ^ T ( x , y ) = G l u - | ▿ 2 I l u |
Wherein
Figure BSA00000165316600043
Be Laplacian, (x y) is the pixel coordinate; It should be noted that in that to carry out step 3 (3) preceding, earlier right
Figure BSA00000165316600044
Figure BSA00000165316600045
Carry out normalization;
(4) to G TCarry out normalization, then according to the gradient behind the following formula acquisition gradient domain transformation
Figure BSA00000165316600046
▿ I T = γ · ▿ I l u · G T G l u ,
Wherein γ is the regulatory factor that presets, and its value can be regulated, and is made as 2～6 usually, can regulate the readability of image border under the situation that iterations is fixed.
Step 3: to each frame, according to the gradient after the conversion in the step 2 Rebuild high-definition picture.Use energy-optimised method to rebuild high-definition picture (also can adopt the Poisson method) in the present embodiment.Normal for guaranteeing the image color after the reconstruction, the image gradient after the reconstruction is subjected to The constraint of following energy function:
E = | | ( I h * K ) ↓ - I l | | 2 + β | | ▿ I h - ▿ I T | | 2 - - - ( 3 )
Wherein, first has guaranteed that the high-definition picture after the reconstruction is approaching as far as possible through resultant image and input picture after the down-sampling of identical multiple, () ↓ and be the down-sampling operator; Obtain gradient in the gradient of second high-definition picture after having guaranteed to rebuild and the step 2 Approaching as far as possible; β is a proportionality coefficient, is used for adjusting the influence degree of gradient constraint in whole energy function, as preferably, is set to 0.6.
This energy function can use the gradient descent method to find the solution, and flow chart promptly carries out iteration according to following iterative formula as shown in Figure 3:
I h t + 1 = I h t - τ · ∂ E ∂ I h , t＝0，1，2，3，...
Wherein,
Figure BSA000001653166000413
Will
Figure BSA000001653166000414
Be set to initial value
Figure BSA000001653166000415
Promptly stop when reaching maximum iteration time, wherein τ is the iteration step length that presets, as preferred τ=0.5.
Step 4: if the I of present frame lBe gray level image, then execution in step (5);
If the I of present frame lBe coloured image, then step (3) rebuild the high-definition picture that obtains and reconfigure, be converted to the RGB image then as Cb and the Cr channel data preserved in Y passage and the step 1;
Step 5: the image that step 4 obtains is each frame of processed video, and it is formed video data.
When utilizing this method to handle single image, regard it as only have a frame video, the operation of carrying out step 1-5 gets final product.
Usually carry out 1～5 iteration and just can obtain satisfied effect.So the method that the present invention proposes can be finished the task of up-sampling fast.
The method that proposes among the present invention can be amplified 1～5 times respectively along x and y direction with image or video, and the assurance edge is clear, for higher multiplication factor, carrying out that can iteration repeatedly handled, the output of last time as next time input, so just can be obtained the up-sampling result of higher multiple.
The result that is provided by Fig. 4 as can be seen, the method that is proposed among the present invention can be carried out up-sampling with cartoon image and video fast, and vision is improved greatly than the bicubic interpolation mode, has not only recovered the high-frequency information of losing, make fuzzy edge become clear, and the color nature.
Above-described specific descriptions; purpose, technical scheme and beneficial effect to invention further describe; institute is understood that; the above only is specific embodiments of the invention; and be not intended to limit the scope of the invention; within the spirit and principles in the present invention all, any modification of being made, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (5)

1. based on the cartoon image and the video up-sampling method of gradient domain transformation, it is characterized in that, may further comprise the steps:
Step 1: for each the two field picture I in the video of input l, carry out image according to the multiplication factor of appointment and amplify;
If image I lBe gray level image, obtain initial pictures after then amplifying
Figure FSA00000165316500011
If image I lBe coloured image, the image after then will amplifying is transformed into the YCbCr color space by rgb color space, and to get the Y passage be initial pictures
Figure FSA00000165316500012
And preserve the data of Cb passage and Cr passage;
Step 2: each two field picture correspondence that step 1 is obtained
Figure FSA00000165316500013
Carry out gradient domain transformation, that is:
(1) asks
Figure FSA00000165316500014
Gradient
Figure FSA00000165316500015
And the mould of this gradient And it is right
Figure FSA00000165316500017
Carry out normalization;
(2) ask
Figure FSA00000165316500018
Laplce's gradient And it is right Carry out normalization;
(3) ask for G according to following formula T:
G ^ T ( x , y ) = G l u - | ▿ 2 I l u |
Figure FSA000001653165000112
Wherein Be Laplacian, (x y) is the pixel coordinate;
(4) to G TCarry out normalization, then according to the gradient behind the following formula acquisition gradient domain transformation
Figure FSA000001653165000114
▿ I T = γ · ▿ I l u · G T G l u ,
Wherein γ is the regulatory factor that presets, and is used for regulating under the situation that iterations is fixed the readability of image border;
Step 3: to each frame, according to the gradient behind the gradient domain transformation in the step 2
Figure FSA000001653165000116
Rebuild high-definition picture;
Step 4: if the I of present frame lBe gray level image, then execution in step (5);
If the I of present frame lBe coloured image, then step (3) rebuild the high-definition picture that obtains and reconfigure, be converted to the RGB image then as Cb and the Cr channel data preserved in Y passage and the step 1;
Step 5: the image that step 4 obtains is each frame of processed video, and it is formed video data.When utilizing this method to handle single image, regard it as only have a frame video, the operation of carrying out step 1-5 gets final product.
2. according to claim 1 described cartoon image and video up-sampling method based on gradient domain transformation, it is characterized in that, use energy-optimised method to rebuild high-definition picture in the step 3, normal for guaranteeing the image color after the reconstruction, the image gradient after the reconstruction is subjected to
Figure FSA00000165316500021
The constraint of following energy function:
E = | | ( I h * K ) ↓ - I l | | 2 + β | | ▿ I h - ▿ I T | | 2
Wherein () ↓ be the down-sampling operator, β is a proportionality coefficient, is used for adjusting the influence degree of gradient constraint in whole energy function, as preferably, is set to 0.6;
3. according to claim 2 described cartoon image and video up-sampling method based on gradient domain transformation, it is characterized in that described energy function uses the gradient descent method to find the solution, method is as follows:
Carry out iteration according to following iterative formula:
I h t + 1 = I h t - τ · ∂ E ∂ I h , t＝0，1，2，3，...
Wherein,
Figure FSA00000165316500024
Will
Figure FSA00000165316500025
Be set to initial value
Promptly stop when reaching maximum iteration time, wherein τ is the iteration step length that presets, as preferred τ=0.5.
4. according to claim 1 described cartoon image and video up-sampling method, it is characterized in that, use the bicubic interpolation method that image is amplified in the step 1 based on gradient domain transformation.
5. according to claim 1 described cartoon image and video up-sampling method, it is characterized in that, when utilizing this method to handle single image, regard it as only have a frame video, carry out the operation of step 1-5 based on gradient domain transformation.
CN 201010199109 2010-06-12 2010-06-12 Cartoon image and video up-sampling method based on gradient domain transformation Pending CN101888557A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010199109 CN101888557A (en) 2010-06-12 2010-06-12 Cartoon image and video up-sampling method based on gradient domain transformation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010199109 CN101888557A (en) 2010-06-12 2010-06-12 Cartoon image and video up-sampling method based on gradient domain transformation

Publications (1)

Publication Number Publication Date
CN101888557A true CN101888557A (en) 2010-11-17

Family

ID=43074238

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010199109 Pending CN101888557A (en) 2010-06-12 2010-06-12 Cartoon image and video up-sampling method based on gradient domain transformation

Country Status (1)

Country Link
CN (1) CN101888557A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10540749B2 (en) 2018-03-29 2020-01-21 Mitsubishi Electric Research Laboratories, Inc. System and method for learning-based image super-resolution
CN115829842A (en) * 2023-01-05 2023-03-21 武汉图科智能科技有限公司 Device for realizing picture super-resolution reconstruction based on FPGA

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
《2010 International Conference on Image Analysis and Signal Processing》 20100411 Xing Yan, Jianbing Shen Fast Gradient-Aware Upsampling for Cartoon Video 636-639 1-5 , 2 *
《ACM Transactions on Graphics - Proceedings of ACM SIGGRAPH 2007》 20070731 Raanan Fattal Image Upsampling via Imposed Edge Statistics 1-5 第26卷, 第3期 2 *
《IEEE Conference on Computer Vision and Pattern Recognition,2008》 20080628 Jian Sun et al. Image Super-Resolution using Gradient Profile Prior 1-8 1-5 , 2 *
《Proceedings of ACM SIGGRAPH Asia 2008》 20081231 Qi Shan et al. Fast Image/Video Upsampling 1-5 第27卷, 第5期 2 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10540749B2 (en) 2018-03-29 2020-01-21 Mitsubishi Electric Research Laboratories, Inc. System and method for learning-based image super-resolution
CN115829842A (en) * 2023-01-05 2023-03-21 武汉图科智能科技有限公司 Device for realizing picture super-resolution reconstruction based on FPGA
CN115829842B (en) * 2023-01-05 2023-04-25 武汉图科智能科技有限公司 Device for realizing super-resolution reconstruction of picture based on FPGA

Similar Documents

Publication Publication Date Title
Suryanarayana et al. Accurate magnetic resonance image super-resolution using deep networks and Gaussian filtering in the stationary wavelet domain
CN106683067B (en) Deep learning super-resolution reconstruction method based on residual sub-images
CN107895345B (en) Method and device for improving resolution of face image
CN110634147B (en) Image matting method based on bilateral guide up-sampling
CN102842115B (en) Based on the compressed sensing image super-resolution rebuilding method of double dictionary study
US8335394B2 (en) Image processing method for boundary resolution enhancement
CN105046672A (en) Method for image super-resolution reconstruction
Yang et al. Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction
CN104299193B (en) Image super-resolution reconstruction method based on high-frequency information and medium-frequency information
US10007970B2 (en) Image up-sampling with relative edge growth rate priors
CN104851077A (en) Adaptive remote sensing image panchromatic sharpening method
CN104766272B (en) A kind of image super-resolution rebuilding method based on sub-pix skew model
CN107833182A (en) The infrared image super resolution ratio reconstruction method of feature based extraction
CN106910215A (en) A kind of super-resolution method based on fractional order gradient interpolation
DE102020133244A1 (en) Edge-conscious upscaling for improved screen content quality
Xie et al. Multiscale self-similarity and sparse representation based single image super-resolution
CN105279742B (en) A kind of image de-noising method quickly based on piecemeal estimation of noise energy
CN102844786A (en) Image enlargement device, image enlargement program, memory medium on which image enlargement program is stored, and display device
CN106846250A (en) A kind of super resolution ratio reconstruction method based on multi-scale filtering
CN106548452A (en) A kind of image-enhancing equipment and method
CN101888557A (en) Cartoon image and video up-sampling method based on gradient domain transformation
CN103903239A (en) Video super-resolution reconstruction method and system based on sparse principal component analysis and continued fraction interpolation technology
CN110895790A (en) Scene image super-resolution method based on posterior degradation information estimation
CN105528772A (en) Image fusion method based on guidance filtering
CN115760638A (en) End-to-end deblurring super-resolution method based on deep learning

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20101117