CN102769745B - Image self-adaptation down-sampling method depending on interpretation - Google Patents

Abstract

An image self-adaptation down-sampling method depending on interpretation relates to image and video processing, and solves a problem that the current down-sampling and up-sampling are always independent, but image and video processing always needs both the down-sampling and the up-sampling synchronously. The method comprises the steps as follows: steps I, on a given input image, a down-sampling image is generated, and the down-sampling image interpolates an up-sampling image; steps II, adopting a down-sampling method based on a block, the quality of sampled images is improved; steps III, whether the images is related to content or not is judged, if so, a step IV is executed, and otherwise, a step V is executed; steps IV, for down-sampling is not related to the content, interpretation coefficient forms a interpretation matrix, a down-sampling image is obtained via the inversion of an interpretation process; and steps V, if not related to the content, initial coefficient is provided, an initial down-sampling image is obtained by using a method not related to the content, new interpretation coefficient is trained according to obtained images, and the down-sampling image is obtained via iteration. The method is applicable to the image and video processing.

Description

The image adaptive Downsapling method that interpolation relies on

Technical field

The present invention relates to image and Video processing.

Background technology

Along with the develop rapidly of electronic technology and computer hardware technology, the display device with different resolution of all kinds emerges in an endless stream, and how same piece image or same sequence is effectively changed the technology that will be a very challenging property between the display device of different resolution.Efficient image down sampling and the quality of up-sampling on converted images have important impact, except changing between different display devices, image down sampling and up-sampling are also widely applied in low bit-rate image compression and spatial domain scalable video.

Image down sampling can be divided into spatial domain down-sampling and frequency domain down-sampling usually, in the down-sampling of spatial domain, down-sampling, by having come with the identical original pixels of reservation evenly and at intervals (direct down-sampling), also can first carry out low-pass filtering to original image and then carry out down-sampling.Frequency domain down-sampling obtains down-sampled images by the low frequency coefficient retaining wavelet coefficient or discrete cosine transform coefficient.

These image down sampling methods are widely used all at present, but they all do not consider that the image after on down-sampling carries out the impact of up-sampling again.Often independently, and the process of image and video needs down-sampling and up-sampling usually simultaneously for current down-sampling and up-sampling.

Summary of the invention

The object of the invention is to solve current down-sampling and up-sampling often independently, and the process of image and video needs the problem of down-sampling and up-sampling usually simultaneously, provides the image adaptive Downsapling method that a kind of interpolation relies on.

The image adaptive Downsapling method that interpolation relies on, it is as follows that it comprises concrete steps:

Step one, to given interpolation method and input picture, generate a width down-sampled images, high-quality up-sampling image is gone out to this down-sampled images interpolation;

Step 2, the up-sampling image that step one is obtained, adopt block-based image adaptive Downsapling method, utilize the information relevant to current block and its cumulative sum with the product of corresponding interpolation coefficient improved the quality of down-sampled images as a constant vector;

Step 3, to determine whether and interpolation that content is irrelevant; Then perform step 4; No, then perform step 5;

The image adaptive Downsapling method that step 4, the interpolation had nothing to do with content rely on, is combined into interpolating matrix by interpolation coefficient, by asking the inverse operation of its correspondence to obtain the down-sampled images of the optimum of correspondence to Interpolation Process;

The image adaptive Downsapling method that step 5, the interpolation relevant to content rely on, first one group of initial interpolation coefficient is provided, the image down sampling method using the interpolation had nothing to do with content to rely on obtains the initial down-sampled images of a width, train the interpolation coefficient made new advances according to the down-sampled images obtained, and then iteration obtains down-sampled images.

The present invention is used for the image down sampling of resolution decreasing, utilizes interpolation coefficient and the original-resolution image content of picture up-sampling, realizes the image down sampling that interpolation relies on; For generated down-sampled images, we utilize interpolation to obtain high-quality up-sampling image from this down-sampled images.Be applicable to image or Video processing.

Accompanying drawing explanation

Fig. 1 is the schematic diagram of the image adaptive down-sampling that interpolation relies on, the image adaptive down-sampling schematic diagram of the interpolation dependence of Fig. 2 of the present invention piece of level, in figure, white round dot represents the pixel after down-sampling, black round dot represents the pixel of diagonally interpolation generation, rectangle represents the pixel generated along horizontal direction interpolation, and triangle represents the pixel of vertically interpolation generation.

Embodiment

Embodiment one: composition graphs 1 illustrates present embodiment, the image adaptive Downsapling method that interpolation described in present embodiment relies on, it is as follows that it comprises concrete steps:

Step one, to given interpolation method and input picture, generate a width down-sampled images, high-quality up-sampling image is gone out to this down-sampled images interpolation;

Step 2, the up-sampling image that step one is obtained, adopt block-based image adaptive Downsapling method, utilize the information relevant to current block and its cumulative sum with the product of corresponding interpolation coefficient improved the quality of down-sampled images as a constant vector;

Step 3, to determine whether and interpolation that content is irrelevant; Then perform step 4; No, then perform step 5;

The image adaptive Downsapling method that step 4, the interpolation had nothing to do with content rely on, is combined into interpolating matrix by interpolation coefficient, by asking the inverse operation of its correspondence to obtain the down-sampled images of the optimum of correspondence to Interpolation Process;

The image adaptive Downsapling method that step 5, the interpolation relevant to content rely on, first one group of initial interpolation coefficient is provided, the image down sampling method using the interpolation had nothing to do with content to rely on obtains the initial down-sampled images of a width, train the interpolation coefficient made new advances according to the down-sampled images obtained, and then iteration obtains down-sampled images.

The present invention is used for the image down sampling of resolution decreasing, utilizes interpolation coefficient and the original-resolution image content of picture up-sampling, realizes the image down sampling that interpolation relies on; For generated down-sampled images, we utilize interpolation to obtain high-quality up-sampling image from this down-sampled images.Be applicable to image or Video processing.

Embodiment two: the detailed process of the step one in the image adaptive Downsapling method that interpolation described in embodiment one relies on is:

Size is the input picture of M × N to make Y represent, X represents that size is the image after the down-sampling of M/2 × N/2, represent the up-sampling image generated after interpolation; The object of adaptive down-sampling tries to achieve a down-sampled images, and from this down-sampled images, we can obtain the interpolation image of better quality; Therefore, X demand fulfillment:

$X=\arg \underset{x}{\min }{\left|\right|\hat{Y}-Y\left|\right|}^2---\left(1\right)$

Wherein X=[X _0,0, X _0,1..., X _{0, N/2-1}, X _1,0, X _1,1..., X _{1, N/2-1}..., X _m/2-1,0, X _m/2-1,1..., X _{m/2-1, N/21}] ^tand Y=[Y _0,0, Y _0,1..., Y _{0, N-1}, Y _1,0, Y _1,1..., Y _{1, N-1}..., Y _m/2-1,0, Y _m-1,1..., Y _{m-1, N-1}] ^t, obviously, formula (1) shows that this Downsapling method depends on interpolation algorithm; Also be, to arbitrary down-sampling pixel X (i, j) (in the middle of 9 black circles as shown in Figure 1 that), first orient all neighborhood territory pixels (gray pixels as shown in Figure 1) that will use X (i, j) in Interpolation Process; The optimization aim of this image adaptive Downsapling method minimizes the error sum of squares between the original pixels in pixel and input picture that locating area interpolate value goes out; It is pointed out that Fig. 1 is for bilinear interpolation (pixel that each interpolation goes out only depends on four nearest with it pixels), in fact this Downsapling method is applicable to other various interpolation algorithms;

The interpolation of Y is represented as:

$\hat{Y}=\mathrm{HX}---\left(2\right)$

Wherein, H represents interpolating matrix; The H be made up of interpolation coefficient is represented as:

$H=\left[\begin{array}{cccc}{h}_{\mathrm{0,0}}& h_{\mathrm{0,1}}& ..& h_{0,M/2\×N/2-1}\\ h_{\mathrm{1,0}}& h_{\mathrm{1,1}}& ..& h_{1,M/2\×N/2-1}\\ h_{\mathrm{2,0}}& h_{\mathrm{2,1}}& ..& h_{2,M/2\×N/2-1}\\ ..& ..& ..& ..\\ h_{M\×N-\mathrm{1,0}}& h_{M\×N-\mathrm{1,1}}& ..& h_{M\×N-1,M/2\×N/2-1}\end{array}\right]---\left(3\right)$

Wherein, h _k,linterpolation coefficient when representing interpolation kth pixel corresponding to l down-sampling pixel; Bring formula (3) and formula (2) into formula (1), the target function obtaining optimum down-sampled images is:

$J=\arg \underset{x}{\min }{\left|\right|\mathrm{HX}-Y\left|\right|}^2---\left(4\right)$

The local derviation making J is zero, obtains

$\frac{\∂J}{\∂X}=H^T(\mathrm{HX}-Y)=0---\left(5\right)$

Then optimum down-sampled images is obtained by following formula

X ^*=(H ^TH) ^-1H ^TY （6）。

Embodiment three: the detailed process of the step 2 in the image adaptive Downsapling method that interpolation described in embodiment one relies on is:

The image adaptive Downsapling method that the interpolation carrying out description block level with the block of m × n size and bilinear interpolation relies on, in Fig. 2, the size of block is 8x8 and is allly belonged to same piece by the point that solid line surrounds.Most interpolating pixel all utilizes the pixel in same obviously.But for some boundary pixels, its interpolation also relates to the partial pixel outside current block, such as, gray pixels in Fig. 2, its Interpolation Process has related to the down-sampling pixel outside current window.If all down-sampling pixels in same are expressed as a column vector, namely solution of the present invention, then can introduce other known variables (outside current block, the down-sampling pixel simultaneously related to when interpolation current block boundary pixel).We utilize the information outside current block and it can be used as a constant vector.Definition Φ is a constant vector, be then expressed the interpolation of Y:

$\hat{Y}=\mathrm{HX}+\mathrm{\Φ}---\left(7\right)$

Wherein Φ is a length is the column vector of mn.Most elements in Φ are all zero, and we only give appropriate value to the element corresponding with the gray pixels in Fig. 2.To the assignment false code of interpolating matrix H and the column vector Φ corresponding with it as shown in Table 1 and Table 2, wherein B _lrepresent the down-sampling block corresponding with current block B, N (i) represents the neighborhood territory pixel that interpolation i-th pixel relates to.

The calculating of interpolating matrix H:

Bring formula (7) into formula (4), the target function calculating optimum down-sampling block is expressed as:

$J=\arg \underset{x}{\min }{\left|\right|\mathrm{HX}+\mathrm{\Φ}-Y\left|\right|}^2---\left(8\right)$

The local derviation of J in formula (8) is set to zero, obtains:

$\frac{\∂J}{\∂X}=H^T(\mathrm{HX}+\mathrm{\Φ}-Y)=0---\left(9\right)$

Therefore, optimum down-sampling block is obtained by following formula

X ^*=(H ^TH) ^-1[H ^TY-H ^TΦ]（10）

The calculating of constant vector Φ:

The computation complexity of image adaptive down-sampling mainly concentrates on interpolating matrix H that formula (10) relates to, the derivation of constant vector Φ, matrix multiple, matrix inversion operation.The complexity of H is O (m ²n ²), the computation complexity of matrix multiple and matrix inversion is respectively O (m ³n ³) and O (m ⁶n ⁶).It is pointed out that the most elements due to H and Φ are all zero, sparse matrix can be utilized to store H and Φ, thus cut down storage and computation complexity further.The prerequisite of formula (10) is that interpolating matrix H before down-sampling must be known.

Embodiment four: the detailed process of the step 5 in the image adaptive Downsapling method that interpolation described in embodiment one relies on is:

The interpolation method relevant to some contents, its interpolating matrix H is unknown before down-sampling, therefore can not directly utilize formula (10) to obtain optimal solution.In order to address this problem, the invention allows for the image adaptive Downsapling method of the interpolation dependence that a content is correlated with, at initial phase, first given H ⁰and X ⁰initial value; In each iterative process, according to the result X of last iteration ^i-1upgrade H ⁱ, then according to the H that formula (10) obtains ⁱcalculate X ⁱ; Next, X is calculated ⁱand X ^i-1between error; If this error is lower than a threshold value, just thinks that this algorithm is restrained, thus stop iterative process in advance; Otherwise this algorithm enters next iteration;

The image adaptive Downsapling method that content dependent interpolation relies on:

Claims (1)

1. the image adaptive Downsapling method of interpolation dependence, it is characterized in that, it is as follows that it comprises concrete steps:

Step one, to given interpolation method and input picture, generate a width down-sampled images, go out high-quality up-sampling image to this down-sampled images interpolation, concrete steps are:

Size is the input picture of M × N to make Y represent, X represents that size is the image after the down-sampling of M/2 × N/2, represent the up-sampling image generated after interpolation, X will meet:

$X=\arg \underset{x}{\min }{\left|\right|\hat{Y}-Y\left|\right|}^2---\left(1\right)$

Wherein, X=[X _0,0, X _0,1..., X _{0, N2-1}, X _1,0, X _1,1..., X _{1, N/2-1}..., X _m/2-1,0, X _m/2-1,1..., X _{m/2-1, N/2-1}] ^t, and Y=[Y _0,0, Y _0,1..., Y _{0, N-1}, Y _1,0, Y _1,1..., Y _{1, N-1}..., Y _m/2-1,0, Y _m-1,1..., Y _{m-1, N-1}] ^t, the interpolation of Y is:

$\hat{Y}=\mathrm{HX}---\left(2\right)$

Wherein, H is interpolating matrix; The H be made up of interpolation coefficient is represented as:

$H=\left[\begin{array}{cccc}{h}_{\mathrm{0,0}}& h_{\mathrm{0,1}}& ..& h_{0,M/2\×N/2-1}\\ h_{\mathrm{1,0}}& h_{\mathrm{1,1}}& ..& h_{1,M/2\×N/2-1}\\ h_{\mathrm{2,0}}& h_{\mathrm{2,1}}& ..& h_{2,M/2\×N/2-1}\\ ..& ..& ..& ..\\ h_{M\×N-\mathrm{1,0}}& h_{M\×N-\mathrm{1,1}}& ..& h_{M\×N-1,M/2\×N/2-1}\end{array}\right]---\left(3\right)$

Wherein, h _k,linterpolation coefficient when representing interpolation kth pixel corresponding to l down-sampling pixel; Bring formula (3) and formula (2) into formula (1), the target function obtaining optimum down-sampled images is:

$J=\arg \underset{x}{\min }{\left|\right|\mathrm{HX}-Y\left|\right|}^2---\left(4\right)$

The local derviation making J is zero, obtains:

$\frac{\∂J}{\∂X}=H^T(\mathrm{HX}-Y)=0---\left(5\right)$

Then optimum down-sampled images is obtained by following formula:

X ^*＝(H ^TH) ^-1H ^TY (6)；

Step 2, the up-sampling image that step one is obtained, adopt block-based image adaptive Downsapling method, utilize the information relevant to current block and its cumulative sum with the product of corresponding interpolation coefficient improved the quality of down-sampled images as a constant vector, concrete steps are:

The image adaptive Downsapling method that the interpolation carrying out description block level with the block of m × n size and bilinear interpolation relies on, definition Φ is a constant vector, be then expressed the interpolation of Y:

$\hat{Y}=\mathrm{HX}+\mathrm{\Φ}---\left(7\right)$

Wherein Φ is a length is the column vector of mn, and the most elements in Φ are all zero, only gives appropriate value to the element corresponding with boundary pixel, to the assignment of interpolating matrix H and the column vector Φ corresponding with it is:

The calculating of interpolating matrix H:

Input picture Y and current block B;

As the pixel ∈ B that i capable corresponding pixel ∈ B, j row are corresponding _ltime;

In interpolating matrix H, column locations is the element of (i, j) is H _ij;

By the interpolating matrix element H corresponding with current block B dimension _ijbe initialized as 0;

If the corresponding pixel of j row be included in i capable corresponding to pixel contiguous range N (i) within;

Then by the matrix element H of this column locations (i, j) _ijassignment is the interpolation coefficient W that this columns is corresponding _j;

Export interpolating matrix H;

The calculating of constant vector Φ:

Input picture Y and current block B;

As the capable corresponding pixel ∈ B of i;

By constant vector element Φ corresponding for position i in current block B _ibe initialized as 0;

Variable sum is initialized as 0;

For the corresponding pixel of j row be included in i capable corresponding to pixel contiguous range N (i) within;

If the corresponding pixel of j row is not included in current block B, by the product accumulation of jth row pixel and the corresponding interpolation coefficient of jth row pixel to variable sum;

Variable sum is added to constant vector element Φ _i;

Output constant vector Φ;

Wherein B _lrepresent the down-sampling block corresponding with current block B, N (i) represents the neighborhood territory pixel that interpolation i-th pixel relates to;

Bring formula (7) into formula (4), the target function calculating optimum down-sampling block is expressed as:

$J=\arg \underset{x}{\min }{\left|\right|\mathrm{HX}+\mathrm{\Φ}-Y\left|\right|}^2---\left(8\right)$

The local derviation of J in formula (8) is set to zero, obtains:

$\frac{\∂J}{\∂X}=H^T(\mathrm{HX}+\mathrm{\Φ}-Y)=0---\left(9\right)$

Optimum down-sampling block is obtained by following formula:

X ^*＝(H ^TH) ^-1[H ^TY-H ^TΦ] (10)；

Step 3, to determine whether and interpolation that content is irrelevant; Then perform step 4; No, then perform step 5;

The image adaptive Downsapling method that step 4, the interpolation had nothing to do with content rely on, is combined into interpolating matrix by interpolation coefficient, by asking the inverse operation of its correspondence to obtain the down-sampled images of the optimum of correspondence to Interpolation Process;

The image adaptive Downsapling method that step 5, the interpolation relevant to content rely on, first one group of initial interpolation coefficient is provided, the image down sampling method using the interpolation had nothing to do with content to rely on obtains the initial down-sampled images of a width, train the interpolation coefficient made new advances according to the down-sampled images obtained, and then iteration obtains down-sampled images.