Background technology
Image is one of the mankind's main carrier of obtaining outside objective world information.Along with human society gets into the information age, computer technology, modern communications technology and the information processing technology have obtained development fast, and consumption electronic product has spread all over each corner of human lives.Along with the appearance of digital picture and application widely; People are also increasingly high to the requirement of picture quality; Improving the image capture device hardware capabilities under the situation near the limit, adopting software engineering to improve original image resolution ratio becomes one of hot issue of digital image processing field gradually.
Image interpolation is traditional one of method that improves image resolution ratio that is used for.The interpolation method of traditional digital picture has a variety of; Wherein fastest is Nearest Neighbor (nearest-neighbor) method; This method through copy be inserted into the pixel value of a little adjacent nearest pixel value as the insertion point; This method speed is fast, realizes simply, but in interpolation image, the zigzag flaw occurs through regular meeting; Bilinear (bilinearity) interpolation method is through getting each mean value that is inserted into four immediate pixel numerical value a little as the insertion point pixel value; This method has been eliminated the zigzag phenomenon; But computing time is long slightly, and blurring effect appears in boundary in image easily.Bicubic (bicubic) convolution method with the interpolation point (i+u, j+v) 16 neighbor pixel points are considered to come on every side, through following interpolation formula can obtain this interpolation point pixel value f (i+u, j+v):
f(i+u,j+v)=A*B*C
A=[s(u+1)s(u+0)s(u-1)s(u-2)]
C=[s(v+1)s(v+0)s(v-1)s(v-2)]
T
Compare arest neighbors and bilinear interpolation, the bicubic convolution method has bigger improvement to the image interpolation effect, and interpolation place numerical value and derivative are all continuous, and the details performance is clearer, has overcome the image flaw of above two kinds of methods to a certain extent.But owing to relate to a cube computing in the calculating formula, and the matrix convolution is arranged, so calculated amount is compared, and the above two are bigger.
Comparatively level and smooth zone in image, it is fine that above classic method such as Bilinear and Bicubic can handle.But for image-region at boundary, more than traditional interpolation method then have tangible flaw, as fuzzy, sawtooth etc.And the human visual system is very responsive for boundary characteristic, therefore makes this flaw cause bigger harmful effect to image visual effect.Therefore, realize a main target of better image interpolation is when interpolation, how to keep the characteristic of image at boundary better, avoid producing the various flaws of traditional interpolation method.In order to realize such target, need carry out comparatively reasonably modeling to picture signal.The difficult point of picture signal modeling is the unstable state characteristic of himself.Simultaneously, natural image often has geometrical rule property, and promptly image changes on boundary direction slowly, on perpendicular to the direction on border, changes rapidly.This character has reflected the segmentation statistic steady state characteristic of picture signal.According to a such characteristic, people such as Li have proposed a kind of covariance information that utilizes and have come implicit expression to comprise the adaptive interpolation method of boundary information.The high resolving power covariance information estimates through the low resolution covariance, the low resolution covariance information then is that statistic keeps stable in the hypothesis window, in a local window, samples and estimates.This method can be handled the border condition than large scale in the image well; But can't handle some border and textural characteristics well than small scale; Simultaneously because its method has only been utilized the neighbor pixel dot information on the diagonal; The interpolation sharpness on border is compared original image and is also had a certain distance, and has some false narrow and small edge effect image visual effects in the interpolation image, seems to lack the nature and the sense of reality.Zhang and Wu introduce into more pixel space correlationship; Not only utilized the correlationship of diagonal; And considered the correlationship on level and the vertical direction; Keep stable through as if statistics amount in a local window, this correlationship restriction is forced on the pixels all in window, estimate finally to obtain the value of one group of pixel to be inserted through piece optimization.This method has made full use of the correlationship between high low-resolution image, has adaptability preferably for the structural information of natural image, therefore bigger improvement people's such as Li algorithm.
More than two kinds of algorithms all be that hypothesis image statistics in selected window keeps stable; When image boundary characteristic dimension during less than window size; This hypothesis also is false; Therefore the estimation for statistic information in the window is inaccurate, causes the error of system model, thereby causes interpolation image flaw to occur at boundary.
Summary of the invention
Occur the problem of flaw in order to solve the prior art interpolation image at boundary, the embodiment of the invention provides a kind of image interpolation method that improves image resolution ratio, comprising:
Similarity probability between the partial structurtes of preset reference image vegetarian refreshments in the partial structurtes of each pixel and the selected window in the selected window of computed image;
Pixel value according to known low-resolution pixel point in the selected window is estimated first parameter and second parameter; First parameter is to be used for portraying in the window model parameter of neighbor pixel point regression relation on each pixel and its diagonal, and second parameter is the model parameter that is used for portraying neighbor pixel point regression relation on the perpendicular and horizontal line direction of interior each pixel of window;
Utilize the pixel value of low-resolution pixel point, the estimated result of first parameter and the estimated result of second parameter, set up the pixel value of interpolation full-resolution picture vegetarian refreshments in the selected window and the correlationship of matching error;
Set up the optimization aim function according to similarity probability and matching error, the optimization aim function is carried out the pixel value that optimum estimate obtains the full-resolution picture vegetarian refreshments;
The pixel value of the full-resolution picture vegetarian refreshments that obtains by optimum estimate inserts selected window with the similarity probability greater than the full-resolution picture vegetarian refreshments of setting threshold.
The embodiment of the invention also provides a kind of image interpolation device that improves image resolution ratio simultaneously, comprising:
The probability generation module is used for computed image and selectes the similarity probability between the partial structurtes of the partial structurtes of each pixel in the window and the reference image vegetarian refreshments that selected window is preset;
First estimation module; Be used for first parameter and second parameter being estimated according to the pixel value of known low-resolution pixel point in the selected window; First parameter is to be used for portraying in the window model parameter of neighbor pixel point regression relation on each pixel and its diagonal, and second parameter is the model parameter that is used for portraying neighbor pixel point regression relation on the perpendicular and horizontal line direction of interior each pixel of window;
Correlationship is set up module, is used to utilize the pixel value of low-resolution pixel point, the estimated result of first parameter and the estimated result of second parameter, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in the selected window and the correlationship of matching error;
Second estimation module is used for setting up the optimization aim function according to similarity probability and matching error, and the optimization aim function is carried out the pixel value that optimum estimate obtains the full-resolution picture vegetarian refreshments;
Insert module, the pixel value of the full-resolution picture vegetarian refreshments that is used for obtaining by optimum estimate inserts selected window with the similarity probability greater than the full-resolution picture vegetarian refreshments of setting threshold.
Can find out by specific embodiments provided by the invention, the weight when having introduced the similarity probability as the high-resolution pixel point estimation, and then make evaluated error minimize, reduced the flaw that boundary occurs.
Embodiment
In order to solve flaw appears in interpolation image in the prior art at boundary problem; The embodiment of the invention provides a kind of image interpolation method that improves image resolution ratio; In the scheme of present embodiment, the ground that is without loss of generality, regarding in the high-definition picture pixel in the low-resolution image as through rule the factor is that 2 down-sampling obtains; Therefore these pixels can be regarded the part in the original high resolution image as, and Fig. 1 has provided the interpolation synoptic diagram twice.The task of interpolation is selected 10 interpolation from existing low-resolution pixel exactly and is gone out first's full-resolution picture vegetarian refreshments 11 and second portion full-resolution picture vegetarian refreshments 12 so.Whole Interpolation Process was divided into for two steps.The first step is that interpolation obtains first's full-resolution picture vegetarian refreshments 11.Second step was to select 10 according to first's full-resolution picture vegetarian refreshments 11 that first step interpolation obtains with existing low-resolution pixel to come interpolation to obtain second portion full-resolution picture vegetarian refreshments 12.The algorithm that two steps handled is the same, only at rotation that 45 degree are arranged on the direction and the scaling on the yardstick.Therefore, the core objective of interpolation algorithm can be regarded as from existing low-resolution pixel and selects the process that 10 interpolation go out first's full-resolution picture vegetarian refreshments 11.
The image interpolation method of raising image resolution ratio provided by the invention sees also Fig. 2, comprises the steps:
Step 101, the similarity probability P between the partial structurtes of center pixel to be inserted in the partial structurtes of each pixel and the selected window in the selected window of computed image
LAnd P
H
For the ease of explanation; Present embodiment with center pixel to be inserted as reference image vegetarian refreshments preset in the selected window; But this is not limited, the reference image vegetarian refreshments also can be any pixel that comprises first pixel and last pixel in the selected window of image.Present embodiment is to be in the local window at center with center pixel to be inserted, and the pixel value of high-resolution pixel in this window is estimated.The local window with different similarity probability that is provided with is like this shown by Fig. 3, comprises first 21, second portion 22 and third part 23.In a such local window; Not all pixel all has the similar topography's characteristic of concentricity pixel to be inserted 20; Be partial structurtes and dissimilar; There are some possibly also have bigger difference; Wherein the pixel of first 21 is similar with local window center pixel partial structurtes to be inserted, and the pixel of second portion 22 and third part 23 and local window center pixel 20 partial structurtes to be inserted are dissimilar, therefore need the similarity difference that a kind of index is reacted partial structurtes between each pixel.This similarity difference can be measured by the similarity probability between them.
In the window in the partial structurtes of j pixel and the selected window computing formula as the similarity probability between the partial structurtes of d pixel of reference image vegetarian refreshments be
P (d wherein; J) for being the similarity probability between the partial structurtes of the center pixel to be inserted in the present embodiment as d pixel of reference image vegetarian refreshments in the partial structurtes of j pixel in the selected window and the selected window;
is the vector of the pixel value formation of order of the neighbours on four diagonals of center pixel to be inserted in the selected window; D is a positive integer; D≤m+n; N is the number of selected window middle high-resolution pixel; M is the number of low-resolution pixel point; ε is a positive number of avoiding removing zero overflow error;
is the vector of the pixel value formation of ordering of the neighbours on four diagonals of j pixel in the selected window; J is a positive integer; J≤m+n; H is used for the distribution of shapes parameter of control characteristic function, and Fig. 4 is two right synoptic diagram of pixel with different similarity probability.
According to above formula
Can the similarity probability calculation between the concentricity pixel to be inserted of all pixels in the window be come out, form two probability vector P
LAnd P
H, wherein
Specific as follows; Utilize the low resolution neighbours' point on four diagonals of low-resolution pixel point; Similarity probability in partial structurtes that each known low-resolution pixel is selected in the selected window of calculating and the selected window between the partial structurtes of preset reference image vegetarian refreshments, and form low-resolution pixel point similarity probability vector P
L, wherein
Represent in the 1st partial structurtes that low-resolution pixel is selected and the selected window similarity probability between the partial structurtes of preset reference image vegetarian refreshments,
Represent the similarity probability i≤m between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes that i low-resolution pixel selected and the selected window,
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes that m low-resolution pixel selected and the selected window.Utilize the low resolution neighbours' point on four diagonals of full-resolution picture vegetarian refreshments; Similarity probability between the partial structurtes of preset reference image vegetarian refreshments in the partial structurtes of each full-resolution picture vegetarian refreshments and the selected window in the selected window of calculating, and form full-resolution picture vegetarian refreshments probability vector P
H, wherein
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of the 1st full-resolution picture vegetarian refreshments,
Represent the similarity probability k≤n between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of k full-resolution picture vegetarian refreshments,
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of n full-resolution picture vegetarian refreshments.When the similarity probability that calculates between full-resolution picture vegetarian refreshments and the center pixel to be inserted; Can directly utilize low resolution neighbours on 4 diagonals of each full-resolution picture vegetarian refreshments to select pixel value and constitute low resolution neighbours on 4 diagonals that partial structurtes vector
utilizes each full-resolution picture vegetarian refreshments and put pixel value and constitute the partial structurtes vector shown in Fig. 5 a, directly utilize formula
to calculate then.When the similarity probability that calculates between low-resolution pixel point and the center pixel to be inserted; Because it is the full-resolution picture vegetarian refreshments of the unknown that low-resolution pixel select that neighbours on the diagonal select; Can be according to the structural similarity between high low resolution hypothesis, utilize low resolution neighbours on its diagonal to put low resolution neighbours on 4 diagonals that pixel value constitutes
utilizes each low-resolution pixel point and put pixel value and constitute the partial structurtes vector shown in Fig. 5 b.
Step 102; Pixel value according to known low-resolution pixel point in the selected window is estimated parameter a and parameter b; Parameter a is used for portraying in the window model parameter of neighbor pixel point regression relation on each pixel and its diagonal, and parameter b is the model parameter that is used for portraying neighbor pixel point regression relation on the perpendicular and horizontal line direction of interior each pixel of window.
In the present embodiment, utilize known low-resolution pixel to select and estimate to carry out modeling to the relation between them for unknown full-resolution picture vegetarian refreshments.Adopt two group model parameter a, b to come to portray respectively this space correlation relation.Represent the regression relation on the diagonal with
; With the regression relation on
expression level and the vertical direction; With t ∈ { 1; 2; 3,4} representes t neighbours on the respective direction.For example
With
Difference remarked pixel x
iAnd y
iT neighbours on diagonal;
With
Difference remarked pixel x
iAnd y
iT neighbours' point in the horizontal and vertical directions.Through supposing that the parameter between high low-resolution image is consistent, can estimate parameter a and b according to the pixel value of low-resolution image pixel in the local window.Between high low resolution pixel level with have consistent model parameter on the vertical direction shown in Fig. 6 a, parameter b can directly be carried out weighted least-squares and estimated to try to achieve its optimal value.Has consistent model parameter between high low resolution pixel on the diagonal shown in Fig. 6 b; According to the conforming hypothesis of model parameter between the high low resolution; Yardstick put be twice, adopt
and come parameter a is estimated.
To parameter a, when parameter b is estimated, directly pass through
Parameter a is carried out weighted least-squares estimate, wherein
Represent the first parameter optimal value estimated result, a
tRepresent a vectorial composition value with the parameter a of vector representation, t ∈ 1,2,3,4}, f (x
i) i low-resolution pixel point x of expression
iPixel value,
Remarked pixel x
iT low resolution neighbours point on diagonal
Pixel value.Directly pass through
Second parameter is carried out weighted least-squares method estimate, wherein
Represent the second parameter optimal value estimated result, b
tRepresent a vectorial composition value with second parameter of vector representation,
Expression low-resolution pixel x
iT neighbours' point on vertical and horizontal line direction
Pixel value.
Step 103 is utilized the pixel value of low-resolution pixel point, the estimated result of parameter a and the estimated result of parameter b, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in the selected window and the correlationship of matching error.
Come the correlationship on the local diagonal of picture engraving through parameter
, can get
portrays the level of topography and the correlationship on the vertical direction through parameter, can get
and
Describe in the integrating
step 102, adopt two group model parameter a, b to come to portray respectively this space correlation relation.Represent the regression relation on the diagonal with
; With the regression relation on
expression level and the vertical direction; With t ∈ { 1; 2; 3; 4} representes t neighbours on the respective direction, and the configuration schematic diagram of space correlation relation is as shown in Figure 7 between interpolation neighbours' point.
Through the correlationship that use a model parameter a and b come the picture engraving local space, the matching error of above-mentioned expression can be expressed as following form:
Expression is passed through
Portrayal full-resolution picture vegetarian refreshments y
K 'With neighbor pixel point on its diagonal
Matching error during regression relation, Q representes to have the neighbor pixel point on the selected window inner opposite angle line direction
Full-resolution picture vegetarian refreshments y
K 'Number,
Expression is passed through
Portrayal low-resolution pixel point x
I 'With neighbor pixel point on its diagonal
Matching error during regression relation, P representes to have the neighbor pixel point on the selected window inner opposite angle line direction
Low-resolution pixel point x
I 'Number,
Expression is passed through
Portrayal full-resolution picture vegetarian refreshments y
K "With neighbor pixel point on its diagonal
Matching error during regression relation, K representes vertically and on the horizontal line direction have the neighbor pixel point in the selected window
Full-resolution picture vegetarian refreshments y
K "Number,
Expression is passed through
Portrayal low-resolution pixel point x
I "With neighbor pixel point on its diagonal
Matching error during regression relation, S representes vertically and on the horizontal line direction have the neighbor pixel point in the selected window
Low-resolution pixel point x
I "Number.
Step 104 is according to the similarity probability P
LAnd P
HSet up the optimization aim function F with matching error, the optimization aim function F is carried out the best estimate that optimum estimate obtains the full-resolution picture vegetarian refreshments in the group window.
Set up optimization aim function
satisfy simultaneously following constraint condition
wherein F be the optimization aim function; The pixel value of f (y) expression full-resolution picture vegetarian refreshments, w representes selected window.Through finding the solution this Estimation Optimization problem, finally can obtain the best estimate of the full-resolution picture vegetarian refreshments in the group window.
The best estimate of full-resolution picture vegetarian refreshments promptly carries out the pixel value that optimum estimate obtains the full-resolution picture vegetarian refreshments to the optimization aim function.
Step 105, the pixel value of the full-resolution picture vegetarian refreshments that obtains by optimum estimate, the best estimate of output center pixel to be inserted.
Wherein, center pixel to be inserted is the main interpolation point of paying close attention in the present embodiment, therefore only exports the numerical value of this point.In order to reduce the time complexity of total algorithm; Can consider that the value of the point that those and center pixel similarity to be inserted probability is higher also exports as its final pixel value, be about to the similarity probability and insert selected window greater than the full-resolution picture vegetarian refreshments of setting threshold.
Through with local window in the slip of carrying out on the image on horizontal direction and the vertical direction, we can finally accomplish the process of image interpolation in the hope of the optimal estimation value of all pixels to be inserted.
Second embodiment provided by the invention is a kind of image interpolation device that improves image resolution ratio, and its structure is as shown in Figure 8, comprising:
Probability generation module 201 is used for computed image and selectes the similarity probability between the partial structurtes of the partial structurtes of each pixel in the window and the reference image vegetarian refreshments that selected window is preset;
First estimation module 202; Be used for first parameter and second parameter being estimated according to the pixel value of known low-resolution pixel point in the selected window; First parameter is to be used for portraying in the window model parameter of neighbor pixel point regression relation on each pixel and its diagonal, and second parameter is the model parameter that is used for portraying neighbor pixel point regression relation on the perpendicular and horizontal line direction of interior each pixel of window;
Correlationship is set up module 203, is used to utilize the pixel value of low-resolution pixel point, the estimated result of first parameter and the estimated result of second parameter, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in the selected window and the correlationship of matching error;
Second estimation module 204 is used for setting up the optimization aim function according to similarity probability and matching error, and the optimization aim function is carried out the pixel value that optimum estimate obtains the full-resolution picture vegetarian refreshments;
Insert module 205, the pixel value of the full-resolution picture vegetarian refreshments that is used for obtaining by optimum estimate inserts selected window with the similarity probability greater than the full-resolution picture vegetarian refreshments of setting threshold.
Further;
Probability generation module 201; Also be used for carrying out the similarity probability calculation through
; P (d wherein; J) in the partial structurtes of j pixel in the selected window and the selected window as the similarity probability between the partial structurtes of d pixel of reference image vegetarian refreshments;
is the vector of the pixel value formation of ordering of the neighbours on four diagonals of d pixel in the selected window; D is a positive integer; D≤m+n; N is the number of selected window middle high-resolution pixel; M is the number of low-resolution pixel point; ε is a positive number of avoiding removing zero overflow error,
be the vector of the pixel value formation of ordering of the neighbours on four diagonals of j pixel in the selected window, j is a positive integer; J≤m+n, h are used for the distribution of shapes parameter of control characteristic function.
Further;
Probability generation module 201; Also be used to utilize the low resolution neighbours' point on four diagonals of low-resolution pixel point; Similarity probability in partial structurtes that each known low-resolution pixel is selected in the selected window of calculating and the selected window between the partial structurtes of preset reference image vegetarian refreshments, and form low-resolution pixel point similarity probability vector P
L, wherein
Represent in the 1st partial structurtes that low-resolution pixel is selected and the selected window similarity probability between the partial structurtes of preset reference image vegetarian refreshments,
Represent the similarity probability i≤m between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes that i low-resolution pixel selected and the selected window,
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes that m low-resolution pixel selected and the selected window; And
Utilize the low resolution neighbours' point on four diagonals of full-resolution picture vegetarian refreshments; Similarity probability between the partial structurtes of preset reference image vegetarian refreshments in the partial structurtes of each full-resolution picture vegetarian refreshments and the selected window in the selected window of calculating, and form full-resolution picture vegetarian refreshments probability vector P
H, wherein
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of the 1st full-resolution picture vegetarian refreshments,
Represent the similarity probability k≤n between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of k full-resolution picture vegetarian refreshments,
Represent the similarity probability between the partial structurtes of reference image vegetarian refreshments preset in partial structurtes and the selected window of n full-resolution picture vegetarian refreshments.
Further, first estimation module 202 also is used for directly passing through
First parameter is carried out weighted least-squares estimate, wherein
Represent the first parameter optimal value estimated result, a
tRepresent a vectorial composition value with first parameter of vector representation, t ∈ 1,2,3,4}, f (x
i) i low-resolution pixel point x of expression
iPixel value,
Remarked pixel x
iT low resolution neighbours point on diagonal
Pixel value;
Directly pass through
Second parameter is carried out weighted least-squares method estimate, wherein
Represent the second parameter optimal value estimated result, b
tRepresent a vectorial composition value with second parameter of vector representation,
Expression low-resolution pixel x
iT neighbours' point on vertical and horizontal line direction
Pixel value.
Further, second estimation module 204 also is used to set up the optimization aim function
Satisfy following constraint condition simultaneously
Wherein F is the optimization aim function, the pixel value of f (y) expression full-resolution picture vegetarian refreshments, and w representes selected window,
Expression is passed through
Portrayal full-resolution picture vegetarian refreshments y
K 'With neighbor pixel point on its diagonal
Matching error during regression relation, Q representes to have the neighbor pixel point on the selected window inner opposite angle line direction
Full-resolution picture vegetarian refreshments y
K 'Number,
Expression is passed through
Portrayal low-resolution pixel point x
I 'With neighbor pixel point on its diagonal
Matching error during regression relation, P representes to have the neighbor pixel point on the selected window inner opposite angle line direction
Low-resolution pixel point x
I 'Number,
Expression is passed through
Portrayal full-resolution picture vegetarian refreshments y
K "With neighbor pixel point on its diagonal
Matching error during regression relation, K representes vertically and on the horizontal line direction have the neighbor pixel point in the selected window
Full-resolution picture vegetarian refreshments y
K "Number,
Expression is passed through
Portrayal low-resolution pixel point x
I "With neighbor pixel point on its diagonal
Matching error during regression relation, S representes vertically and on the horizontal line direction have the neighbor pixel point in the selected window
Low-resolution pixel point x
I "Number.
Further, insert module 205 also is used for if the reference image vegetarian refreshments is the full-resolution picture vegetarian refreshments, will insert selected window as the full-resolution picture vegetarian refreshments of reference image vegetarian refreshments.
Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, belong within the scope of claim of the present invention and equivalent technologies thereof if of the present invention these are revised with modification, then the present invention also is intended to comprise these changes and modification interior.