Embodiment
As previously mentioned, prior art in the time going interlacing to process, in the time adopting interpolation method based on border, can be because border error in judgement obtains wrong boundary information.Therefore, the interpolation result obtaining is also not accurate enough, affects picture quality.Present invention is directed at this, proposed a kind of new interlace-removing method, by the reliability of boundary direction is associated with interpolation result, go interlacing effect preferably to obtain.
Fig. 1 shows a kind of execution mode of interlace-removing method of the present invention, and it comprises:
Execution step s1, for interpolation pixel, obtains the existing pixel information in its neighborhood;
Execution step s2, according to the existing pixel information in described neighborhood, obtains the boundary direction detection information through all directions of described interpolation pixel;
Execution step s3, carries out reliability detection to described boundary direction detection information, obtains corresponding reliability, and described reliability reflects that current boundary direction detects the possibility of direction corresponding to information as the boundary direction of interpolation pixel;
Execution step s4, chooses the boundary direction that reliability is the highest and detects the boundary direction of direction corresponding to information as interpolation pixel;
Execution step s5, described interpolation pixel is carried out to vertical direction interpolation and the boundary direction interpolation based on described interpolation pixel, described vertical direction interpolation result and described boundary direction interpolation result are weighted and are added the pixel value that obtains described interpolation pixel; The weight of described boundary direction interpolation result is relevant to the reliability of the boundary direction of described interpolation pixel, and the weight sum of described vertical direction interpolation result and described boundary direction interpolation result is 1; Described vertical direction is the vertical direction through described interpolation pixel.
In above-mentioned execution mode, by the Boundary Detection of the direction to through interpolation pixel, obtain its corresponding reliability information, described reliability reflects that current boundary direction detects the possibility of direction corresponding to information as the boundary direction of interpolation pixel.Obtaining after described reliability information, therefrom choosing the direction of the process interpolation pixel that reliability is the highest as the boundary direction of interpolation pixel.And, the interpolation of the interpolation of interpolation pixel vertical direction and the boundary direction based on described interpolation pixel is merged, the reliability with described boundary direction when fusion is determined the interpolation proportion in fusion based on boundary direction.Thereby the result after interpolation can reflect the border condition of interpolation pixel more realistically, interpolation result is more accurate, has obtained and has gone interlacing effect preferably.
By concrete example, each step of above-mentioned execution mode is elaborated below.
Shown in Fig. 2, go interlacing interlaced video signal before treatment to comprise multirow pixel (representing with round dot) information, for example i-1 is capable, i is capable and i+1 is capable etc.Suppose, interpolation pixel (representing with asterisk point) while going interlacing to process at i-1 capable and i capable between, the pixel that i-1 is capable and i is capable can be thought in interpolation pixel neighborhood of a point, and the pixel that certain i+1 is capable also can be thought in interpolation pixel neighborhood of a point.Continue supposition, described interpolation pixel is positioned at j row that it is expert at, and the existing pixel that is positioned at (i-1, j) and (i, j) two positions is described interpolation pixel adjacent two pixels in vertical direction.And i-1 is capable and i other pixels on capable should be through the pixel adjacent with described interpolation pixel on multiple incline directions of described interpolation pixel.Definition is tilted to the left take described vertical direction as benchmark as a kind of dominant direction, and is tilted to the right take described vertical direction as benchmark as another kind of dominant direction., for a certain dominant direction, wherein comprised the incline direction of the described interpolation pixel of multiple processes.
For example, line through asterisk point in Fig. 2 has represented wherein a kind of incline direction being tilted to the right, existing pixel (i-1 on the i-1 of described incline direction process is capable, j+4) and the existing pixel (i, j-4) of i on capable be described interpolation pixel adjacent two pixels on described incline direction.Capable two pixels on capable with i of described i-1 and the position skew identical (horizontal offset is 4) of two adjacent pixels in vertical direction of described interpolation pixel, and offset direction, position contrary (skew left, a skew to the right).
, for the boundary direction of the incline direction that is tilted to the right described in obtaining detects information, need first determine corresponding calculation window.For example, can include in calculation window being less than or equal to 2 existing pixel with (i-1, j+4) and (i, j-4) two existing pixel horizontal-shifts, the width of described calculation window is 2, is highly 1.Particularly, in described calculation window, also comprise: (i-1, j+6) and (i, j-2) two locational existing pixels; (i-1, j+5) and (i, j-3) two locational existing pixels; (i-1, j+3) and (i, j-5) two locational existing pixels; (i-1, j+2) and (i, j-6) two locational existing pixels.Can find out, the definite incline direction of other existing pixels in described calculation window is also identical with the definite incline direction of (i-1, j+4) and (i, j-4) two existing pixels.It should be noted that, determining of calculation window is herein only for example, not in order to limit.In general, the size of calculation window is relevant to incline direction, larger with respect to the inclination angle of vertical direction, and calculation window also can be corresponding larger.For example, just relatively little by calculation window corresponding to the definite incline direction of (i-1, j+1) and (i, j-1) two locational existing pixels.Certainly, the size of the calculation window of each incline direction also can be identical, can regulate according to actual needs.
Calculate respectively the difference between the multiple existing pixels of described calculation window, calculate respectively (i-1, and (i j+6), j-2) difference between two locational existing pixels, (i-1, and (i j+5), j-3) difference between two locational existing pixels, (i-1, and (i j+3), j-5) difference between two locational existing pixels, (i-1, and (i j+2), j-6) difference and the (i-1 between two locational existing pixels, and (i j+4), j-4) difference between two locational existing pixels.
The absolute value of above-mentioned obtained difference is cumulative, obtain the difference degree between existing pixel on the described incline direction that described calculation window is corresponding, the boundary direction using described difference degree as incline direction corresponding to described calculation window detects information.Described cumulative computational process is represented by following formula:
Wherein, the mark that n is described incline direction, C
nfor existing pixel adjacent with interpolation pixel on described incline direction with respect to the horizontal offset of neighbor pixel in interpolation pixel vertical direction (conventionally larger with respect to the inclination angle of vertical direction, C
nlarger), k is the vertical position in the calculation window that described incline direction is corresponding, the horizontal level in the calculation window that l is corresponding described incline direction, kdifw is the relevant weight coefficient of calculation window that described incline direction is corresponding, f
w(i-1+k, j+C
n+ l) and f
w(i-1+k, j+C
n+ l) representative needs the pixel value of existing pixel of calculated difference respectively.For by (i-1, j+4) and (i, j-4) two definite incline directions of locational existing pixel, n=4, C
n=3, height_n=1, width_n=2.
In other implementations, also can will after above-mentioned obtained squared difference, add up, obtain the difference degree between existing pixel on the incline direction that described calculation window is corresponding, the boundary direction using described difference degree as incline direction corresponding to described calculation window detects information.
Still, take the cumulative account form of absolute value as example, the acquisition process that detects information for the boundary direction of the incline direction being tilted to the left also can carry out with reference to aforesaid way, and the cumulative computing formula of respective absolute values is as follows:
The wherein corresponding parameter of each parameter in can reference formula (1).
More than determine that selection range when calculation window is the existing pixel of the capable and i of i-1 on capable, in other implementations, also can be using the pixel in the adjacent lines that i-1 is capable and i is capable reference information during also as the described difference degree of acquisition.
Continue with reference to Fig. 2, due to (i, and (i+1 j+4), j-4) two incline directions and (i-1 that locational existing pixel is definite, and (i j+4), j-4) the definite incline direction of two existing pixels is identical, at definite (i-1, and (i j+4), j-4) when the definite incline direction of two existing pixels is corresponding calculation window, also can determine (i according to sample, and (i+1 j+4), j-4) (routine width described above is 2 to two definite calculation windows corresponding to incline direction of locational existing pixel, be highly 1 calculation window).Subsequently, the difference between the each existing pixel in two calculation windows is carried out to difference absolute value cumulative (can reference formula (1) and formula (2)) or difference square cumulative, finally to obtain boundary direction detection information.
With reference to Fig. 3, due to (i-1, and (i-2 j+4), j+12) the definite incline direction of two locational existing pixels with in (i-1, and (i j+4), j-4) on the extended line of the definite incline direction of two existing pixels, at definite (i-1, and (i j+4), j-4) when the definite incline direction of two existing pixels is corresponding calculation window, also can determine (i-1 according to sample, and (i-2 j+4), j+12) (routine width described above is 2 to two definite calculation windows corresponding to incline direction of locational existing pixel, be highly 1 calculation window).Subsequently, the difference between the each existing pixel in two calculation windows is carried out to difference absolute value and add up, described cumulative process can be represented by following formula:
Wherein, shift comprises central point on the incline direction of the described extended line horizontal offset with respect to interpolation pixel, and other parameters can reference formula (1).For (i, j-4), (i-1, j+4), (i-2, j+12) on the incline direction that comprises described extended line that three locational existing pixels are determined, (i-1, j+4) this locational pixel is described central point, and its horizontal offset with respect to interpolation pixel is 4.
Reference formula (2) and formula (3) also can be derived from the absolute value accumulation calculating formula of the incline direction being tilted to the left.
Certainly, with front similar, also can will after above-mentioned obtained squared difference, add up, obtain the difference degree between existing pixel on the incline direction that described calculation window is corresponding, the boundary direction using described difference degree as incline direction corresponding to described calculation window detects information.
Obtaining after boundary direction detection information, detect to obtain corresponding reliability by reliability, described reliability reflects that current boundary direction detects the possibility of direction corresponding to information as the boundary direction of interpolation pixel.
In order to ensure the accuracy of the boundary direction of the interpolation pixel finally obtaining, reduce the impact of the disturbing factors such as noise, obtaining after boundary direction detection information, can carry out consistency detection.The object of consistency detection is in order to obtain at the difference degree along pixel in current direction.And in interpolation pixel neighborhood of a point, on locus, the closer to current direction to be detected (through the incline direction of interpolation pixel), correlation is better, but noise resisting ability is poor, and institute's amount of information that provides is fewer; On the contrary, distance consistency check correlation far away is poor, but has good antinoise and information provides value, in practical application, should make Balancing selection.
Shown in Fig. 4, continue that the capable and i j between capable classifies example as and describes at i-1 with interpolation pixel, the direction that need carry out consistency detection (i-1, j+4) and (i, j-4) two the definite incline directions of existing pixel of serving as reasons., detect in the direction that direction corresponding to information is identical difference between pixel (consistency detection value) between two by obtaining in described neighborhood with described boundary direction, and detect information comparison with described boundary direction, obtain consistency detection result.
Described pixel between two can be the existing pixel in described neighborhood, for example in Fig. 4 by (i, j) with (i+1, j-8) the definite incline direction of two locational existing pixels with by (i-1, j+4) identical with the definite incline direction of (i, j-4) two existing pixels.; can obtain (i; j) with (i+1; j-8) difference between two locational existing pixels; and with (i-1; j+4) and boundary direction corresponding to the definite incline direction of (i, j-4) two existing pixels detect information comparison, obtain consistency detection result., using the difference of described difference and described boundary direction detection information as consistency detection result.
Or described pixel between two can be also to have other interpolation pixels in pixel and described neighborhood in described neighborhood, the pixel value of described other interpolation pixels has pixel interpolation around by it and obtains.For example in Fig. 4 by (i-1, j-4) the definite incline direction of the existing pixel on and the interpolation pixel that i-1 is capable and the j-8 of i between capable lists with by (i-1, j+4) identical with the definite incline direction of (i, j-4) two existing pixels.Wherein, the interpolation pixel that the j-8 between the capable and i of i-1 is capable lists can be according to existing pixel information around, and by bilinear interpolation, the existing interpolation method such as bicubic interpolation method obtains.For example, can obtain according to (i-1, j-8) and (i, j-8) two existing pixel interpolation, obtain by its vertical direction interpolation.; can obtain (i-1; difference between the existing pixel j-4) and i-1 is capable and the j-8 of i between capable lists interpolation pixel; and with (i-1; and (i j+4); j-4) two definite boundary directions corresponding to incline direction of existing pixel detect information comparison, obtain consistency detection result., using the difference of described difference and described boundary direction detection information as consistency detection result.Difference between described existing pixel and described interpolation pixel can be calculated and be obtained by following formula:
chkdifL
n=abs(f
w(i-1,j)-(f
w(i-1,j+c
n)+f
w(i,j+c
n))/2) (4)
The wherein corresponding parameter of each parameter in can reference formula (1).
Similarly, also can choose (i+1, difference and described boundary direction between the existing pixel j+4) and i is capable and the j+8 of i+1 between capable lists interpolation pixel detect information comparison, obtain consistency detection result, and it is calculated and repeated no more in detail.
Or described pixel between two can also be other interpolation pixels in described neighborhood, the pixel value of described other interpolation pixels is by its existing pixel interpolation acquisition around.For example, the definite incline direction of the interpolation pixel that the capable and i j+8 between capable lists by i-1 and the interpolation pixel that i is capable and the j of i+1 between capable lists with by (i-1, j+4) identical with the definite incline direction of (i, j-4) two existing pixels.Wherein, described two interpolation pixels can be according to having pixel information around, and by bilinear interpolation, the existing interpolation method such as bicubic interpolation method obtains.For example, the interpolation pixel that the j between the capable and i+1 of i is capable lists can obtain according to (i, j) and (i+1, j) two existing pixel interpolation, obtains by its vertical direction interpolation.And the interpolation pixel that i-1 is capable and the j+8 of i between capable lists can obtain according to (i-1, j+8) and (i, j+8) two existing pixel interpolation, obtain by its vertical direction interpolation.Subsequently, boundary direction corresponding incline direction definite two existing pixels of the difference between described two interpolation pixels and (i-1, j+4) and (i, j-4) is detected to information comparison, obtain consistency detection result., using the difference of described difference and described boundary direction detection information as consistency detection result.
Above-mentioned difference between pixel is between two all in interpolation pixel neighborhood of a point, to choose one group of difference to compare, in other implementations, can also choose many group differences, many groups difference is averaged, and mean value and described boundary direction are detected to information comparison, obtain consistency detection result.For example, the difference between the difference between above-mentioned two existing pixels, existing pixel and other interpolation pixels, the difference between other interpolation pixels are averaging, described mean value and described boundary direction are detected to information comparison.Certainly, also can from described many group differences, choose one group of maximum difference, one group of difference of described maximum and described boundary direction are detected to information comparison, obtain consistency detection result.The consistency detection result obtaining is less, represents that current direction to be detected is larger as the possibility of the boundary direction of interpolation pixel.
Fig. 5 shows the another kind of mode schematic diagram of consistency detection, its consistency detection corresponding with Fig. 4 is similar, can be by choosing one group of difference between pixel, the difference of choosing between many group pixels is averaged, choose one group of difference of maximum of the difference between many group pixels, detect information comparison with described boundary direction, obtain consistency detection result.Its consistency detection corresponding with Fig. 4 difference is to choose the mode of pixel between two.For example, pixel is chosen along two existing pixels on the definite incline direction of (i-1, j+4) and (i, j-4) two existing pixels between two: (i-1, and (i-2, j+12) locational two existing pixels j+4).
In real image processing, between pixel, also there is the situation such as intersect edges and angle, for further increasing the accuracy that reliability detects, also can detect the correlation of current direction to be detected.Particularly, choose along described boundary direction and detect direction corresponding to information, two pixels adjacent with described interpolation pixel, respectively with selected two pixels adjacent two pixels in vertical direction, and with described interpolation pixel adjacent two pixels in vertical direction, the line between selected multiple pixels and described interpolation pixel forms two right-angled triangles up and down that share hypotenuse; Upper and lower two triangles are carried out to correlation detection, obtain relevance detection results.
Shown in Fig. 6, with the capable pixel A of i-1
x, A
0, A
xthe pixel B capable with i
-xbetween line form upper right-angled triangle, the hypotenuse of described upper right-angled triangle is through interpolation pixel.With the capable pixel B of i
x, B
0, B
-xthe pixel A capable with i-1
xbetween line form lower right-angled triangle, the hypotenuse of described lower right-angled triangle passes through interpolation pixel equally., described correlation detection realizes by following formula:
adj
R=max(abs(B
-x-avg)-fnt,0)+max(abs(A
x-avg)-fnt,0)
fnt=min(abs(A
0-B
0)/2,fdelta)
fdelta=max(0,C
0-C
1*min(delta
up,delta
down))
delta
up=max(A
-x,A
0,A
x,B
-x)-min(A
-x,A
0,A
x,B
-x)
delta
down=max(B
-x,B
0,B
x,A
x)-min(B
-x,B
0,B
x,A
x)
Wherein, A
-x, A
0, A
x, B
x, B
0, B
-xbe respectively the pixel value of institute's corresponding pixel points, avg is A
0, B
0the average pixel value of institute's corresponding pixel points, C
0for the threshold value relevant to the bit wide of current pending image, C
1for with delta
up, delta
downrelevant weight coefficient, C
1span can be 0 to 511.
Correlation detection value adj
rless, show that direction to be detected is larger as the possibility of the boundary direction of interpolation pixel.
Fig. 7 shows the another kind of mode schematic diagram of correlation detection, and its correlation detection corresponding with Fig. 6 is similar, and difference is that upper and lower leg-of-mutton formation is different.Shown in Fig. 7, with the capable pixel A of i-1
-x, A
0, A
xthe pixel B capable with i
xbetween line form upper right-angled triangle, the hypotenuse of described upper right-angled triangle is through interpolation pixel.With the capable pixel B of i
x, B
0, B
-xthe pixel A capable with i-1
-xbetween line form lower right-angled triangle, the hypotenuse of described lower right-angled triangle passes through interpolation pixel equally., described correlation detection realizes by following formula:
adj
L=max(abs(A
-x-avg)-fnt,0)+max(abs(B
x-avg)-fnt,0)
fnt=min(abs(A
0-B
n)/2,fdelta)
fdelta=max(0,C
0-C
1*min(delta
up,delta
down))
delta
up=max(A
x,A
0,A
-x,B
x)-min(A
x,A
0,A
-x,B
x)
delta
down=max(B
-x,B
0,B
x,A
-x)-min(B
-x,B
0,B
x,A
-x)
Wherein, A
-x, A
0, A
x, B
x, B
0, B
-xbe respectively the pixel value of institute's corresponding pixel points, avg is A
0, B
0the average pixel value of institute's corresponding pixel points, C
0for the threshold value relevant to the figure place of current pending image, C
1for with delta
up, delta
downrelevant weight coefficient, C
1span can be 0 to 511.
Correlation detection value adj
lless, show that direction to be detected is larger as the possibility of the boundary direction of interpolation pixel
For obtaining more comprehensive reliability testing result, make reliability can reflect more realistically the possibility of direction to be detected as the boundary direction of the pixel of interpolation, can also, according to described consistency detection value and described correlation detection value and direction corresponding to described boundary direction detection information, obtain gradient testing result.
Described gradient detects and comprises:
Compare the boundary direction detection information with respect to the incline direction that is tilted to the left and is tilted to the right of the inclination angle of vertical direction identical (being that position offset is identical);
If described in the boundary direction of the incline direction that is tilted to the right described in being more than or equal to of the boundary direction detection information of the incline direction that is tilted to the left detect information, gradient detected value corresponding to incline direction being tilted to the left described in obtaining according to corresponding consistency detection value and correlation detection value;
Otherwise, according to gradient detected value corresponding to incline direction being tilted to the right described in corresponding consistency detection value and the acquisition of correlation detection value.
For gradient detected value corresponding to the incline direction being tilted to the left, it is calculated and is obtained by following formula:
grad
n=kwd[n]*[djfL
n-kneg*difR
n]-kwc[n]*chkdifR
n-kc*adj
R
For gradient detected value corresponding to the incline direction being tilted to the right, it is calculated and is obtained by following formula:
grad
n=kwd[n]*[difR
n-kneg*difL
n]-kwc[n]*kchk*chkdifL
n-kc*adj
L
Wherein kwd[n] be weight coefficient relevant to boundary direction detection information on each incline direction, Kneg is gradient factor, kwc[n] and be the relevant weight coefficient of consistency detection on each incline direction, Kc is the weight coefficient that correlation detection is relevant.Above-mentioned coefficient all can regulate according to practical application, repeats no more herein.
The grad calculating
nindicating that this incline direction n is that the possibility of interpolation pixel boundary direction is larger greatlyr, by certain searching algorithm and tactful maximizing grad
n, its corresponding direction can be thought the boundary direction of current interpolation pixel.In search procedure, can limit searched for scope by certain Heuristics, can improve the speed of search, also can reject impossible boundary direction, make obtained direction more approach real boundary direction.For example can be current in the pixel value distributed degrees laterally and on vertical by calculating, or calculate the current some method such as simple sobel operator gradient direction around, estimate border possibility direction and determine maximum value search scope.
In addition, in order further to improve the accuracy of the boundary direction obtaining, also can integrated survey described in the continuity of adjacent direction of incline direction.Described continuity refers to, supposes that the border showing on incline direction n is the border that is tilted to the left, and requiring on incline direction n-1 and incline direction n+1 at least one or 2 is all the border that is tilted to the left.Particularly, realize by the continuity detected value of more each incline direction.The continuity of described incline direction detects by following formula and calculates and obtain:
positive
n=sign(difL
n-difR
n)
At difL
n-difR
nvalue be greater than at 0 o'clock, positive
n=1; At difL
n-difR
nvalue equal at 0 o'clock, positive
n=0; At difL
n-difR
nvalue be less than at 0 o'clock, positive
n=-1.
This continuity is defined as:
positive
n=positive
n-1or/and positive
n=positive
n+1
Be positive
nas long as and positive
n-1, positive
n+1in any one equate, just can think that adjacent direction meets continuity, if certain positive
nwith positive
n-1, positive
n+1all equate, adjacent direction also meets continuity.
From each incline direction to be detected, choose Grad maximum and meet the boundary direction of successional incline direction as interpolation pixel., the interpolation based on boundary direction is calculated by following formula:
Boundary direction for the incline direction being tilted to the right as interpolation pixel
value
max=(f
w(i,j+C
n)+f
w(i+1,j-C
n)+1)/2
Boundary direction for the incline direction being tilted to the left as interpolation pixel
value
max=(f
w(i,j-C
n)+f
w(i+1,j+C
n)+1)/2
Wherein, C
nfor existing pixel adjacent with interpolation pixel on described incline direction is with respect to the horizontal offset of neighbor pixel in interpolation pixel vertical direction, f
w(i, j+C
n), f
w(i+1, j-C
n) or f
w(i, j-C
n), f
w(i+1, j+C
n) pixel value of existing pixel used while representing respectively along right bank boundary direction or left bank boundary direction interpolation.
For preventing that possible false border from causing image fault phenomenon, also in final interpolation calculation, introduce protection mechanism, with the mean value value of two existing pixels adjacent in interpolation pixel vertical direction
avgas border do not exist or the undesirable situation of testing result under interpolation, and in other cases, regulate the scale of this mean value and the interpolation based on boundary direction according to the gradient of boundary direction, may with what avoid that distortion appears in image.The quality of Boundary Detection result can directly be passed through calculated Grad and obtain, and this Grad is larger, and corresponding border reliability is better, and vice versa.Obtaining integration percentage by the size of Grad is calculated as follows:
Wherein, the check threshold value to gradient when ThC is integration percentage calculating, associated weight coefficient when ThA is integration percentage calculating, the span of 2d_alpha is [0,1].
Conventionally, ThA and ThC all can calculate by test of many times, and adjust according to the image effect after interpolation, to obtain comparatively suitable value.
The interpolation result of the interpolation pixel finally obtaining is as follows:
Dresult=value
avg*(1-2d_alpha)+value
max*2d_alpha
In sum, interlace-removing method of the present invention detects by the reliability of all directions to through interpolation pixel, determine the possibility (reliability) of all directions as the boundary direction of described interpolation pixel, and therefrom select direction that reliability the is the highest boundary direction as interpolation pixel, the interpolation of described interpolation pixel is combined to the interpolation of vertical direction and the interpolation based on boundary direction, and all relevant to described reliability, therefore, interpolation result is more accurate, has obtained and has gone interlacing effect preferably
More than disclose many aspects of the present invention and execution mode, it will be understood by those skilled in the art that other side of the present invention and execution mode.In the present invention, disclosed many aspects and execution mode, just for illustrating, are not limitation of the invention, and real protection range of the present invention and spirit should be as the criterion with claims.