TWI429274B - Image processing system having scaling and sharpness device and method thereof - Google Patents

Description

Image processing system with zooming and sharpening device and method thereof

The present invention relates to an image processing system and method thereof, and more particularly to an image processing system having a zooming and sharpening device and a method thereof.

In general, the conventional image processing technology needs to scale a video image to any size. The usual way is to separately image the images separately in the horizontal direction and the vertical direction. It is easy to cause image quality deterioration, such as forming a jagged edge, called a stair stepping.

In the prior art, when the image scaling process is performed, the directional scaling interpolation method is used to improve the image quality to reduce the staircase effect. Basically, when the directional scaling interpolation method is performed on the image, the image has a plurality of input pixels, and each input pixel is regarded as a reference pixel in sequence, and each input pixel corresponds to a two-dimensional The window, such as the dimension of the window, is 6×8. When re-establishing an enlarged image through the original size image, an interpolation process must be performed between the original input pixels to form upsampled pixels between successive line segments, and to the original The input pixels are interpolated to insert new pixels into the magnified image.

In the prior art, the algorithm using directional scaling interpolation is to analyze the local gradient characteristics of the input pixels in the image, and then according to these in the lowest frequency direction or The local gradient characteristic of the minimum gradient level is used to perform the interpolation, and the interpolation is performed along the horizontal direction or the local gradient characteristic orthogonal to the lowest frequency direction.

In addition, when the image frames are sequentially played, each image frame is composed of three dimensional directions, including a vertical direction, a horizontal direction, and a temporal direction (ie, a time-based). However, the noise doped in the temporal direction erroneously affects the calculation of the local gradient characteristics along the lowest frequency direction, thus affecting the decision of other directions orthogonal to the lowest frequency direction. Therefore, the noise is doped. The input pixels flash in an image frame along at least two different time directions (the correct one can only be the same time direction), for example, in the same image area of a still image frame, in the time direction "t" The lowest frequency direction is determined by the angle "A", but the lowest frequency direction in the time direction "t+1" is determined by the angle "B", where the angle "A" and the angle "B" are different. It is called "flick" or "sparkle" effect, so when playing an image, the angle "B" and the angle "A" are not equal, resulting in the lowest frequency direction. Move so that the input pixels flash at the same image (different time points, ie, front and back screens).

In addition, in the prior art, a sharpness procedure is performed on the scaled input pixels in the horizontal direction and the vertical direction, respectively, but the image quality of the scaled pixels subjected to the sharpness processing is not good, for example, forming a sawtooth. Jagged edge. In other words, when the sharpness processing is performed, the shape of the sawtooth edge has a shape of a jagged edge along the lowest frequency direction of the scaled pixel, and thus it is necessary to develop a new image processing system to overcome the above problem.

It is an object of the present invention to provide an image processing system having a zooming and sharpening device and method thereof to solve the problem of image flicker.

Another object of the present invention is to provide an image processing system having a zooming and sharpening device and a method thereof. When performing image scaling, the image is correctly decoded in a direction orthogonal to the lowest frequency direction to solve image generation. The problem of jagged edges.

To achieve the above object, the present invention provides an image processing system having a zooming and sharpening device and a method thereof, the image processing system including a global frequency detecting device, a gradient calculating sum unit, a threshold adjusting device, and an image mixing device, the image mixing The device includes an orthogonal angle determining module, a minimum frequency determining module, a weight evaluation unit, a first zoom/clear module, a second zoom/clear module, a noise filter, and a mixing unit.

The global frequency detecting unit is configured to detect a frequency of the plurality of input pixels to calculate vertical frequency levels of the input pixels along a vertical direction, and calculate a horizontal frequency level of the input pixels along a horizontal direction. The gradient calculation summation unit is respectively along a plurality of directions, and a set of gradient luminance levels is calculated according to a part of the input pixels. The threshold adjustment device is coupled to the global frequency detecting unit, and adjusts the first threshold and the second threshold according to the relationship between the vertical frequency level and the horizontal frequency level. The image mixing device is coupled to the gradient calculation summation unit, by using a first threshold value from the threshold value adjustment device for determining an orthogonal angular direction, and by using a second threshold value from the threshold value adjustment device, Determining a lowest frequency direction, wherein the image mixing device forms a first pattern related to the orthogonal angular direction, and forms a second pattern related to the lowest frequency direction, and the image mixing device is mixed according to a weighting factor value The first pattern and the second pattern.

The global frequency detecting unit calculates the vertical frequency levels of all the input pixels globally and coarsely in a vertical direction, and is globally and coarsely along the horizontal direction. Calculate the horizontal frequency level of all input pixels. That is, the vertical frequency level and the horizontal frequency level system related to the first threshold value and the second threshold value are all in the two-dimensional window along the vertical direction and the horizontal direction, and all input pixels are calculated in a global manner. The sum of the difference between the brightness levels, when calculating the vertical and horizontal frequencies, each input pixel is sequentially treated as a reference pixel to form each of the two Dimensional windows, such as the dimension of a window, are 6 x 8.

When the weighting factor is increased, the interpolated result indicates that the input pixel approaches the lowest frequency direction. When the weighting factor decreases, the interpolated result indicates that the input pixel approaches the orthogonal angular direction. When the image is played, the weighting factor is dynamically adjusted to increase or decrease the weighting factor to dynamically correct the interpolation result along the direction of the lowest frequency and the orthogonal angle, thus coming from the noise and affecting the direction along the lowest frequency. The flicker effect of the interpolated results will be removed from the image.

The image processing method of the present invention comprises the following steps:

(a) receiving a plurality of input pixels using the global frequency detecting means and the gradient calculating summing unit.

(b) using the global frequency detecting unit to calculate the vertical frequency levels of the input pixels along the vertical direction, and to calculate the horizontal frequency levels of the input pixels along the horizontal direction.

(c) adjusting the first threshold and the second threshold according to the relationship between the vertical frequency level and the horizontal frequency level by using a threshold adjustment device.

(d) Calculating a summation unit using a gradient, respectively, along a set of directions and according to a portion of the input pixels, to calculate a set of gradient brightness levels.

(e) using an orthogonal angle determining module to compare the set of gradient luminance levels to determine whether the orthogonal direction is along a horizontal direction or a vertical direction to form a first pattern using the first threshold.

(f) using the lowest frequency determining module to compare the set of gradient luminance levels to determine a lowest frequency direction along one of the set of directions to form a second pattern using the second threshold.

(g) using the image mixing device to mix the first pattern and the second pattern from the global frequency detecting unit according to the weight factor value, wherein the first pattern is related to the first threshold, and the second pattern is related to the second aspect Limit.

In order to make the above-mentioned contents of the present invention more comprehensible, the preferred embodiments are described below, and the detailed description is as follows:

1 is a schematic diagram of a plurality of input pixels of an image in accordance with an embodiment of the present invention, wherein a direction scaling mechanism is performed on the pixels in the lowest frequency direction 100 and the orthogonal angle direction 102 using an image processing system. Figure 1 shows a target output pixel P _tar , four central input pixels P23, P24, P33, P34 forming a rectangular area around the target output pixel P _tar , and the brightness level of the pixel P _tar is calculated along the input pixel The lowest frequency direction 100 and the brightness level of the orthogonal angular direction 102 are generated, wherein the orthogonal angular direction 102 is horizontal or vertical, as shown in FIG. The size of the image of the dimensional level is 6×8, and the direction of the orthogonal angle 102 is along the horizontal direction 102. It should be noted that the window size of the input pixel can be any dimension. Bit.

The input pixel of the image is identified as shown in FIG. 1 , and the pixel P00 located in the upper left corner of the window is used as the reference position coordinate, and the target output pixel P _tar is known to be represented with respect to the reference pixel P23, wherein the reference pixel P23 is located in the rectangle. In the upper left corner of the area, in general, the coordinate value of the target output pixel P _{tar is} a non-integer.

According to the directional scaling algorithm of the directional scaling mechanism of the present invention, the final interpolation result is as shown in the formula E1:

Final interpolation result of target output pixel P _tar = (intp_min) * (weighting factor value) + (intp_cross) * (1-weighting factor value)... (E1)

Where intp_min is the interpolated value along the lowest frequency direction, and intp_cross is the interpolated value along the orthogonal angular direction, wherein the orthogonal angular direction is either horizontal or vertical, and the lowest frequency direction is except positive Angle of direction other than the angle of intersection. The lowest frequency direction and the orthogonal angular direction of the image are selected from a plurality of gradient directions, and the brightness levels of the input pixels are calculated along various angles to determine the gradient directions, as shown in FIG. 2A.

In the embodiment of the first and second F-picture lowest frequency directions 100, the interpolation result "intp_min" is determined by interpolation of intermediate points m0 to m5, where m0 to m5 can be regarded as virtual pixels in the lowest frequency direction (virutal) Pixel), the line 104 of these intermediate points passes through the target output pixel P _tar and is perpendicular to the lowest frequency direction, that is, the lowest frequency direction 100, and the intermediate points m0 to m5 are pixels P22, P22, P14, P31, P23, P15, P32 , P24, P33, P25P42, P34, P26, P43, and P35 are determined by interpolation values in the lowest frequency direction 100. Similarly, in the embodiment of the orthogonal angle direction 102 of FIG. 1 and FIG. 2C, the interpolation result "intp_cross" is determined by interpolation of the intermediate points m0 to m5, where m0 to m5 can be regarded as orthogonal angular directions. Virtual pixel, the line 104 of these intermediate points passes through the target output pixel P _tar and is perpendicular to the lowest frequency direction, that is, the orthogonal angle direction 102, as shown in FIG. 2C, the intermediate points m0 to m5 are Pixels P31, P21, P32, P22, P43, P33, P23, P13, P44, P34P24, P14, P35, P25, P36 and P26 are determined along the interpolated value in the lowest frequency direction 100. Therefore, the target output pixel P _tar can be determined by using the interpolation result "intp_min" and the interpolation result "intp_cross".

2A is a schematic diagram of a plurality of gradient directions associated with an input pixel in accordance with an embodiment of the present invention, wherein when the image processing system performs the scaling image step, the lowest frequency direction and the orthogonal angular direction may be determined. The lowest frequency direction and the orthogonal angle direction are respectively selected from the gradient directions, for example, the gradient directions are the direction “0” to the direction “9”, and it should be noted that according to the arrangement of the windows of different numbers of input pixels, The number of gradient directions may be greater than or less than ten directions from the direction "0" to the direction "9".

In an embodiment, the gradient level along the lowest frequency direction is the minimum brightness change amount, and in the lowest frequency direction, the minimum gradient level is a set of brightness gradient levels along the right side plane and the left side plane The calculation of the brightness gradient level is generated, wherein the brightness gradient level of the right side plane includes directions "1", "3", "5", and "7", and the brightness gradient level of the left side plane includes the direction "2", "4", "6" and "8", those skilled in the art should note that depending on the arrangement size of the input pixels of the image, the minimum brightness variation can be selected by the number of the above eight directions. The gradient level along the orthogonal angle is selected from one of a plurality of luminance gradient levels, and the luminance gradient level is generated by calculation of orthogonal angular directions (eg, direction "0" and direction "9").

The 2B-2G diagram is a schematic diagram for calculating the sum of absolute values of luminance differences according to an embodiment of the present invention, wherein the luminance difference refers to a luminance difference value between different pairs of input pixels along different gradient directions.

As shown in Fig. 2B, along the direction "9" (horizontal direction), the absolute difference sum (sum) of the brightness gradient level is calculated by the formula E2:

S0=|P14-P13|+|P23-P22|+|P24-P23|+|P25-P24|+|P33-P32|+|P34-P33|+|P35-P34|+|P44-P43|. ........(E2)

As shown in Fig. 2C, along the direction "0" (vertical direction), the sum of the absolute differences of the brightness gradient levels is calculated using the formula E3:

S9=|P32-P22|+|P43-P33|+|P33-P23|+|P23-P13|+|P44-P34|+|P34-P24|+|P24-P14|+|P35-P25|. ........(E3)

As shown in Fig. 2D, along the direction "1", the sum of the absolute differences of the brightness gradient levels is calculated using the formula E4:

S1=|P32-P13|+|P42-P23|+|P33-P14|+|P43-P24|+|P34-P15|+|P44-P25|.........(E4)

As shown in Fig. 2E, along the direction "3", the sum of the absolute differences of the brightness gradient levels is calculated using the formula E5:

S3=|P32-P23|+|P23-P14|+|P42-P33|+|P33-P24|+|P24-P15|+|P43-P34|+|P34-P25|....... ..(E5)

As shown in Figure 2F, along the direction "7", the sum of the absolute differences of the brightness gradient levels is calculated using Equation E6:

S5=|P22-P14|+|P31-P23|+|P23-P15|+|P32-P24|+|P33-P25|+|P42-P34|+|P34-P26|+|P43-P35|. ........(E6)

As shown in Fig. 2G, along the direction "7", the sum of the absolute differences of the brightness gradient levels is calculated using Equation E7:

S7=|P22-P15|+|P30-P23|+|P23-P16|+|P31-P24|+|P24-P17|+|P32-P25|+|P40-P33|+|P33-P26|+ |P41-P34|+|P34-P27|.........(E7)

Since "2", "4", "6", and "8" are respectively symmetric with respect to "1", "3", "5", and "7", as shown in Fig. 2A, therefore, along the direction "2", The sum of the absolute differences of the luminance gradient levels of "4", "6", and "8", S2, S4, S6, and S8 can use the sum of the absolute differences of the luminance gradient levels of the above formulas E4 to E7, S1, S3, and S5. And the S7 calculation is generated. According to the sum of the absolute differences of the calculated brightness gradient levels, the gradient level along the lowest frequency direction is the minimum brightness change value selected from S1, S2, S3, S4, S5, S6, S7, and S8, as in the above formula. E4 to E7. Similarly, the gradient level along the orthogonal angular direction is the minimum brightness change value selected from S0 and S9.

Figure 3 is a block diagram of an image processing system 300 in accordance with an embodiment of the present invention. The image processing system 300 includes a global frequency detecting unit 302, a gradient-calculating summation unit 304, a threshold-adjusting device 306, and an image blending device. The global frequency detecting unit 302 is coupled to the threshold adjusting device 306. The gradient calculating unit 304 is coupled to the image mixing device 308. The threshold adjusting device 306 is coupled to the image mixing device 308. . The image mixing device 308 further includes an orthogonal angle determining module 310, a lowest frequency determining module 312, a weight estimating unit 314, a first zooming/clearing module 316, a second zooming/clearing module 318, and a noise filter 320. And a mixing unit 322. The gradient calculation summation unit 304 is coupled to the orthogonal angle determination module 310, the lowest frequency determination module 312, and the weight estimation unit 314. The first zoom/clear module 316 is coupled to the orthogonal angle determination module 310. To the mixing unit 322 , the second zooming/clearing module 318 is coupled to the lowest frequency determining module 312 to the mixing unit 322 , and the noise filter 320 is coupled to the weight estimating unit 314 to the mixing unit 322 .

The global frequency detecting unit 302 is configured to detect a frequency of the plurality of input pixels to calculate a vertical frequency level of the input pixels along a vertical direction, and calculate a level of the input pixels along a horizontal direction. Frequency level, wherein one of the input pixels is designated as a reference pixel to form a two-dimensional window. The gradient calculation summation unit 304, respectively, along a plurality of directions, such as direction "0" to direction "9", and based on a portion of the input pixels, is used to calculate a set of gradient luminance levels, as shown in FIG. 2A. The threshold adjustment device 304 adjusts the first threshold and the second threshold according to the relationship between the vertical frequency level from the global frequency detecting unit 302 and the horizontal frequency level. First, the image mixing device 308 is used to determine the orthogonal angle direction by using the first threshold value from the threshold value adjusting device 306, and is used by the second threshold value from the threshold value adjusting device 306. Determining a lowest frequency direction, wherein the image mixing device 308 forms a first pattern associated with the orthogonal angular direction and forms a second pattern associated with the lowest frequency direction, and the image mixing device is based on a weighting factor Value) mixing the first pattern with the second pattern. In the present invention, the first threshold is stored in the value of the register of the threshold adjusting device 306, and is related to the orthogonal angle determining module 310, and can be adjusted; the second threshold is stored. The value of the register of the threshold adjusting device 306 is related to the lowest frequency determining module 312 and can be adjusted.

The global frequency detecting unit 302 calculates the vertical frequency levels of all input pixels globally and coarsely in a vertical direction and globally and roughly along the horizontal direction. (coarsely) Calculate the horizontal frequency level of all input pixels. That is, in the two-dimensional window, along the vertical direction and the horizontal direction, the vertical frequency level related to the first critical value and the second critical value is calculated globally according to the sum of the absolute differences of the luminance levels of all the input pixels. With the horizontal frequency level, when calculating the vertical and horizontal frequency levels, each input frequency is treated as a reference pixel to form a two-dimensional window, for example, 6x8.

The first zoom/clear module 316 performs one directional scaling and sharpness steps according to one of a vertical direction or a horizontal direction, and the second zoom/clear module 318 is based on the lowest frequency direction (except The vertical direction and the horizontal direction are performed to perform a directional scaling and sharpness step.

Referring to FIG. 1 and the above formula E1, when the weighting factor is increased, the interpolation result indicates that the input pixels tend to the lowest frequency direction, and when the weighting factor is decreased, the interpolation result indicates that the input pixels tend to The cross-angle angle direction dynamically adjusts the weighting factor value when the image is played, that is, increases or decreases the weighting factor value, so that the edge The interpolation results in the lowest frequency direction and the orthogonal angle direction can be dynamically corrected, so that the sparkles that affect the interpolation result due to noise interference along the lowest frequency direction can be filtered in the image. .

Referring to FIG. 1 and FIG. 3, the global frequency detecting unit 302 further includes a vertical frequency detecting unit 302a and a horizontal frequency detecting unit 302b. The vertical frequency detecting unit 302a calculates the relevant input pixels along the vertical direction. a vertical frequency level, the horizontal frequency detecting unit 302b calculates a horizontal frequency level related to the entire input pixel along a horizontal direction, wherein the horizontal frequency level "hsum(n, m)" is expressed as the following formula (E8) ), the vertical frequency level "vsum(n,m)" is expressed as the following formula (E9):

Here, "n" and "m" are reference pixel indexes, which respectively represent a column position indicator and a column position, and the reference pixel is one input pixel in the two-dimensional window. Where "i" is the column index associated with the reference pixel index "n", "j" is the column index associated with the reference pixel index "m", and reg_hedge_th1 and reg_vedge_th1 are frequency thresholds The value "frequency", the symbol "min" indicates the minimum of the sums and the symbol "abs" indicates the sum of the absolute differences of the brightness levels. According to the formulas (E8) and (E9), each input pixel is regarded as a reference pixel, and each reference pixel corresponds to a two-dimensional window, for example, a 6×8 window size, so that formulas (E8) and (E9) can be utilized. The calculation produces a vertical frequency level along the vertical direction and a vertical frequency level along the horizontal direction.

The threshold adjustment device 306 adjusts the first threshold and the second threshold according to the relationship between the horizontal frequency level "hsum(n, m)" and the vertical frequency level "vsum(n, m)", The orthogonal angle determining module 310 and the lowest frequency determining module 312 are controlled so that the orthogonal angle determining module 310 and the lowest frequency determining module 312 respectively determine the orthogonal angular direction and the lowest frequency direction correctly. An example of the calculation result of the orthogonal angle determining module 310 is as follows.

When the horizontal frequency level is greater than the vertical frequency level, an orthogonal angle direction associated with the input pixel approaches the vertical direction, and when the vertical frequency level is greater than the horizontal frequency level, associated with the input pixel The orthogonal angular direction approaches the horizontal direction. The lowest frequency direction is the direction of the minimum brightness change associated with the input pixel. In an embodiment, the vertical frequency detecting unit 302a is a high-pass filter, and the horizontal frequency detecting unit 302b is a high-pass filter, and the threshold adjusting device 306 determines the horizontal frequency. An example of five relationships between the level "hsum(n, m)" and the vertical frequency level "vsum(n, m)".

In the first example, when the horizontal frequency level "hsum(n, m)" is greater than a maximum horizontal threshold (the maximum horizontal threshold) and the vertical frequency level "vsum(n, m)" is less than a minimum vertical threshold (minimum vertical threshold), the orthogonal angular direction of the entire input pixel is globally approaching to the vertical direction (ie, the low frequency direction), that is, the vertical direction is the determined low frequency direction, and the preset is The absolute difference sum of the values (dscale_sad_cross_th) is the largest value, and the orthogonal factor (dscale_sad_cross factor) is the smallest value. The orthogonality factor (dscale_sad_cross_factor) can be read by a quadrature factor register set in the threshold adjustment device 306. When the orthogonal factor (dscale_sad_cross factor) is a small value, the cross alpha (that is, the weight factor value along the orthogonal direction) is reduced. The orthogonal sum of the absolute difference (SAD) threshold (dscale_sad_cross_th) may be read by the orthogonal absolute sum sum register set in the threshold adjusting means 306, when the orthogonal absolute difference When the sum threshold is a large value, the cross alpha (i.e., the weight factor value along the orthogonal direction) is reduced. The decrease in the orthogonal value (cross_alpha) means that the chance of selecting the vertical direction as the orthogonal angle becomes larger.

In the second example, when the vertical frequency level "vsum(n, m)" is greater than a maximum vertical threshold and the horizontal frequency level "hsum(n, m)" is less than a minimum level The minimum horizontal angle is globally related to the horizontal direction (ie, the low frequency direction), that is, the horizontal direction is the determined low frequency direction, and the positive direction is positive. The absolute difference sum threshold (dscale_sad_cross_th) of the intersection is the smallest value, and the orthogonal factor (dscale_sad_cross factor) is the largest value. The orthogonality factor (dscale_sad_cross_factor) can be read by a quadrature factor register set in the threshold adjustment device 306. When the cross factor is a large value, the cross alpha increases. The orthogonal sum of the absolute difference (SAD) threshold (dscale_sad_cross_th) may be read by the orthogonal absolute sum sum register set in the threshold adjusting means 306, when the orthogonal absolute difference When the sum threshold is a small value, the cross alpha (i.e., the weight factor value along the orthogonal direction) increases. An increase in the orthogonal value (cross_alpha) means that the chance of selecting the horizontal direction as the orthogonal angle direction becomes larger.

In the third example, when the vertical frequency level "vsum(n, m)" is greater than a maximum vertical threshold and the horizontal frequency level "hsum(n, m)" is a maximum level The threshold (maximum horiaontal threshold) is related to the fact that all the input pixels are in a check board state. In this case, the horizontal direction or the vertical direction is not determined as the low frequency direction, and the vertical direction is viewed at this time. For the preset value in the low frequency direction, the setting is as described in the first example above. The orthogonal factor (dscale_sad_cross_factor) can be read by the orthogonal factor register set in the threshold adjusting device 306, and the sum of absolute difference (SAD) threshold (dscale_sad_cross_th) can be The orthogonal absolute difference sum set in the threshold adjustment means 306 is read by the register.

In the fourth example, when the above three exemplary conditions are not satisfied, when the vertical frequency level "vsum(n, m)" is a multiple of the horizontal frequency level "hsum(n, m)", With respect to the vertical direction, the orthogonal angular direction associated with the entire input pixel approaches the horizontal direction (ie, the low frequency direction), that is, the horizontal direction is the low frequency direction (but not determined by the second example above) Therefore, the orthogonal absolute difference sum threshold (dscale_sad_cross_th) is a small value, and the orthogonal factor (dscale_sad_cross factor) is a larger value, so that the horizontal direction is selected as the opportunity change of the last orthogonal angle direction. Big.

In the fifth example, the orthogonal angular direction of the all input pixels approaches the preset direction, that is, the vertical direction, but the intensity is not as good as the first example and the third example described above, so the orthogonal absolute difference The value sum threshold (dscale_sad_cross_th) is a larger value, and the orthogonal factor (dscale_sad_cross factor) is a smaller value, so that the chance that the vertical direction is selected as the last orthogonal angle direction becomes larger.

As described in the above example, according to the global horizontal frequency level "hsum(n, m)" and the vertical frequency level "vsum(n, m)", even if a part of the pixels in the image are subjected to Due to the effects of noise, the noise still cannot change the global gradient of the input pixel (in the direction of the orthogonal angle).

Referring to FIG. 2A and FIG. 3 , the orthogonal angle determining module 310 determines one of the first patterns according to the first threshold value in one of the vertical direction or the horizontal direction. The lowest frequency decision module 312 determines the second pattern according to the second threshold value in any one of a set of directions (excluding the vertical direction and the horizontal direction).

In one embodiment, the orthogonal angle decision module 310 uses the following program segments to determine the cross alpha, the cross angle, and the cross absolute difference (cross SAD):

According to the above procedure, SAD (sad[i][j][9]) along the direction "9" (horizontal direction) and the sum of the absolute difference (SAD) threshold (dscale_sad_cross_th) The sum of the sum is less than or equal to SAD (sad[i][j][0]) along the direction "0" (vertical direction), and the cross alpha can be calculated by the above formula E10, so it can be calculated A cross alpha along the direction "9" is produced, otherwise a cross alpha along the direction "0" is generated. The threshold value adjusting means 306 determines a cross factor (dscale_sad_cross_factor). When the cross alpha is greater than or equal to the cross alpha threshold (reg_dscale_cross_alpha_th), the orthogonal angle (cross_angle) is along the direction "9"; otherwise the orthogonal angle (cross_angle) is along the direction "0". When the SAD value along the direction "9" is smaller than the SAD value along the direction "0", the orthogonal absolute difference sum (cross SAD, cross_sad) is the SAD along the direction "0"; otherwise the sum of the orthogonal absolute differences (cross SAD, cross_sad) is the SAD along the direction "9". Thus, the orthogonal angle decision module 310 can determine the cross alpha, the cross angle, and the cross absolute difference (cross SAD) along the orthogonal angular direction.

As shown in Fig. 2A, the window having the direction "0" to the direction "9" (e.g., the 3 x 7 window size) partition is divided into two planes according to the vertical direction "0". The lowest frequency decision module 312 is based on four directions "1", "3", "5", and "7" in the right plane and four directions "2", "4", "6" in the left plane, and "8" determines the lowest frequency.

According to the four directions "1", "3", "5" and "7" of the right plane, the lowest frequency decision module 312 determines the minimum SAD value (min_sad_r) and its corresponding direction indicator (min_angle_r), and determines The second small SAD value (min_sad_r_2nd) and its corresponding direction indicator (min_angle_r_2nd); similarly, according to the four directions "2", "4", "6" and "8" of the left plane, the lowest frequency determining module 312 determines a minimum SAD value (min_sad_l) and its corresponding direction indicator (min_angle_l), and determines a second small SAD value (min_sad_l_2nd) and its corresponding direction indicator (min_angle_l_2nd). In addition, the lowest frequency decision module 312 calculates the SAD sum (sumsad_r) of the input pixels of the four directions "1", "3", "5", and "7" in the right plane, and calculates the four directions of the left plane" The SAD sum (sumsad_l) of the input pixels of 2", "4", "6", and "8".

The lowest frequency decision module 312 compares the smallest SAD value (min_sad_r) of the right plane with the smallest SAD value (min_sad_l) of the left side to select the smallest SAD value and its corresponding direction indicator as the minimum SAD value (min_sad_lr ) and the lowest frequency direction (min_angle). In an embodiment, the lowest frequency decision module 312 further determines the second small SAD value and its corresponding direction indicator by (min_sad_l), (min_sad_l_2nd), (min_sad_r), and (min_sad_r_2nd), that is, when the minimum After the SAD value and the lowest frequency direction are initially determined, the lowest frequency decision module 312 utilizes the lowest frequency direction and its second small angular direction to assist in determining whether the minimum SAD value and the lowest frequency direction are indeed minimum.

Basically, when the lowest frequency determining module 312 selects the minimum SAD value between the right side plane and the left side plane and its corresponding direction indicator as the minimum SAD value and the lowest frequency direction, the following criteria may be As a selection criterion:

Here, the operator "?" in the criterion D3 indicates that when (min_sad_1>min_sad_r) is established, the result is min_sad_1>=min_sad_r*dscale_lr_ratio1, otherwise the result min_sad_r>=min_sad_1*dscale_lr_ratio1. Similarly, the operators "?" in the criteria D4, D5, and other formulas have the same meaning.

When the criteria D1 and D3 are satisfied, the difference between min_sad_1 and min_sad_r is sufficient to determine a smaller SAD value and a minimum angular direction, and in some cases, when the difference is not sufficient to determine, the criteria D2 and D4 can be utilized. In the case of preliminary judgment, in these cases, the criterion D5 is used to determine the smaller SAD value and the minimum angular direction, and the criterion D5 refers to the direction in which the lowest frequency direction approaches the right plane or the direction of the left plane, which corresponds to The sum of the SAD values on the plane side of the lowest frequency direction should be much smaller than the other plane side. When the difference between min_sad_1 and min_sad_r satisfies the criteria D1 and D3 at the same time, or both the criteria D2, D4 and D5 are satisfied, the smaller SAD value between min_sad_1 and min_sad_r and the corresponding angular direction are the minimum SAD values. And the lowest frequency direction; otherwise, the minimum SAD value is set to an absolute maximum and the lowest frequency direction can be ignored.

In an embodiment, the threshold adjustment device 306 dynamically changes the threshold of the criterion D1 to D5 according to the detection result of the global frequency detection unit 302 to determine the minimum SAD value and its corresponding lowest frequency direction. As described above, in the first, second, and third examples of the five examples generated by the global frequency detecting unit 302, the threshold adjusting means 306 adjusts the thresholds of the criteria D1 and D5 to a larger value, and causes the The selection of the lowest frequency decision module 312 is limited. The global gradient direction of the window 6×8 tends to be vertical or horizontal in the first and second examples; in the third example, there are higher frequency components in the vertical direction and the horizontal direction. The present invention effectively adjusts the threshold in these three examples to avoid selecting the wrong lowest frequency direction. The weight estimating unit 314 calculates a weighting factor in the lowest frequency direction, and the weight estimating unit 314 estimates four factors to determine the weighting factor value, which is the first factor, the second factor, the third factor, and The sum of the fourth factors is as follows.

The first factor is determined by the SAD value along the orthogonal angular direction and the lowest frequency direction, the first factor being represented by the following formula E11:

Factor1=(min_sad_cross-min_sad_lr-reg_dscale_sad_margin)/min_sad_lr*reg_dscale_sad_factor1;...(E11)

Here, "reg_dscale_sad_margin" and "reg_dscale_sad_factor1" are determined by the threshold value adjusting means 306. The first factor is proportional to the SAD difference between the orthogonal angular direction and the lowest frequency direction. As the first factor increases, it approaches the lowest frequency direction and the weight factor increases.

The second factor is determined by the sum of the SAD values of the left and right planes, which is represented by the following formula E12:

If(min_angle_lr==1||min_angle_lr==3||mi_angle_lr=5||min_angle_lr==7)//min_angle_lr is the right plane direction

Factor2=(sum_sad_1-sum_sad_r*reg_dscale_sad_factor_2)/(sum_sad_r+1);else//min_angle_lr is the left plane direction

Factor 2=(sum_sad_r-sum_sad_1*reg_dscale_sad_factor_2)/(sum_sad_1+1);...(E12)

When factor 2 is positive, the lowest frequency direction "min_angle_lr" is the direction of the left plane or the direction of the right plane. It is judged by comparing the results of sumsad_l and sumsad_r, otherwise factor2 is negative. The larger the second factor, the direction found is approaching the lowest frequency direction.

The third factor is related to the minimum SAD value along the lowest frequency direction, which is represented by the following formula E13:

Factor3=(min_sad_lr_reg_dscale_sad_max)*reg_dscale_sad_factor3;...(E13)

Here, "reg_dscale_sad_max" and "reg_dscale_sad_factor3" are determined by the threshold value adjusting means 306. As the third factor increases, the weighting factor decreases.

The fourth factor is used to identify whether the smallest SAD value along the lowest frequency direction has a second small SAD value (compared to the minimum SAD value). This fourth factor is represented by the following program fragment:

Here, "reg_dscale_sad_factor4_1" and "reg_dscale_sad_factor4_2" are determined by the threshold value adjusting means 306.

Please refer to FIG. 2F. FIG. 2F determines the gradient level along a lowest frequency direction according to an embodiment of the present invention. A scaler filter can be represented by the formula: f _scaler (phase) = [a0 a1 a2 a3], where a0, a1, a2, and a3 are coefficients of a windowed-sinc filter.

The sharpness filter can be expressed by the formula: f _sharp (gain)=[-12-1]*gain+[010]=[s0 s1 s2], where the gain represents the degree of sharpness, and [-12-1] Is a clear filter.

Therefore, the present invention combines a windowed-sinc filter with a sharp gain, and the updated coefficient can be expressed by the following formula: f _{sharp_scaler} (phase,gain)=convolution(f _scale (phase),f _sharp (gain))= Convolution([a0 a1 a2 a3],[s0,s1,s2])=[a0',a1',a2',a3',a4',a5]; and intp_min=interpolation(f _{sharp_scaler} (phase,gain), M0, m1, m2, m3, m4, m5) = a0'*m0+a1'*m1+a2'*m2+a3'*m3+a4'*m4+a5'*m5.

Here, the sharpness gain is related to the difference between the maximum and minimum values in m1 to m4. It should be noted that the intermediate points m0, m1, m2, m3, m4, and m5 are related to the target output pixel P. _Tar , the intermediate points m0 to m5 can be regarded as virtual pixels, and the line 104 of these intermediate points passes through the target output pixel P _tar , and the line is perpendicular to the lowest frequency direction.

Specifically, in the lowest frequency direction 100 of the 2Fth diagram, the interpolation result "intp_min" is determined by the interpolation calculation of the intermediate points m0 to m5. The line 104 of the intermediate point passes through the target output pixel P _tar and is perpendicular to the lowest frequency direction, that is, the lowest frequency direction. The intermediate points m0 to m5 are determined by interpolation calculation of pixels, and in an embodiment, the intermediate points m0 to m5 can be expressed as the following formula.

M0=P14*d0+P22*(1-d0); d0=distance(m0,P22)/distance(P22,P14);

M1=P23*d1+P31*(1-d1);d1=distance(m1,P31)/distance(P31,P23);

M2=P24*d2+P32*(1-d2);d2=distance(m2,P32)/distance(p32,P24);

M3=P25*d3+P33*(1-d3); d3=distance(m3,P33)/distance(P33,P25);

M4=P34*d4+P42*(1-d4); d4=distance(m4,P42)/distance(P42,P34);

M5=P35*d5+P43*(1-d5); d5=distance(m5, P43)/distance(P43, P35).

Here, d0 to d5 represent the distance ratio, for example, the distance ratio d0 is equal to the ratio of the distance of (m0, P22) to the distance of (P22, P14). Similarly, d1 to d5 can be calculated using the ratio of the distance between the pixel and the intermediate point, and thus the interpolation result "intp_min" of the scaler filter and the sharpness filter can be determined. It should be noted that, in addition to the above formulas in the 2Cth and 2Gth drawings, the above formulas are also applicable to different directions of other patterns.

In order to reduce the effect of the sawtooth edge shape generated by the clear filter in the prior art, the image processing system of the present invention incorporates a one-dimensional clear filter to the scaling filter, wherein the one-dimensional clear filter is used to be orthogonal to the A cross-angle direction and an orthogonal to the minimum-angle direction.

The mixing unit 322 mixes a pattern of orthogonal angular directions and a pattern of a lowest frequency direction according to weighting factor values.

According to the above, the image processing system of the present invention having a zoom/sharp device solves the effect of the sawtooth edge shape of the image and the flicker problem. Moreover, when the input pixels are scaled perpendicular to the low frequency direction, the image processing system with the zoom/sharp device performs the clear processing steps on the input pixels correctly and simultaneously.

In Fig. 2F, the interpolation result along the lowest frequency direction is represented by "intp_min", where intp_min = interpolation (f _{sharp_scaler} (Phase _min , Gain _min ), m0, m1, m2, m3, m4, m5), f _{The sharp_scaler} function outputs a sharpness gain Gain _min corresponding to a coefficient of a specific zoom phase Phase _min and its lowest frequency direction, in other words, a weight sum by the intermediate points m0, m1, m2, m3, m4, and m5, Weighting sum) produces "intp_min" and produces a sharpness gain Gain _min and a zoom phase Phase _min from the target output pixel P _tar to any intermediate point of the input pixels.

Referring to FIG. 2C, FIG. 2C determines the gradient level along an orthogonal angle direction in accordance with an embodiment of the present invention. The result of the interpolation along the orthogonal angle direction is expressed as "intp_cross", where intp_cross=interpolation(f _{sharp_scaler} (PHASE _cross , GAIN _cross ), c0, c1, c2, c3, c4, c5), the output of the f _{sharp_scaler} function corresponds The coefficient of the specific zoom phase PHASE _cross and its sharpness gain GAIN _cross . In other words, "intp_cross" is generated by the weighting sum of the intermediate points c0, c1, c2, c3, c4, and c5, and the clearness from the target output pixel P _tar to any intermediate point of the input pixels is generated. Sharpness gain GAIN _cross and zoom phase Phase _min .

The noise filter 320 of the image mixing device 308 is configured to detect whether the lowest frequency direction is correct. When the lowest frequency direction is incorrect, the noise filter 320 can correct the weight factor value. Specifically, the weight factor value is subtracted from the comb factor to reduce the weight factor value. Referring to FIG. 4, FIG. 4 is related to the target pixel P _tar according to the embodiment of the present invention. A pixel diagram in which a rectangular area formed by the four pixels surrounds the target pixel P _tar .

When the comb factor increases, the luminance level of the pixel P _tar may deviate from P23, P24, P33 and P34. This case indicates that the lowest frequency direction may be incorrect, and the interpolation result approaches the orthogonal angle direction. The interpolation result of the pixel P _tar is mixed with values along the orthogonal angular direction and the lowest frequency direction. Pixels P23, P24, P33 and P34 are used to generate pixels Pa(up_y), Pb(dn_y), Pc(1f_y), Pd(rt_y), where Pa and Pb are detected in the vertical direction, and Pc and Pd are in the horizontal direction. Detection.

Please refer to the 2F map, where p[2], p[3] correspond to m2 and m3, and the coordinates of P _tar (tmp_y) are expressed as (row_fract, col_fract).

Here, "reg_dscale_comb_margin1" and "reg_dscale_comb_filter_mth" are determined by the threshold value adjusting means 306.

Please refer to FIG. 3 and FIG. 5. FIG. 5 is a flowchart of a method for executing the image processing system 300 according to an embodiment of the present invention. The image processing system 300 includes a global frequency detecting unit 302, a gradient calculating sum unit 304, a threshold adjusting device 306, and an image mixing device 308. The global frequency detecting unit 302 is coupled to the threshold adjusting device 306. The summing unit 304 is coupled to the image mixing device 308 , and the threshold adjusting device 306 is coupled to the image mixing device 308 . The image mixing device 308 further includes an orthogonal angle determining module 310, a lowest frequency determining module 312, a weight estimating unit 314, a first zooming/clearing module 316, a second zooming/clearing module 318, and a noise filter 320. And a mixing unit 322. The global frequency detecting unit 302 further includes a vertical frequency detecting unit 302a and a horizontal frequency detecting unit 302b. The method for executing the image processing system 300 of the present invention includes the following steps: In step S500, the global frequency detecting unit 302 and the gradient calculating total unit 304 respectively receive a plurality of input pixels.

In step S502, the global frequency detecting unit 302 is used to calculate the vertical frequency levels of the input pixels along the vertical direction, and the horizontal frequency levels of the input pixels are calculated along the horizontal direction. In step S502, The vertical frequency detecting unit 302a further calculates a vertical frequency level along the vertical direction, and the horizontal frequency detecting unit 302b further calculates a horizontal frequency level along the horizontal direction.

In step S504, the threshold adjustment device 306 adjusts the first threshold and the second threshold according to the relationship between the vertical frequency level and the horizontal frequency level.

In step S506, the gradient calculation summation unit 304 is used to calculate a set of gradient luminance levels along a set of directions and according to a part of the input pixels.

In step S508, the orthogonal angle determining module 310 compares the set of gradient brightness levels to determine an orthogonal direction along a horizontal direction or a vertical direction to form a first pattern by using the first threshold.

In step S510, the lowest frequency decision module 312 compares the set of gradient brightness levels to determine a minimum direction along one of the set of directions to form a second pattern using the second threshold.

In step S512, the image mixing device 308 mixes the first pattern and the second pattern from the global frequency detecting unit 302 according to the weight factor value, wherein the first pattern is related to the first threshold, and the second pattern is related. At the second threshold.

In the step S512, the orthogonal angle determining module 310 determines the first pattern according to the first threshold and one of the vertical direction or the horizontal direction. The orthogonal angle determining module 310 determines the second pattern according to the second threshold and in a direction other than the vertical direction or the horizontal direction.

While the invention has been described above in terms of the preferred embodiments, the invention is not intended to limit the invention, and the invention may be practiced without departing from the spirit and scope of the invention. The scope of protection of the present invention is therefore defined by the scope of the appended claims.

100. . . Lowest frequency direction

102. . . Orthogonal angle direction

104. . . Cable

300. . . Image processing system

302. . . Global frequency detection unit

302a. . . Vertical frequency detection unit

302b. . . Horizontal frequency detection unit

304. . . Gradient calculation sum unit

306. . . Threshold adjustment device

308. . . Image mixing device

310. . . Orthogonal angle determination module

312. . . Minimum frequency decision module

314. . . Weight evaluation unit

316. . . First zoom/clear module

318. . . Second zoom/clear module

320. . . Noise filter

322. . . Mixing unit

1 is a schematic diagram of a plurality of input pixels of an image according to an embodiment of the present invention, wherein an image scaling system is used to perform a direction scaling mechanism on the pixel along a lowest frequency direction and an orthogonal angle direction;

2A is a schematic diagram of a plurality of gradient directions related to an input pixel according to an embodiment of the present invention, wherein when the image processing system performs the scaling image step, the lowest frequency direction and the orthogonal angle direction may be determined;

2B-2G is a schematic diagram for calculating a sum of absolute values of luminance differences according to an embodiment of the present invention, wherein the luminance difference refers to a luminance difference value between different pairs of input pixels along different gradient directions;

Figure 3 is a block diagram of an image processing system in accordance with an embodiment of the present invention;

4 is a schematic diagram of four pixels related to a target pixel according to an embodiment of the present invention, wherein a rectangular area formed by the four pixels surrounds the target pixel;

Figure 5 is a flow chart of an image processing method in accordance with an embodiment of the present invention.

300. . . Image processing system

302. . . Global frequency detection unit

302a. . . Vertical frequency detection unit

302b. . . Horizontal frequency detection unit

304. . . Gradient calculation sum unit

306. . . Threshold adjustment device

308. . . Image mixing device

310. . . Orthogonal angle determination module

312. . . Minimum frequency decision module

314. . . Weight evaluation unit

316. . . First zoom/clear module

318. . . Second zoom/clear module

320. . . Noise filter

322. . . Mixing unit

Claims (25)

An image processing system includes: a global frequency detecting unit configured to detect a frequency of a plurality of input pixels to calculate a vertical frequency level of the input pixels along a vertical direction, and calculate along a horizontal direction a horizontal frequency level of the input pixels; a gradient calculation summation unit, respectively, along a plurality of directions, and according to a part of the input pixels, used to calculate a set of gradient brightness levels; a threshold adjustment device, coupled Connected to the global frequency detecting unit, adjusting a first threshold and a second threshold according to the relationship between the vertical frequency level and the horizontal frequency level; and an image mixing device coupled to the a gradient calculation summation unit for determining an orthogonal angular direction by the first threshold value from the threshold value adjusting device, and determining a minimum by using the second threshold value from the threshold value adjusting device a frequency direction, wherein the image mixing device forms a first pattern associated with the orthogonal angular direction and forms a second pattern associated with the lowest frequency direction, and the Like mixing apparatus according to a weight coefficient value mixing the first pattern and the second pattern. The image processing system of claim 1, wherein the global frequency detecting unit further comprises: a vertical frequency detecting unit along the vertical direction to calculate the vertical frequency level of the input pixels; And a horizontal frequency detecting unit along the horizontal direction to calculate the horizontal frequency level of the input pixels. The image processing system of claim 2, wherein the lowest frequency direction is a direction of a minimum brightness change of the input pixels. The image processing system of claim 2, wherein the vertical frequency detecting unit is a high pass filter, and the horizontal frequency detecting unit is a high pass filter. The image processing system of claim 1, wherein the image mixing device further comprises: an orthogonal angle determining module, according to the first threshold value and along the vertical direction or one of the horizontal directions Determining the first pattern; and a lowest frequency determining module determining the second pattern according to the second threshold and in a direction other than the vertical direction or the horizontal direction. The image processing system of claim 5, wherein the image mixing device further comprises a weight estimating unit for calculating the weighting factor value in the lowest frequency direction. The image processing system of claim 6, wherein the weight factor of the weight estimating unit is related to a first factor, the first factor being along the orthogonal angle direction and the lowest frequency direction, respectively. Calculate the sum of the absolute differences of the input pixels. The image processing system of claim 6, wherein the weighting factor of the weight estimating unit is related to a second factor, and the second factor is determined by a sum of all absolute differences of the input pixels. . The image processing system of claim 6, wherein the weight factor of the weight estimating unit is related to a third factor, and the third factor is determined by a sum of minimum absolute differences of the input pixels. . The image processing system of claim 6, wherein the weight factor of the weight estimating unit is related to a fourth factor, and the fourth factor is used to identify a minimum absolute difference along the lowest frequency direction. Whether the sum has a second smallest absolute difference sum. The image processing system of claim 1, wherein the image mixing device further comprises: a first zoom/clear module for performing direction zooming and clearing steps according to the vertical direction and one of the horizontal directions And a second zoom/clear module for performing direction scaling and sharpening steps according to one of the lowest frequency directions except the vertical direction and the horizontal direction. The image processing system of claim 1, wherein the image mixing device further comprises a noise filter. The image processing system of claim 12, wherein the noise filter further comprises a comb factor, the comb factor is subtracted from the weight factor value to reduce the weight factor value. And when the comb factor increases, the interpolation result of the input pixels approaches the orthogonal angular direction. An image processing method includes the following steps: (a) receiving a plurality of input pixels; (b) calculating a vertical frequency level of the input pixels along a vertical direction by using a global frequency detecting unit, and calculating in a horizontal direction The horizontal frequency level of the input pixels; (c) adjusting the first threshold and the second threshold according to the relationship between the vertical frequency level and the horizontal frequency level by using a threshold adjustment device; Calculating a summation unit by using a gradient, respectively, along a set of directions and according to a part of the input pixels, to calculate a set of gradient brightness levels; (e) comparing the set of gradient brightness levels to determine an orthogonal direction a horizontal direction or a vertical direction to form a first pattern by using the first threshold; (f) comparing the set of gradient brightness levels to determine a lowest frequency direction along one of the set of directions to utilize the a second threshold forming a second pattern; and (g) mixing the first pattern and the second pattern from the global frequency detecting unit according to a weight factor value, wherein the first pattern is related to the first threshold, the first Two patterns On the second threshold. The image processing method of claim 14, wherein after the step (b), the method further comprises: using a vertical frequency detecting unit along the vertical direction to calculate the vertical of the input pixels Frequency level; and using a horizontal frequency detecting unit along the horizontal direction to calculate the horizontal frequency level of the input pixels. The image processing method of claim 15, wherein the lowest frequency direction is a direction of a minimum brightness change of the input pixels. The image processing method of claim 14, wherein in the step (e), the method further comprises the following step (e1): determining an module by using an orthogonal angle, according to the first threshold and along the The vertical direction or one of the horizontal directions determines the first pattern. The image processing method of claim 17, wherein in the step (e), the method further comprises the following step (e2): using a lowest frequency determining module according to the second threshold and along the vertical The second pattern is determined by a direction other than the direction or the horizontal direction. The image processing method of claim 14, wherein the weight factor of the weight estimating unit is related to a first factor, the first factor being along the orthogonal angle direction and the lowest frequency direction, respectively. Calculate the sum of the absolute differences of the input pixels. The image processing method of claim 14, wherein the weight factor of the weight estimating unit is related to a second factor, and the second factor is determined by a sum of all absolute differences of the input pixels. . The image processing method of claim 14, wherein the weight factor of the weight estimating unit is related to a third factor, and the third factor is determined by a sum of minimum absolute differences of the input pixels. . The image processing method of claim 14, wherein the weight factor of the weight estimating unit is related to a fourth factor, and the fourth factor is used to identify a minimum absolute difference along the lowest frequency direction. Whether the sum has a second smallest absolute difference sum. The image processing method of claim 14, wherein after the step (f), the method further comprises the following steps: using a first zoom/clear module, according to the vertical direction and one of the horizontal directions, Performing direction scaling and sharpening steps; and utilizing a second zoom/clear module to perform direction scaling and sharpening steps in accordance with one of the lowest frequency directions other than the vertical direction and the horizontal direction. The image processing method of claim 14, further comprising a comb factor, wherein the weight factor value is subtracted from the comb factor to reduce the weight factor value. The image processing method of claim 24, wherein when the comb factor is increased, the interpolation result of the input pixels approaches the orthogonal angle direction.