KR20080066372A - De-interlacing device using modified sobel operation and method of the same - Google Patents

Abstract

An apparatus and a method for de-interlacing, and a recording medium are provided to improve the quality of an image restored through accurate prediction of edge direction and restore the image in real-time under less influence of noise. A first edge direction detector(110) detects a temporary edge direction for a pixel to be interpolated, based on P and Q defined by an E-ELA(Efficient-Edge based Line Average) method. A Sobel operator(120) performs a Sobel operation on the pixel in order to compute sizes and phases of gradient vectors. A second edge direction detector(130) selects a predetermined number of gradient vectors, which are previously set in a size order among the computed gradient vectors. The second edge direction detector detects the phase of a gradient vector with a large phase, among the selected gradient vectors, as an edge direction of the pixel to be interpolated. An interpolating unit(140) interpolates the pixel to be interpolated by pixel values of pixels positioned in a direction of the detected edge direction.

Description

De-interlacing device using modified sobel operation and method of the same}

1 is a view showing the configuration of a preferred embodiment of a deinterlacing apparatus using a modified Sobel operator according to the present invention,

FIG. 2 is a diagram illustrating a pixel column in which a 3 × 3 window is formed around a pixel to be interpolated; FIG.

3A shows a 3 × 3 size Sobel mask,

FIG. 3b illustrates a 3 × 9 modified Sobel mask of the second and fourth columns blank in the vertical direction, FIG.

4 is a diagram illustrating edge angles according to relative positions of pixels;

5 is a flowchart illustrating a preferred embodiment of the deinterlacing method using the modified Sobel operator according to the present invention;

6 is a graph showing the relative execution time of the deinterlacing method and the DOI method according to the present invention, and

7A to 7H illustrate an original finger image, an enlarged image of an original finger image, a finger image interpolated by the LD method, a finger image interpolated by the LA method, a finger image interpolated by the ELA method, and an E-ELA method. The interpolated finger image, the interpolated finger image by the DOI method, and the interpolated finger image according to the present invention.

The present invention relates to a deinterlacing apparatus and method, and more particularly, to a deinterlacing apparatus and method for converting an interlaced scan video signal to a progressive scan video signal.

Conventional analog TVs use a vertical sampling method, and digital TVs take samples at regular intervals in the horizontal and vertical directions and process digital signals created by quantizing these sample values. The analog TV signal uses an interlace signal in which even and odd fields are crossed at a time difference of 1/60 second to form one frame. The analog TV signal is sampled on the vertical axis by these scan lines and also on the time axis by field alteration. Therefore, if the image of analog TV is used as it is in a digital TV or a computer, not only flickering of edges and lines but also stains between lines appear. To solve this problem, a deinterlacing method has been proposed.

Various deinterlacing methods have been proposed so far, and these methods can be broadly divided into a spatial deinterlacing method using spatial correlation in one frame and a temporal deinterlacing method using temporal correlation between several frames. The most widely used spatial deinterlacing method is an edge based line average (ELA) algorithm. The classical ELA algorithm detects the direction of correlation between lines in an image and determines the direction with the highest correlation, and interpolates by taking the average of the pixels of two lines based on the direction. However, it is sensitive to small changes in pixel values and has the disadvantage of using wrong edge information for diagonals greater than or equal to 45 degrees. In addition, it is a big problem that the edge part cannot find and apply a drastic change despite the visual characteristics that are clearly visible to the eyes. An algorithm to compensate for this problem is the E-ELA (Efficient ELA) method. The E-ELA method classifies the spatial form of the image into three types in order to find more accurate orientation than the ELA method. That is, spatial directionality is classified into left, right, or other cases, and the ELA is modified and applied in an appropriate manner in each classified category. However, like the ELA method, the E-ELA method is relatively simple and shows a good subjective efficiency when the diagonal direction is well sensed.However, when the E-ELA method reacts to minute changes due to high frequency components or noise, it is worse than the vertical line average. It may be indicated. Furthermore, there is a problem in implementing optimal performance in view of the estimated edge direction accuracy, and there is a problem in that the restored image quality is degraded as interpolation is performed by an incorrect edge direction.

The Direction Oriented Interpolation (DOI) method is used to find and interpolate the edge direction in more detail than the E-ELA method. The DOI method can more accurately find the direction of the edge through 3x2 block matching centering on the position of the pixel to be interpolated to perform more accurate interpolation. The direction vector used in the DOI method is less sensitive to the influence of noise than the method of detecting the correlation of the directionality used in the ELA method, and thus shows a more accurate result when calculating the correlation with a farther position. . However, the DOI method has a considerable complexity since the direction vector concept is used to consider various directions, which makes it difficult to restore a real-time image.

The technical problem to be achieved by the present invention is to improve the image quality of the reconstructed image through the prediction of the correct edge direction when performing the interpolation in the field, the modified Sobel that can reconstruct the image in real time while being less affected by noise An apparatus and method for deinterlacing using an operator are provided.

Another technical object of the present invention is to improve the image quality of a reconstructed image through accurate edge prediction when performing interpolation in a field, and to modify the image in real time while being less affected by noise. The present invention provides a computer-readable recording medium for executing a deinterlacing method using a Sobel operator on a computer.

In order to achieve the above technical problem, the deinterlacing apparatus according to the present invention comprises: a first edge direction detection unit for detecting a temporary edge direction for a pixel to be interpolated based on P and Q defined in an E-ELA method; A Sobel calculation unit for performing a Sobel operation for calculating the magnitude and phase of the gradient vector for the pixel corresponding to the temporary edge direction; A second edge direction that selects a predetermined number of gradient vectors in order of magnitude from the gradient vectors calculated by the Sobel calculation unit, and detects a phase of the gradient vector having a large phase among the selected gradient vectors in an edge direction of the pixel to be interpolated; Detection unit; And an interpolation unit which interpolates the pixel to be interpolated by the pixel value of the pixel located in the detected edge direction.

According to another aspect of the present invention, a deinterlacing method includes: (a) detecting a temporary edge direction for a pixel to be interpolated based on P and Q defined in an E-ELA method; (b) performing a Sobel operation for calculating the magnitude and phase of the gradient vector for the pixel corresponding to the temporary edge direction; (c) selecting a predetermined number of gradient vectors in order of magnitude from the gradient vectors calculated by the Sobel operation, and detecting a phase of the gradient vector having a large phase among the selected gradient vectors in the edge direction of the pixel to be interpolated; ; And (d) interpolating the pixel to be interpolated by the pixel value of the pixel located in the detected edge direction.

As a result, the image quality of the reconstructed image can be improved by accurately predicting the edge direction when performing interpolation in the field, and the image can be interpolated in real time while being less affected by noise.

Hereinafter, exemplary embodiments of a deinterlacing apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing the configuration of a preferred embodiment of a deinterlacing apparatus using a modified Sobel operator according to the present invention.

Referring to FIG. 1, the deinterlacing apparatus 100 using the modified Sobel operator 100 according to the present invention includes a first edge direction detector 110, a Sobel operator 120, a second edge direction detector 130, and an interpolator 140. ).

The first edge direction detection unit 110 detects a temporary edge direction for the pixel to be interpolated based on P and Q defined in the E-ELA method. FIG. 2 shows a 3 × 3 window centered on pixel D to be interpolated. 2, P and Q for the pixel D to be interpolated are defined by the following equation.

The first edge direction detection unit 110 compares P and Q obtained by Equation 1 to find the edge direction. If P is larger than Q, it means that the edge flows in the Q direction. Accordingly, the first edge direction detection unit 110 determines the linear direction connecting the pixel B and the pixel E as the temporary edge direction. On the other hand, if P is smaller than Q, the edge flows in the P direction, and thus, the first edge direction detection unit 110 determines the linear direction connecting the pixel B and the pixel G as the temporary edge direction. On the other hand, if P and Q are the same, it is highly likely that the direction of the edge is not clear or there is no edge, so the first edge direction detection unit 110 determines the linear direction connecting the pixel B and the pixel F as the temporary edge direction. .

The Sobel operation unit 110 performs Sobel operation on a pixel corresponding to the temporary edge direction determined by the first edge direction detection unit 110. In Fig. 3a, a 3 × 3 Sobel mask is shown. Sobel operation is the most representative edge detection method among various edge detection algorithms and is an edge detection algorithm using the second-order change of brightness value between pixels. The gradient vector ▽ f for the (x, y) position in the Sobel operation is defined by the following equation.

In addition, the magnitude and phase of the gradient vector are defined by the following equation.

The magnitude of the gradient vector | ▽ f | means the maximum rate of change of f (x, y) per unit distance according to the direction of the gradient vector at the position (x, y), and the phase ∠ (▽ f) is (x, y). ) Means the direction of the gradient vector at the position.

Meanwhile, the magnitude and phase of the gradient vector are obtained by the following equation using the Sobel mask shown in FIG. 3A.

However, in an interlaced scan image, an empty column exists between pixel columns, and thus, the Sobel mask as described above cannot be directly applied to the Sobel operation. Therefore, it is necessary to change the Sobel mask in order to apply the Sobel operation to the interlaced image. FIG. 3B shows a 5 × 3 modified Sobel mask with the second and fourth columns blank in the vertical direction. Referring to FIG. 3B, the second and fourth columns of the modified Sobel mask are filled with zeros. Furthermore, in order to increase the accuracy of Gx and Gy obtained by the modified Sobel mask, it is necessary to multiply the conversion coefficients α _x and α _y respectively. These conversion coefficients are determined experimentally. For example, α _x and α _y may be set to 0.8 and 3.0, respectively. In the Sobel mask shown in FIG. 3B, Z ₅ is a pixel position where a gradient vector is calculated.

Accordingly, the Sobel operator 120 performs Sobel operation on the pixel corresponding to the temporary edge direction determined by the first edge direction detector 110 in the modified Sobel mask illustrated in FIG. 3B. If the temporary edge direction is the P direction, the Sobel operation unit 120 performs the Sobel operation on the pixel A, the pixel B, the pixel D, the pixel F, and the pixel G. The Sobel operation unit 120 performs the Sobel operation by applying the modified Sobel mask shown in FIG. 3B to the pixels A, B, F, and G, and applies the Sobel mask shown in FIG. 3A to the pixel D. Sobel operation is performed. In this case, since the value of the pixel X is empty, the Sobel operation cannot be performed. Therefore, the Sobel operator 120 first sets the average of the pixels H and I to the value of the pixel X, and then performs the Sobel operation. The magnitude and phase of five gradient vectors are obtained through the Sobel operation on the pixels corresponding to the temporary edge directions as described above. Meanwhile, when the temporary edge direction is the Q direction, the Sobel calculation unit 120 performs the operation described above with respect to the pixel B, the pixel C, the pixel D, the pixel E, and the pixel F. FIG.

The second edge direction detection unit 130 selects two gradient vectors having a large size among the five gradient vectors calculated by the Sobel calculation unit 120. Since the larger the magnitude of the gradient vector is, the stronger the edge is. Therefore, it is reasonable to determine the edge direction based on the size of the gradient vector. Next, the second edge direction detection unit 130 determines the phase of the gradient vector having a large phase among the two selected gradient vectors in the edge direction. At this time, the second edge direction detection unit 130 determines that the edge direction detection has failed when the phases of the two selected gradient vectors are smaller than the preset threshold. 4 shows the edge angle according to the relative position of the pixel. Referring to FIG. 4, the edge angles according to the relative positions from the pixel D to be interpolated to each pixel are 90.00 °, 45.00 °, 26.57 °, 18.43 °, 14.04 °, 11.31 ° and 9.46 °, respectively. In the present invention, when the pixel for calculating the edge direction is located at a position exceeding 4 pixels from the pixel to be interpolated (that is, when the edge angle is smaller than 14.04 °), it is determined that the edge direction is incorrectly obtained. Therefore, the threshold value of the edge angle is set to 14.04 °. The threshold of this edge angle is determined experimentally according to the characteristics of the image. As described above, if the edge angle is determined, edges of all angles from ± 90 ° to ± 14.04 ° can be discriminated, enabling more precise edge detection.

The interpolator 140 interpolates the pixels to be interpolated based on the edge direction detected by the second edge direction detector 130. Therefore, when the edge direction is determined to be 45 °, the interpolator 140 interpolates the pixel D with the average value of the pixel C and the pixel E. FIG. In addition, when the edge direction is determined to be −45, the interpolator 140 interpolates the pixel D with an average value of the pixel A and the pixel G. In addition, when information indicating that the edge direction detection has failed is input from the second edge direction detection unit 130, the interpolation unit 140 interpolates the pixel D with an average value of the pixels B and F.

On the other hand, when the edge is determined using the Sobel operation as described above, the edge direction can be detected quite accurately, but there is a disadvantage in that the complexity of the calculation is high. In order to reduce the complexity of the calculation, it is preferable to selectively apply the Sobel operation by determining whether the pixel to be interpolated is a flat area or an edge area. To this end, in the present invention, an evaluation value defined by the following equation is larger than a reference value, which is an absolute difference value between an upper pixel (ie, pixel B) and a lower pixel (ie, pixel F) of a pixel to be interpolated (ie, pixel D). In this case the Sobel operation is applied.

Here, T2B is an evaluation value and Pel (i, j) is a pixel value of a pixel to be interpolated.

Therefore, the evaluation value T2B (D) for the pixel D in the pixel column shown in FIG. 2 is | Pel (A) -Pel (E) | + | Pel (B) -Pel (F) | + | Pel (C) Obtained as -Pel (G) | This evaluation process may be performed by an edge evaluation unit (not shown) provided separately or by the first edge direction detection unit 110. If the evaluation value is smaller than the reference value, the edge evaluation unit or the first edge direction detection unit 110 outputs a signal indicating the flat area to the interpolation unit 140, where the interpolation unit 140 is a pixel to be interpolated (that is, The pixel to be interpolated is interpolated by the average value of the upper pixel (i.e. pixel B) and the lower pixel (i.e. pixel F) of pixel D).

5 is a flowchart illustrating a preferred embodiment of the deinterlacing method using the modified Sobel operator according to the present invention.

Referring to FIG. 5, the edge evaluation unit calculates an evaluation value for a pixel to be interpolated by Equation 5 (S500). Next, the edge evaluation unit compares the calculated evaluation value with a reference value which is an absolute difference value between the upper pixel and the lower pixel of the pixel to be interpolated and determines whether the Sobel operation is performed (S510). If the evaluation value is larger than the reference value, the first edge direction detection unit 110 detects the temporary edge direction for the pixel to be interpolated based on P and Q defined in the E-ELA method (S520). Next, the Sobel calculation unit 120 performs a Sobel operation on the pixel corresponding to the temporary edge direction determined by the first edge direction detection unit 110 (S530). In this case, the Sobel operation unit 120 performs a Sobel operation by applying an existing Sobel mask to the pixels to be interpolated, and performs the Sobel operation by applying a modified Sobel mask to pixels other than the pixels to be interpolated.

The second edge direction detection unit 130 selects two gradient vectors having a large size among the five gradient vectors calculated by the Sobel calculation unit 120 (S540). Next, the second edge direction detection unit 130 checks whether the phase of the two selected gradient vectors is greater than a preset threshold value (S550). When it is determined that the phases of the two selected gradient vectors are greater than the preset threshold value, the second edge direction detection unit 130 detects the phase of the gradient vector having the larger phase among the two selected gradient vectors in the edge direction (S560). . The interpolator 140 interpolates the pixels to be interpolated based on the edge direction detected by the second edge direction detector 130 (S570). On the other hand, if it is determined in step S510 that the evaluation value is smaller than the reference value or the phases of the two gradient vectors selected in step S550 are smaller than the preset threshold value, the interpolator 140 is positioned on the upper and lower parts of the pixel to be interpolated. A pixel to be interpolated is interpolated by an average value of the located pixels (S580).

Table 1 is a table showing the objective image quality of the image obtained by performing the deinterlacing method according to the present invention and the conventional deinterlacing method for various still images by PSNR (dB). The size of the still image used in the experiment is 512 x 512 (pixels).

LD LA ELA E-ELA DOI The present invention airplane 29.32 34.25 33.04 33.33 34.01 34.49 baboon 21.82 23.94 23.35 23.57 23.86 23.98 barbara 27.25 32.14 25.21 30.67 29.65 32.34 boat 30.17 35.38 32.47 33.45 34.76 35.57 finger 25.96 31.69 28.82 29.34 29.23 31.75 girl 33.01 38.29 37.18 37.47 37.79 38.61 goldhill 30.23 33.63 32.27 32.63 35.53 33.75 lena 32.77 37.65 35.96 36.88 37.66 38.16 man 26.38 29.31 28.60 29.15 29.41 29.62 peppers 30.37 33.78 34.13 34.21 33.99 34.25 tiffany 32.94 35.65 34.86 35.40 35.84 35.88 toys 29.19 33.31 32.55 32.97 33.25 33.39 zelda 35.87 41.30 39.38 39.78 40.69 41.40 Average 29.64 33.87 32.14 32.99 33.36 34.09

Referring to Table 1, it can be seen that the deinterlacing method according to the present invention has an average improvement of 0.73 dB in the objective image quality of the interpolated image than the DOI method. Meanwhile, FIG. 6 is a graph showing the relative execution times of the deinterlacing method and the DOI method according to the present invention. The relative execution time shown in FIG. 6 is calculated based on 100 average times. Referring to FIG. 6, it can be seen that the deinterlacing method according to the present invention is improved by 36.34% in complexity than the DOI method. 7A to 7H also show an original finger image, an enlarged image of an original finger image, a finger image interpolated by the LD method, a finger image interpolated by the LA method, a finger image interpolated by the ELA method, and E-. A finger image interpolated by the ELA method, a finger image interpolated by the DOI method, and a finger image interpolated by the present invention are shown. 7A to 7H, it can be seen that the subjective quality of the interpolated image is also excellent in the deinterlacing method according to the present invention.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific preferred embodiments described above, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

According to the deinterlacing apparatus and method using the modified Sobel operator according to the present invention, the edge direction is detected by selectively applying the Sobel operator after detecting the approximate edge orientation. It is possible to improve the quality of the interpolated image, and real-time image interpolation is possible while minimizing the influence of noise. In particular, by determining whether the pixel to be interpolated is a flat area or an edge area and selectively applying the Sobel operation, it is possible to improve the objective and subjective picture quality of the interpolated image while reducing the complexity of the calculation.

Claims (11)

A first edge direction detector for detecting a temporary edge direction for a pixel to be interpolated based on P and Q defined in the E-ELA method; A Sobel calculation unit for performing a Sobel operation for calculating the magnitude and phase of the gradient vector for the pixel corresponding to the temporary edge direction; A second edge direction that selects a predetermined number of gradient vectors in order of magnitude from the gradient vectors calculated by the Sobel calculation unit, and detects a phase of the gradient vector having a large phase among the selected gradient vectors in an edge direction of the pixel to be interpolated; Detection unit; And And an interpolation unit which interpolates the pixel to be interpolated by the pixel value of the pixel located in the detected edge direction. The method of claim 1, And an edge evaluator which compares the evaluation value calculated by the following equation with a reference value which is an absolute difference value between the upper pixel and the lower pixel of the pixel to be interpolated and determines whether the Sobel operation is performed. The interpolation unit, when the evaluation value is smaller than the reference value, interpolating the pixel to be interpolated by an average value of upper and lower pixels of the pixel to be interpolated;

, Here, T2B is an evaluation value and Pel (i, j) is a pixel value of a pixel to be interpolated. The method according to claim 1 or 2, The Sobel calculation unit performs a Sobel operation by applying a 3 × 3 size Sobel mask to the pixels to be interpolated, and a 5 × 3 sized Sobel mask having a value of even columns equal to 0 for pixels other than the pixels to be interpolated. De-interlacing apparatus characterized in that to perform the Sobel operation. The method according to claim 1 or 2, And when the second edge direction detector determines that the phase of the selected gradient vector is greater than a preset threshold, determining the phase of the gradient vector having a larger phase among the selected gradient vectors in an edge direction. The method of claim 4, wherein And when the phase of the selected gradient vector is determined to be smaller than a preset threshold, the interpolation unit interpolates the pixel to be interpolated by an average value of upper and lower pixels of the pixel to be interpolated. (a) detecting a temporary edge direction for the pixel to be interpolated based on P and Q defined in the E-ELA method; (b) performing a Sobel operation for calculating the magnitude and phase of the gradient vector for the pixel corresponding to the temporary edge direction; (c) selecting a predetermined number of gradient vectors in order of magnitude from the gradient vectors calculated by the Sobel operation, and detecting a phase of the gradient vector having a large phase among the selected gradient vectors in the edge direction of the pixel to be interpolated; ; And (d) interpolating the pixels to be interpolated by pixel values of pixels located in the detected edge direction. The method of claim 6, (e) determining whether the Sobel operation is performed by comparing the evaluation value calculated by the following equation with a reference value which is an absolute difference value between the upper and lower pixels of the pixel to be interpolated. In the step (d), if the evaluation value is less than the reference value, the de-interlacing method characterized in that for interpolating the pixel to be interpolated by the average value of the upper and lower pixels of the pixel to be interpolated:

, Here, T2B is an evaluation value and Pel (i, j) is a pixel value of a pixel to be interpolated. The method according to claim 6 or 7, In the step (b), the Sobel operation is performed by applying a 3 × 3 size Sobel mask to the pixels to be interpolated, and a 5 × 3 size deformation with an even number of 0 for pixels other than the pixels to be interpolated. The de-interlacing method characterized in that performing the Sobel operation by applying the Sobel mask. The method according to claim 6 or 7, In the step (c), if it is determined that the phase of the selected gradient vector is larger than a preset threshold, the phase of the gradient vector having a larger phase among the selected gradient vectors is determined in an edge direction. The method of claim 9, In step (d), if it is determined that the phase of the selected gradient vector is smaller than a preset threshold, the pixel to be interpolated is interpolated by an average value of upper and lower pixels of the pixel to be interpolated. De-interlacing method. A computer-readable recording medium having recorded thereon a program for executing the deinterlacing method according to claim 6 or 7.

Images