KR20010068516A - Deinterlacing method and apparatus - Google Patents

Abstract

PURPOSE: A deinterlacing method and a device thereof are provided to improve image quality after interpolation by performing proper interpolation according to a moving degree and an edge direction of a field to interpolate presently. CONSTITUTION: A field recording unit(100) records four field data of present field data continuous in time, former two field data and latter field data. A moving determination unit(200) produces a BD(Brightness Difference) and a BPPD(Brightness Profile Pattern Difference) inter each field data of the field recording unit(100), and performs mapping of the BD and the BPPD with an established value if the BD and the BPPD is over a threshold value, and produces a moving degree value with reference to the BD and the BPPD. A median filter unit(600) removes a noise component included in the moving degree value to group a region in which the moving degree value exists. A moving extension unit(700) receives the output value of the median filter unit(600) and extends the moving degree value to neighboring pixels. A spacial interpolation unit(300) receives a pixel value of field data of a region to perform interpolation presently and a neighboring pixel value within field from the field recording unit(100), and produces an edge direction included in pixel values around the pixel to interpolate and produces an interpolation value within field according to the direction. A timing interpolation unit(400) averages the pixel value of the former field and the pixel value of the latter field to produce a field average value. A soft switch unit(500) mixes the interpolation value within field considering the edge direction of the spacial interpolation unit(300) with the field average value of the timing interpolation unit(400) in reference to the BD and the BPPD of the moving extension unit(700). A vertical line transform unit(800) receives the present field data of the field recording unit(100) and transforms a vertical line number to suit for display in reference to the interpolation line data of the soft switch unit(500).

Description

Deinterlacing method and apparatus {DEINTERLACING METHOD AND APPARATUS}

The present invention relates to a motion adaptive interpolation technique, and more particularly, to a deinterlacing method and apparatus.

The deinterlacing method is a technique for converting interlaced scanning image data into progressive scanning image data in an image processing method.

In the progressive scan method, the image data of one frame is implemented as two fields of an odd field and an even field, as shown in FIG. 1.

However, depending on the display device, the screen is implemented by processing the image data of the forward scanning method used in a computer monitor without processing the image data of the conventional scanning method.

In this case, in order for the progressive scanning video signal to be normally processed in the display device processing the progressive scanning video data, a separate system for converting the traveling scanning video signal into the progressive scanning video signal is provided in the display device. It must be installed.

A method of converting an image signal of a progressive scan method into an image signal of a forward scanning method may be implemented in various ways as shown in FIGS. 2A to 2C.

First, a line repetition method as shown in FIG. 2A implements a frame by simply repeating line information of a current field.

In addition, inter-field interpolation without motion-compensation as shown in FIG. 2B implements one frame by simply sandwiching the immediately preceding line between the lines of the current field.

Finally, intra-field interpolation, as shown in FIG. 2C, implements a new field by inserting the data of the two lines in the field between two lines in one field.

However, although the line repetition method can be implemented with simple hardware, the image quality is poor after interpolation.

In addition, inter-field interpolation without motion compensation can be implemented by simple hardware. However, when interpolating an image with motion, an error occurs or the screen is deteriorated, thereby degrading image quality.

Intra-field interpolation has a lower image quality than the line repetition method and lower error than inter-field interpolation without motion compensation. However, when interpolating a still image, the image deteriorates and the image quality deteriorates.

That is, the above interpolation methods of FIGS. 2A to 2C all have disadvantages of degrading image quality after interpolation.

Accordingly, a motion compensation interpolation method for interpolating a current screen using field data of a previous screen and field data of a screen to be implemented in the future has been proposed.

The motion compensation interpolation method estimates a motion vector in a screen using temporally continuous field data based on current field data and interpolates a current frame screen with reference to the motion vector.

By the way, the motion compensation interpolation method is composed of quite complicated hardware although the image quality is improved after interpolation.

Accordingly, in order to solve the problem of the motion compensation interpolation method, a motion adaptive interpolation method of estimating the degree of motion and interpolating a frame according to the motion has been proposed.

The motion adaptive interpolation method is implemented in relatively simple hardware compared to the motion compensation interpolation method, and the image quality after the interpolation is generally improved.

Examples of motion adaptive interpolation methods include the Bernard method shown in US Pat. No. 5,027,201 and the Faroudja method shown in US Pat. No. 5,159,451.

However, the conventional motion adaptive interpolation method has the advantage of being implemented with relatively simple hardware and improving the overall image quality after interpolation compared with the motion compensation interpolation method. Noise is generated and the manufacturing cost of the implementation circuit is increased due to the use of a plurality of field memories and complicated processing.

Accordingly, an object of the present invention is to provide a deinterlacing method and apparatus for improving image quality after interpolation by performing appropriate interpolation according to the degree of movement and edge direction of a field to be interpolated in order to improve the conventional problem.

In addition, another object of the present invention is to reduce the manufacturing cost of a circuit to be implemented by simplifying the circuit.

1 is an exemplary diagram of a screen according to a general interlacing method.

2 is a diagram illustrating field generation in the conventional line interpolation.

Figure 2a is an exemplary view showing a conventional line repetition method,

2B is an exemplary diagram showing a conventional interfield interpolation method without motion compensation;

2C is an exemplary diagram showing a conventional intrafield interpolation method.

3 is a block diagram of a deinterlacing apparatus for an embodiment of the present invention.

4 is an exemplary diagram showing region division for mapping of an edge portion.

5 is an exemplary view showing a relationship between an edge direction and a slope direction.

6 is an exemplary diagram of a Sobel mask for obtaining a direction change amount of a pixel value;

6A is an exemplary view showing an image area having a size of 3x3.

6B is an exemplary diagram of a Sobel mask for obtaining an amount of change in the y-axis direction of a pixel value.

6C is an illustration of a Sobel mask for obtaining an amount of change in the x-axis direction of a pixel value.

7 illustrates an example subsampling method for wide-angle edge detection.

8 is an exemplary view showing a field image region and a two-dimensional Gaussian filter.

8A is an exemplary view showing a pixel to be used for interpolation and surrounding pixels.

8B is an illustration of a two-dimensional Gaussian filter for noise component removal.

9 is an exemplary view showing an edge direction detection method based on gradients and hierarchical rules.

10 is an exemplary view showing an interval matching method for interpolation direction detection.

11 is an exemplary view showing observation region division for the interval matching method in the present invention.

12 is an exemplary view showing pixel and field data that are median filtered.

Explanation of symbols on the main parts of the drawings

100: filter storage unit 200: motion detection unit

300: spatial interpolator 400: temporal interpolator

500: soft switch unit 600: median filter unit

700: motion determination unit 800: vertical line conversion unit

According to an aspect of the present invention, there is provided a method of detecting a pixel of a predetermined region including a pixel to be interpolated from m field data corresponding to a perforated scanning image, and a change direction of a pixel to be interpolated with respect to the predetermined region. In the deinterlacing method of interpolating the current field data by performing a step of detecting and calculating an interpolation value with reference to the detected change direction of the pixel to be interpolated, the pixel to be interpolated with respect to a predetermined area is horizontal. Or sub-sampling with respect to neighboring pixels in an arbitrary area around the pixel to be interpolated when not present on the vertical edge, and detecting a change direction of the pixel to be interpolated in the subsampled pixel area. Dividing the region based on the detected change direction of the pixel to be interpolated; Calculating an interpolation value according to the error direction by obtaining a change direction error of a pixel to be interpolated with respect to an area.

In addition, the present invention detects the difference between the brightness and the brightness contour pattern from a plurality of consecutive field data including the field data on the basis of any field data to be interpolated to perform the above step, and obtains motion information. A deinterlacing apparatus comprising a motion determiner, a temporal interpolator, and a spatial interpolator to calculate a linear interpolated value by calculating the motion information and the edge direction of a pixel to be interpolated. Gradient when the edge direction does not exist on the horizontal or vertical edge when detecting the edge direction included in the pixel value of the field data and the pixel value around the pixel to interpolate the pixel value of the field as an input. Try to detect the edge direction by Backside sub-sampling a predetermined area based on the pixel to be interpolated to determine whether the wide angle is edged. If the pixel to be interpolated does not exist at the horizontal or vertical edge with respect to the subsampled area, It is configured to calculate the interpolation value in the field according to the direction indicating the minimum section matching error among the section matching errors of each region by detecting the edge direction and dividing the observation region arbitrarily set based on the detected direction. It is done.

That is, the present invention sub-sampling a region of a peripheral pixel in a predetermined range and a pixel to be interpolated and a peripheral pixel in a predetermined range when the pixel to be interpolated does not exist on a horizontal or vertical edge with respect to a predetermined peripheral region. Apply edge detection method based on gradient and rule to detect edge direction and apply "section matching method" to the observation area divided based on the edge direction. After applying the interval matching errors, the interpolation value is calculated in the direction indicating the minimum interval matching error among the detected interval matching errors.

Hereinafter, the present invention will be described in detail with reference to the drawings.

Hereinafter, the present invention will be described in detail with reference to the drawings.

3 is a block diagram of a de-interlacing apparatus for an embodiment of the present invention, as shown therein, a field storage unit for storing four field data of current field data, two previous field data, and four next field data, which are continuous in time And a brightness difference (BD) and a brightness contour pattern difference (BPPD) between the field data in the field storage unit 100 to calculate the brightness difference BD and the brightness. If the contour pattern difference (BPPD) is greater than or equal to a predetermined threshold, the motion determination unit 200 for mapping to a predetermined value and calculating the movement degree value with reference to the brightness difference (BD) and the brightness contour pattern difference (BPPD), and The median filter unit 600 removes the noise component included in the motion degree value and clusters the region having the motion degree value, and the motion value is inputted using the output value of the median filter part 600. A motion expansion unit 700 for diffusing a value into surrounding pixels, and a pixel value of a field data of a region to be interpolated from the field storage unit 100 and a pixel value of a peripheral pixel in a field to be interpolated. The spatial interpolation unit 300 calculates an edge direction included in the pixel values to calculate an interpolation value in the field according to the direction, and the pixel value of the previous field and the subsequent field with respect to the current field image to be interpolated. The temporal interpolator 400 calculates a field average value by averaging pixel values, and considers an edge direction of the spatial interpolator 300 by referring to a brightness difference and a brightness contour pattern difference in the motion extension unit 700. The soft switch unit 500, which mixes the intra-field interpolation value and the field average value in the temporal interpolation unit 400, and inputs data of the current field in the field storage unit 100 to the soft switch unit 500. The interpolation of the reference line data, and consists of a vertical line conversion unit 800 for converting the number of vertical lines to be suitable for display.

The motion determiner 200 detects the brightness change and the brightness outline pattern of each field data in the field storage unit 100 and compares the brightness change pattern and the brightness outline pattern between the previous field and the subsequent field based on the current field. The BD / BPPD detection unit 210 that calculates the brightness difference BD and the brightness outline pattern difference BPPD, and when the brightness difference BD and the brightness outline pattern difference BPPD are equal to or greater than a predetermined threshold, The BD / BPPD synthesis unit 220 maps the values and determines the movement degree value.

Referring to the operation and effect of the embodiment of the present invention configured as described above are as follows.

The field storage unit 100 m-fields consecutive in time such as image data corresponding to the nth field, the previous field, and the subsequent field among the plurality of field data for implementing the output image. Stores video data corresponding to

In other words, the field storage unit 100 stores field data before and after including the n-th field data and stores the stored field data in the motion determiner 200, the spatial interpolator 300, and the temporal interpolator ( 400).

At this time, the motion determiner 200 detects the difference between the pixel value of the specific lines existing in the field storage unit 100 and the brightness contour pattern and calculates the degree of motion of the video.

That is, the motion determiner 200 detects the brightness change and the brightness outline pattern of each field data in the field storage unit 100 by the BD / BPPD detector 210 to determine the brightness between the previous field and the subsequent field based on the current field. The brightness difference BD and the brightness contour pattern difference BPPD are calculated by comparing the change and the brightness contour pattern, and the BD / BPPD synthesis unit 220 calculates the brightness difference BD and the brightness contour pattern difference BPPD. ) To determine the movement value.

If the brightness difference BD and the brightness outline pattern difference BPPD are equal to or greater than a predetermined threshold value, the BD / BPPD synthesizing unit 220 may determine the brightness difference BD and the brightness outline pattern difference BPPD. Mapping to the set value determines the movement value.

Here, the concept of brightness difference (BD) is already widely used to perform line interpolation in the field of image processing such as digital TV.

The concept of Brightness Profile Pattern Difference (BPPD) is presented in Korean Patent Application No. 97-80719 filed by the present applicant.

In Korean Patent Application No. 97-80719, the quantification of the brightness contour pattern for calculating the brightness difference BD and the brightness contour pattern difference BPPD performs line interpolation for any of several lines required according to the application.

For example, assuming that four field data of n-2, n-1, n, and n + 1 are consecutive in time, the brightness difference BD and the brightness contour pattern difference BPPD in the motion determiner 200 are assumed. The calculation process of is as follows.

First, the brightness outline pattern of the n-2 th field data is extracted using the pixel value of the n-2 th field data.

That is, the brightness outline pattern is extracted at the time point at which the n-th field is input.

Then, the brightness outline pattern of the n-th field data is extracted using the pixel value of the n-th field data.

That is, the brightness outline pattern is extracted at the time when the nth field is input.

Thereafter, the motion determiner 200 compares the brightness contour pattern of the n-th field with the brightness contour pattern of the n-th field to calculate a difference BPPD between the brightness contour patterns of the n-2nd field and the n-th field. In addition, the brightness difference (BD) between the n-th field and the n-th field is calculated.

The motion determiner 200 calculates the brightness difference and the brightness outline pattern difference between the n-1 th field and the n + 1 th field in the same process as described above.

However, an image transmitted through a broadcast channel or the like inevitably causes noise to be mixed in a coding step or a transmission step.

Accordingly, in the present invention, in order to accurately set the output value of the motion determiner 200, in particular, the BD / BPPD synthesizer 220, the noise component is removed from the output value of the BD / BPPD synthesizer 220 and motion in the image is detected. The median filter unit 600 for grouping the divided regions, and the motion extension unit 700 for expanding the movement degree value to other pixels adjacent to the moving pixel by inputting the output value from the median filter unit 600. It is provided.

In this case, the motion extension unit 700 expands the motion degree value to the adjacent pixel as follows.

In general, a motion of a video is not performed only in a specific pixel but in a pixel group of a predetermined region.

Thus, if a motion degree value is detected for a particular pixel, it may be due to the noise component of that pixel or the particular pixel and adjacent neighboring pixels are in a motion state.

However, since the motion degree value corresponding to the noise component is already removed in the median filter unit 600, the surrounding pixels are likely to be in the motion state.

Therefore, the motion extension unit 700 diffuses the motion degree value output from the median filter unit 600 to the periphery of the pixel where the motion degree value is detected.

Meanwhile, the spatial interpolator 300 inputs the field data of the region to be interpolated from the field storage unit 100 to input pixel values of the n-th field data corresponding to the region to be interpolated and peripheral pixels in the field. Then, the edge direction including the pixel values around the pixel to be interpolated is calculated using the correlation of the pixels in the field, and then an appropriate interpolation value according to the edge direction is extracted.

In particular, in the present invention, the spatial interpolator 300 maintains the spatial and temporal coherence of the pixels to be interpolated, and the spatial values adjacent to the pixel values of the pixels that are spatially adjacent from the pixels to be interpolated and the current fields of the pixels to be interpolated. By extracting the interpolation value of the pixel to be interpolated by filtering the pixel value of ", " the edge direction detection method using gradient and rule " and " region matching method "

The operation principle of the "edge direction detection method according to the gradient and the rule" and the "region matching method" are described in Korean Patent Application No. 99-26084 filed on June 30, 1999 by the applicant. Since described in detail, the present invention will be described schematically.

First, the spatial interpolator 300 detects the edge direction by using the correlation of the pixels in the field.

At this time, in order to detect the edge direction of the pixel to be interpolated, as shown in the example of FIG. , , Assuming that the interpolation is performed in three directions, the spatial interpolator 300 correlates the pixels adjacent to the upper, lower, left, and right sides of the pixel to be interpolated to determine in which direction edge the pixel to be interpolated belongs. The relationship is first determined to determine whether the pixel to be interpolated belongs to a vertical or horizontal edge.

For example, in the case of interpolating the j-th pixel of the i-th line, a correlation in the vertical direction between the pixel to be interpolated and the adjacent pixel may be obtained by the following equation.

------ (One)

------ (2)

----- (3)

Thereafter, the spatial interpolator 300 performs the same process as the following program to calculate the edge direction of the pixel to be interpolated.

Here, the pixel to be interpolated This means the pixel in row i, column j of the nth field.

Used in the above formulas (1)-(3) The same meaning can be applied to.

That is, the spatial interpolation unit 300 determines the maximum value of the difference (a, b, c) of the low pass-filled pixels obtained by equations (1) to (3). To calculate its maximum value ( If it is less than this threshold, the edge direction of the pixel to be interpolated Judging by.

Also, in a similar manner to the above vertical correlation detection, the horizontal correlation is detected for the low pass filtered pixels as described above, and when the calculated value is smaller than a predetermined threshold in the horizontal direction, the edge direction of the pixel to be interpolated Also Judging by.

Conversely, the maximum value of the difference (a, b, c) between the low-pass filtered pixels ( If the pixel is equal to or larger than the predetermined threshold, that is, if the pixel to be interpolated exists in a region having low correlation in the vertical or horizontal direction, the spatial interpolator 300 detects the edge direction by the gradient. Method to detect the edge direction, which will be described below.

Since the pixel to be interpolated does not belong to the vertical or horizontal edge, in order to calculate the edge direction to which the pixel to be interpolated belongs, first, use the 3x3 Sobel mask as shown in the example of FIG. do.

6A to 6A illustrate a 3x3 Sobel mask, and FIG. 6B illustrates a 3x3 Sobel mask used to obtain a y-axis slope in the pixel x5, and FIG. 6C illustrates an x-axis slope in the pixel x5. The 3x3 Sobel mask is used to obtain.

At this time, the inclination of the pixel value is calculated by substituting the pixel value in the Sobel mask in Fig. 6 into the following equation (4) (5).

------- (4)

------- (5)

here, Is the angle measured with respect to the x-axis.

Thus, as shown in Fig. 5, the direction of the gradient is determined by the edge direction ( Perpendicular to the edge direction ) Can be obtained by the following operation (6).

------ (6)

Accordingly, the spatial interpolation unit 300 has the edge direction detected by the inclination in the above. Determine if you belong to.

At this time, the edge direction of the operation result is In order to determine whether a pixel to be interpolated exists on a wider edge, as shown in the example of FIG. 7A, subsamples the columns jn to j + n based on the pixel to be interpolated. Get the Sobel mask.

Here, in order to prevent aliasing of pixels during subsampling, low pass filtering may be performed to limit the frequency band to a desired frequency band and then perform subsampling.

Subsequently, the spatial interpolation unit 300 performs the same operation as the above formulas (1) to (3) on the Sobel mask as shown in Fig. 7B, and the maximum value of the difference value (a, b, c) between pixels at that time ( Is compared with a threshold to determine whether the pixel to be interpolated exists at the horizontal or vertical edge as mentioned above.

Accordingly, when the pixel to be interpolated is present at the horizontal or vertical edge as a result of the determination on the subsampled pixel, the spatial interpolator 300 adjusts the edge direction of the pixel to be interpolated. Judging by

On the contrary, if the pixel to be interpolated does not exist at the horizontal or vertical edge, the spatial interpolator 300 calculates the edge direction by performing the same operation as in Equations (4) to (6).

The initial estimated pixel value of the line to be interpolated used in a given pixel when detecting the edge direction due to the inclination is simply linearly interpolated in the vertical direction as shown in FIG. 8 or a two-dimensional Gaussian filter. Use the value applied.

That is, FIG. 8A shows a 3x3 image region of a progressive scan field image, and FIG. 8B shows an example of a two-dimensional Gaussian filter for removing noise components. In FIG. 8A, x is linearly interpolated in the vertical direction in FIG. 8A. The initial estimated pixel value The initial estimated pixel value at x after linear interpolation in the vertical direction and passing through the two-dimensional Gaussian filter as shown in FIG. Becomes

Therefore, the spatial interpolator 300 performs the series of processes by the following program.

IF (on horizontal or vertical edge) THEN

edge direction =

ELSE

Edge direction detection by gradient calculation

IF (edge direction = ) THEN

Subsampling around the pixel you want to interpolate

IF (on horizontal or vertical edge) THEN

edge direction =

ELSE

Edge direction detection by gradient calculation

ENDIF

ENDIF

ENDIF

That is, if the above process is briefly described, if the pixel to be interpolated detects a correlation between the pixel to be interpolated and the pixel to be interpolated exists on a horizontal or vertical edge, the edge direction is Judging by.

If the edge direction of the pixel to be interpolated If not, apply the "edge direction detection method by the gradient" to calculate the edge direction, Determine if it is.

At this time, the edge direction If it is determined, subsampling is performed centering on the pixel to be interpolated, and it is determined whether the pixel to be interpolated exists on a horizontal or vertical edge by using Equations (1) to (3).

Accordingly, when the pixel to be interpolated exists on the horizontal or vertical edge, the edge direction of the pixel to be interpolated is finally changed. Judging by.

On the contrary, if the pixel to be interpolated is not present on the horizontal or vertical edge after the subsampling of the pixel, the edge direction is calculated by applying the edge direction detection method by the gradient to the subsampled pixel.

Subsequently, the above process is repeated for the pixels placed on the positions of j + 1, j + 2, j + 3, j + 4, ..., and the direction values Dir (j + 1) of each pixel are repeated. , Dir (j + 2), Dir (j + 3), Dir (j + 4), ... are continuously calculated.

In addition, when the spatial interpolator 300 detects the edge direction of the pixel to be interpolated in the above-described process, the spatial interpolation unit 300 is initially misused by the noise component using the "edge detection method by hierarchical rule" as shown in FIG. The detected edge direction is corrected to the correct edge direction by comparing and analyzing the edge direction to which edge pixels belong to.

First, the first program is described with reference to the direction values Dir (j + 1), Dir (j + 2), Dir (j + 3), Dir (j + 4), and Dir (j + 5) of each pixel. By performing the operation of the module, the direction values NewDir (j + 3) of pixels positioned in the horizontal direction adjacent to the pixel from which the original direction value is to be estimated are estimated.

Further, by replacing j above with j-1, the direction value (NewDir (j + 2)) of the adjacent locator is estimated in the same manner.

Thereafter, the direction values NewDir (j + 2) and NewDir (j + 3) calculated above and the direction values of the pixel and the previous pixel OutDir (j) and OutDir (j + 1) for which the current direction value is to be obtained. The direction value OutDir (j + 2) of adjacent pixels is calculated after performing the calculation process of the second program module using)).

Accordingly, the direction values (OutDir (j + 2)) of the adjacent pixels and the pixel values for which the current direction value is to be obtained and the direction values of the previous and subsequent pixels (OutDir (j-2), OutDir (j-1), and OutDir ( j), by performing the calculation process of the third program module using OutDir (j + 1)), the edge direction FinalDir (j) of the pixel to be interpolated is corrected in the correct direction.

Subsequently, the region matching is performed to detect a finer edge direction with respect to the edge direction FinalDir (j) determined macroscopically for the pixel to be interpolated and to calculate interpolation by linearly interpolating peripheral pixel values. Method ".

The interval matching method is a method of extracting an interpolation value by using a property that pixel values of an edge portion of an area to be interpolated are continuous along the edge direction.

First, a pixel value existing in a sliding window existing on an arbitrary edge including a pixel to be interpolated is detected.

In this case, the size of the moving area may be arbitrarily adjusted according to the system.

Then, while shifting the up and down moving areas in opposite directions at regular intervals, the difference between pixels corresponding to each other in the up and down moving areas is continuously detected for various edge directions corresponding to the shifted amounts.

At the same time, the direction of the edge including the pixel to be interpolated is detected by referring to the amount of change in the pixel value which is continuously detected.

For example, FIG. 10 shows the macroscopic edge direction FinalDir (j) determined by the “edge direction detection method based on gradient and hierarchical rule”. A section matching method for determining a detailed direction in one case is illustrated.

Accordingly, a predetermined sliding window is set in the i + 1th line and the i-1th line in a constant observation window based on the ith line to be interpolated.

Then, the moving area of the i-1th line moves from left to right and the moving area of the i + 1th line moves from right to left.

The difference value of the pixels located on the diagonal lines between the moving areas is detected and the difference is summed, or the sum is converted into an average difference value per pixel in the predetermined moving area to calculate the change amount of the pixel value.

Here, the case in which the movement interval of the movement region is set in half of the adjacent pixel interval is taken as an example. However, for the sake of precision, the movement interval of the movement region can be adjusted at fine intervals such as 1/4, 1/8, ... have.

In this case, the pixel values of pixels horizontally adjacent to each other in the viewing area are linearly interpolated, and the linearly interpolated pixel values are set to values between horizontally adjacent pixels in the viewing area.

Subsequently, the difference in pixel values of pixels located in the moving area is calculated by Equation 7 below, or The average difference per pixel divided by the number of pixels located in the moving area is calculated.

-------- (7)

Here, m may be determined up to the diagonal direction to be obtained according to the application.

For example, the range of m can be set from -3 to +3, and the interval of m values between -3 and +3 is defined by the moving interval of the moving area.

Then, the minimum value is extracted from the value obtained by the above equation (7) and the Extract the value.

As an example, if the range of m is -3 to +3 and the moving interval of the moving area is defined as half pixel To calculate the interpolation value.

That is, the process of calculating the interpolation value by the “interval matching method” will be described as follows.

First, pixel values of pixels included in the moving area are linearly interpolated between pixels adjacent to each other horizontally.

Then, pixel values interpolated between horizontally adjacent pixels are inserted between the horizontally adjacent pixels, and the difference between the pixels between the moving areas is calculated and summed.

Then, the minimum value of these sum values is calculated, and the interpolation value is calculated by linearly interpolating pixel values existing at the center of the up and down moving area on the diagonal on which the minimum value is calculated.

If the minimum value of the value calculated by the above equation (7) If two or more are calculated, the interpolation value of the nearest point is given priority as a reference point when m is zero.

Then, two m is set as a reference point when m is 0. Is computed at this same distance point, two The interpolation value is calculated by filtering the pixel value located at the center of the existing moving area.

However, the sum of the difference of pixel values between the moving areas along each diagonal direction ( ) Has a value above the predetermined threshold, the interpolation value is Is 0, i.e., equal to the value when linear interpolation is performed in the vertical direction.

In the present invention, the size of the viewing area and the number of pixels included in the viewing area may be arbitrarily adjusted.

Of course, the size of the moving area may be arbitrarily adjusted within the range of the viewing area, and the number of pixels included in the moving area may be arbitrarily adjusted.

However, in the present invention, when applying the "section matching method" in detail according to a given edge direction, the "section matching method" is applied to each area by dividing into a plurality of areas.

Accordingly, the minimum section matching error of each region ( ) Is interpolated and the pixel value of the pixel position to be interpolated is interpolated according to the direction indicating the minimum interval matching error.

For example, Figure 11 shows the macroscopic edge direction FinalDir (j) determined by " edge direction detection method based on gradient and hierarchical rules " In this case, the "section matching method" for each region is illustrated by dividing it into two regions A and B.

Therefore, referring to FIG. 11, a process of interpolating a pixel value of a pixel position to be interpolated at present is as follows.

In Fig. 11, area A is the edge direction. Detected based on From the area up to a certain edge direction and the area (B) Direction to the end of the region to be observed in the particular edge direction.

In addition, the specific edge direction which becomes the upper boundary can be arbitrarily appropriately selected in consideration of the size of the application and the moving area.

In this case, interval matching errors are obtained for each region A and B by the same operation as in Equation (7), and the region matching errors for each region A and B are compared to each region A. Minimum interval matching error in (B) ) ( ).

After that, the minimum interval matching error for each area (A) (B) ( ) ( ).

At this time, the interval matching error ( ) Is an interval matching error ( If less than or equal to, the pixel value to be interpolated is the minimum section matching error ( The interpolated value depends on the direction indicated by

Conversely, the interval matching error ( ) Is an interval matching error ( Greater than), the minimum interval matching error ( The interpolated pixel value is obtained according to the direction indicated by) to determine whether the pixel value exists between the pixel values existing immediately above and below the pixel to be interpolated.

Accordingly, the minimum interval matching error ( If the interpolated pixel value is between the pixel values immediately above and below the pixel to be interpolated, the pixel value becomes the pixel value to be interpolated.

If the minimum interval matching error ( If the interpolated pixel value is not between the pixel values immediately above and below the pixel to be interpolated, the pixel value to be interpolated is the minimum section matching error of the region A. The interpolated value depends on the direction indicated by

That is, the above process is performed by the following program.

IF ( ) THEN

ELSE

IF (Median { } = ) THEN

ELSE

ENDIF

ENDIF

here, Denotes a pixel in row i, column j of the nth field, which is the pixel to be interpolated at present.

Also, Wow Denotes values interpolated along directions indicating minimum section matching errors in regions A and B, respectively.

Therefore, the spatial interpolation unit 300 performs interpolation using pixels that are spatially adjacent to the pixel to be interpolated as much as possible, and interpolates pixels to be interpolated when pixels are relatively far apart. By checking the correlation with adjacent pixels and interpolating using these pixels only when the correlation is high, more precise and stable interpolation can be performed.

In addition, the spatial interpolator 300 uses the direction detection method based on the slope and the rule first and then the interval matching method later to determine the pixel value to be interpolated.

The interval matching method has a simple operation principle, but has a disadvantage in that it is difficult to accurately detect an interpolation value due to a large amount of calculations for detecting one edge direction.

On the other hand, the direction detection method based on the slope and the rule has an advantage that the edge direction can be easily detected by a simple operation, but has a disadvantage in that it is difficult to accurately detect the edge direction.

Therefore, the present invention uses the "interval matching method" only when extracting the linear interpolation value of the pixel where the edge direction is detected by using only the advantages of the above methods, and the "inclination degree" when detecting the edge direction to which the pixel to be interpolated actually belongs. And an edge direction detection method based on rules ".

Accordingly, the present invention provides an efficient edge detection method by greatly reducing the amount of computation rather than applying only the "section matching method", and more precise edge direction detection is achieved than when only the "edge direction detection method based on gradient and rule" is applied. It becomes possible.

Then, the median for the surrounding pixels of the pixel to be interpolated in the current field, in particular, two pixels in the diagonal direction used for linear interpolation, the pixel at the interpolated position, and the two pixels at the current interpolated position of the previous and subsequent fields. Perform filtering.

For example, FIG. 12 is an exemplary diagram showing adjacent pixels used for temporal and spatial filtering for interpolation pixel determination.

At this time, the peripheral pixels of the pixel to be interpolated in the current field (n-th field), in particular, two pixels in the diagonal direction used for linear interpolation ( , ) And the pixel at the interpolated position ( Then, the pixel at the currently interpolated position of the previous field (n-1th field) and the subsequent field (n + 1th field) , ), Median filtering is performed by the operation as shown in Equation (8) below to remove the noise component of the pixel.

------ (8)

Therefore, by performing the median filtering as shown in Equation (8), not only the noise component is removed by temporally and spatially filtering the pixels to be interpolated at present, but also the temporal interpolation of the still image which is considered as moving image by the motion extension unit 700. The effect can be exhibited and stability can be increased by maintaining consistency in the time axis direction.

In order to calculate a smoother and more stable interpolation value in the current field (nth field), median filtering is performed by giving a predetermined weight to the interpolated pixel value according to the edge direction calculated above. can do.

Median filtering considering weight is performed by the operation as shown in Equation (9) below.

------ (9)

However, as described above, the interpolation pixel in which all the median filtering is performed ( ) Is determined by other pixels that are spatially far from the current interpolation position or when they are present in very fast motion, the interpolation pixel value may represent a significantly different value than other adjacent pixels.

In order to prevent such a phenomenon, the spatial interpolation unit 300 performs median filtering in the vertical direction by the following program with reference to pixels adjacent in the vertical direction from the position of the pixel to be interpolated in the field.

The operation of the vertical median filtering is as follows.

If the distance horizontally away from the current position of the pixel to be interpolated is smaller than the m value determined by Equation (7), the interpolation pixel estimated by Equation (8) or (9) is determined as an interpolation value. do.

However, if the distance horizontally away from the position of the pixel to be interpolated at present is not smaller than the m value determined by Equation (7), the interpolation value is interpolated with the interpolation pixel estimated by Equation (8) or (9). The median filtered value is determined by referring to pixel values of adjacent pixels in a vertical direction of the pixel to be used.

The temporal interpolation unit 400 extracts a field average value by averaging pixel values of previous field data and pixel values of subsequent field data corresponding to the same position as the pixel to be interpolated at present.

For example, if it is necessary to interpolate the j-th pixel of the i-th line to newly generate the n-th field image, the temporal interpolation unit 400 includes the pixels of the n-1 th field data having the image data of the j-th pixel of the i-th line. The field average value is extracted by averaging the value and the pixel value of the n + 1th field data.

That is, the field average value in the temporal interpolation unit 400 is calculated by the following equation (10).

------ (10)

In this case, the soft switch unit 500 receives the brightness difference BD and the brightness contour pattern difference BPPD calculated by the motion determiner 200 and spatially refers to the motion degree value output from the motion expansion unit 700. The interpolation value in consideration of the edge direction output from the interpolation unit 300 and the field average value output from the temporal interpolation unit 400 are mixed and output.

The soft switch unit 500 outputs an operation value as shown in Equation (11) below.

----- (11)

here, Is a motion degree value output from the motion expansion unit 700 and is a value in the range of '0 or more and 1 or less'.

Accordingly, the vertical line converter 800 refers to the interpolation value output from the soft switch unit 500 and the current field data values stored in the field storage unit 100 to generate an interpolation line suitable for the display device. Convert the number of vertical lines on the screen.

If only the deinterlaced field data is needed without line conversion, the vertical line converter 800 passes the values output from the field storage unit 100 and the soft switch unit 500 and outputs the same.

As described in detail above, the present invention has an effect of improving image quality performance by performing more precise edge direction detection and interpolation than the conventional deinterlacing method.

Claims (10)

detecting pixels in a predetermined area including pixels to be interpolated from m field data, calculating a correlation between the pixels in the predetermined area, and detecting a change direction of the pixels to be interpolated; A deinterlacing method for interpolating current field data by performing an operation of calculating an interpolation value with reference to a change direction of a pixel, wherein the pixel to be interpolated is selected if there is no pixel to be interpolated for the predetermined area on a horizontal or vertical edge. A first step of sub-sampling the peripheral pixels in an arbitrary area around the center, a second step of detecting a change direction of a pixel to be interpolated in the subsampled pixel area, and the detected interpolation And a third step of calculating an interpolation value with reference to a change direction of a pixel to be performed. The method of claim 1, wherein the first step of subsampling a predetermined area around the pixel to be interpolated comprises: a first process of obtaining a correlation between the pixel to be interpolated and a neighboring pixel; A second process of determining whether it exists on a vertical or horizontal edge, a third process of calculating an inclination of an edge portion if the pixel to be interpolated does not exist on a vertical or horizontal edge, and a pixel to interpolate with reference to the inclination The fourth process of determining the change direction of the pixel and the change direction of the pixel And a fifth process of subsampling the predetermined area pixels of the upper and lower lines based on the pixel to be interpolated. The deinterlacing method of claim 2, wherein the third step of calculating the gradient comprises previously performing two-dimensional Gaussian filtering on values linearly interpolated in the vertical direction. The method of claim 2, wherein the fifth step of subsampling comprises performing low-pass filtering to limit a frequency band before subsampling. 2. The method of claim 1, wherein the second step is a first process of obtaining a correlation between pixels to be interpolated and neighboring pixels in a subsampled region, and determining whether a pixel to be interpolated exists on a vertical or horizontal edge by referring to the correlation. A second process of performing an edge direction if the pixel to be interpolated exists on a vertical or horizontal edge A third step of determining a, a fourth step of calculating an inclination of an edge portion when the pixel to be interpolated does not exist on a vertical or horizontal edge, and a step of determining a change direction of a pixel to be interpolated with reference to the inclination Deinterlacing method characterized in that consisting of five processes. The method of claim 1, wherein the third step of calculating the interpolation value according to the edge direction comprises: a first process of dividing the first and second regions based on the change direction of the pixel to be interpolated detected in the second step; And a second process of obtaining a change direction error of a pixel to be interpolated for the divided region and calculating an interpolation value according to the error direction. The method of claim 6, wherein the first area is in the edge direction Deinterlacing method characterized in that it is set to the edge direction detected by the slope based on the second region to the end of the region to be observed in the detected edge direction. The method of claim 6, wherein the second process includes obtaining and comparing interval matching errors of the first and second regions, and if the interval matching error of the first region is equal to or smaller than the interval matching error of the second region, Calculating linearly interpolated values in the direction indicating the minimum section matching error of the region; and if the section matching error of the first region is greater than the section matching error of the second region, Interpolating and comparing the interpolation values between pixel values existing above and below the pixel to be interpolated; and pixel values linearly interpolated in the direction indicating the minimum section matching error of the second region. Calculating linearly interpolated values in a direction indicating a minimum section matching error of the second region when the pixel values exist between the upper and lower pixel values of the second region; If the pixel value linearly interpolated in the direction indicating the minimum interval matching error of is not between the pixel values existing above and below the pixel to be interpolated, the linearly interpolated value is calculated in the direction indicating the minimum interval matching error of the first region. Deinterlacing method comprising the process of. A motion determiner that detects the difference between brightness and brightness contour pattern by inputting arbitrary field data to be interpolated and its surrounding field data and calculates the degree of motion of the image; and A temporal interpolation unit that calculates a field average value by averaging pixel values of a field, and pixel values around pixels to interpolate pixel values of field data of a region currently interpolated from the field storage unit and peripheral pixel values in a field as inputs. A spatial interpolation unit for calculating an edge direction included in the interpolation unit and calculating an interpolation value in a field according to a direction indicating a minimum section matching error according to the direction, and a brightness difference and a brightness contour pattern difference in the motion extension unit; Vertical line transformation by mixing the intra-field interpolation value in the spatial interpolation unit and the field average value in the temporal interpolation unit according to Softswitch de-interlacing apparatus comprising: a unit configured to output. 10. The method of claim 9, wherein the spatial interpolation unit inputs an edge direction included in the pixel values of the field data of the area to be interpolated from the field storage unit and the pixel values around the pixel to be interpolated by inputting peripheral pixel values in the field. When detected, the edge direction does not exist at the horizontal or vertical edge. If the pixel to be interpolated is not present at the horizontal or vertical edge with respect to the subsampled region by sub-sampling a predetermined region based on the pixel to be interpolated. Detects the edge direction by gradient, divides randomly set observation area based on the detected direction, and detects the minimum section matching error for each region, and then detects the edge of the detected minimum section matching error. direction And calculating an interpolation value in the field according to the direction indicating the minimum interval matching error closest to the.