KR100931110B1 - Deinterlacing apparatus and method using fuzzy rule-based edge recovery algorithm - Google Patents

Abstract

The present invention provides a deinterlacing apparatus and method capable of preventing deterioration of image quality in a vertical direction even when the frequency of the time axis increases. According to an embodiment of the present invention, a deinterlacing apparatus includes a plurality of first pixels and a current field included in a previous field image and a next field image adjacent in a time direction with respect to a current field image including an interpolation target pixel of an input-parallel scan signal. An edge for determining the detailed direction along with whether the main edge direction for the interpolation target pixel belongs to the time domain or the spatial domain using a plurality of second pixels included in the image and adjacent in the spatial direction to the interpolation target pixel; And a third pixel adjacent upward and downward in the main edge direction with respect to the second pixel adjacent in the spatial direction to the interpolation target pixel when it is determined that the direction detecting unit and the main edge direction belong to the spatial domain. To calculate the upper difference and the lower difference for the interpolation target pixel using A lower difference calculator, an edge type recognition unit for determining an edge type for the interpolation target pixel using characteristics of the upper difference and the lower difference, and the main edge direction belonging to the time domain Linear interpolation is performed using the first pixels adjacent to the main edge direction, and when it is determined that the main edge direction belongs to the spatial domain, the second and third pixels adjacent to the main edge direction are used. And an interpolation unit for adaptively performing interpolation according to the edge type determined by the edge type recognition unit to output a progressive scan signal.

Description

Deinterlacing apparatus and method using fuzzy rule-based edge reconstruction algorithm {Equipment and method for de-interlacing using fuzzy rule-based edge-restoration algorithm}

The present invention relates to de-interlacing of video signals, and more particularly, to an apparatus and method for deinterlacing using fuzzy rule-based edge recovery algorithm.

Interlacing is the division of an entire frame into two parts called fields. One of the two fields is an image composed of even-numbered horizontal lines, and the other is an image composed of odd-numbered horizontal lines. In broadcast or storage devices, such field images are used to encode, transmit, and / or reproduce in a widely accompaniment scanning format, which has an advantage of efficiently using a limited bandwidth without significant deterioration in image quality. For example, such a conventional scanning format has been widely used in broadcast standards such as the National Television System Committee (NTSC), Phase Alternation Line (PAL), and Secam (SECAM).

Conventional scanning methods for displaying field images have shown satisfactory effects in human visual systems, but interlacing artifacts have begun to appear due to the increase in screen size of flat panel display devices with the advent of HDTV. An interlacing defect is a phenomenon in which image quality deteriorates, for example, a line crawl phenomenon, a serration distortion, and an inter-line flicker. Therefore, the recent HDTV system supports a progressive scan method to provide an image of improved quality using a flat screen display device having a large screen size.

De-interlacing is a technique of converting a scan format so that an image of a progressive scan format can be converted into an image of a progressive scan format and output. This deinterlacing is important in how exactly the predicted or interpolated pixel values for the empty lines of the field image match the pixel values of the actual image. In the deinterlacing, it is necessary to accurately restore the pixel value of the original frame image and maintain edge sharpness or the like. In particular, since low-pass filtering and decimated video signals have lower resolution than the original video, it is particularly important to reconstruct the original video including edge sharpness more accurately and precisely by deinterlacing. Do.

De-interlacing can be broadly classified into two methods, a motion compensation method and a motion compensation method. The latter method is generally widely used because of the disadvantages of error propagation and complicated calculations. De-interlacing also depends on the temporal method, spatial method, spatial method, depending on which domain information, i.e., spatial domain information and / or temporal domain information, is used as the reference pixel in interpolating. And Spatio-Temporal Adaptive methods.

A typical deinterlacing algorithm that uses spatial domain information is a method called Bob. Bob creates a frame using each line of the field twice. In other words, a new frame image is reconstructed by interpolating interpolation by inserting average data of two upper and lower lines into an empty line between two lines of the field image. As described above, the method of using spatial domain information is also referred to as intra-field interpolation because it uses pixel information at different positions in the same field. The edge-based line averaging (ELA) algorithm is a method of using edge direction in the same field image for interpolation, which is a kind of intrafield interpolation method.

A typical deinterlacing algorithm using time domain information is a method called 'weave'. According to Weave, a single frame image is generated by combining a top-field image and a bottom-field image. That is, one frame is implemented by simply inserting the line of the immediately preceding field between the lines of the current field. As described above, the method of using time domain information is also referred to as inter-field interpolation because it interpolates using pixel information of another field.

In addition, there is a spatial-temporal edge-based line averaging (STELA) method as one of methods for using spatial domain information and time domain information together and considering edge orientation. The STELA algorithm uses an edge-based line averaging technique over space-time windows.

However, various deinterlacing methods proposed to date show good performance in a specific environment, but when the environment changes, the performance may not meet expectations. In other words, the existing various methods do not produce satisfactory results in various environments, and still need to improve the quality of the deinterlaced image and the operation speed for deinterlacing. In particular, the conventional space-time-based interpolation algorithm such as STELA shows a problem that the image quality in the vertical direction drops as the frequency of the time axis increases.

An object of the present invention is to provide a deinterlacing apparatus and method in which the image quality in the vertical direction of the deinterlaced image does not degrade even when the frequency of the time axis increases.

Another object of the present invention is to provide a deinterlacing apparatus and method capable of reconstructing a more detailed image and improving image quality by improving sharpness of a boundary portion.

Another object of the present invention is to provide a deinterlacing apparatus and method which can relatively alleviate an increase in the amount of computation in improving the image quality of a deinterlacing image.

Deinterlacing apparatus or method according to an embodiment of the present invention for solving the above problems is included in the previous field image and the next field image adjacent in the time direction with respect to the current field image including the interpolation target pixel of the input-parallel scan signal By using a plurality of first pixels and a plurality of second pixels included in the current field image and adjacent to the interpolation target pixel in a spatial direction, whether the main edge direction of the interpolation target pixel belongs to a time domain or a spatial domain Edge direction detecting unit or edge direction detecting step for determining a detailed direction together; When it is determined that the main edge direction belongs to the spatial domain, the interpolation is performed using third pixels adjacent to the interpolation target pixel in the spatial direction and upward and downward in the main edge direction, respectively. An upper and lower difference calculation unit or upper and lower difference calculation steps for calculating an upper difference and a lower difference for the target pixel; An edge type recognition unit or edge type recognition step for determining an edge type for the interpolation target pixel using characteristics of the upper difference and the lower difference; And when the main edge direction is determined to belong to the time domain, linear interpolation is performed by using the first pixel adjacent to the main edge direction, and when the main edge direction is determined to belong to the spatial domain, And an interpolation unit or interpolation step for outputting a progressive scan signal by adaptively interpolating according to the edge type determined by the edge type recognition unit by using the second and third pixels adjacent in the edge direction.

According to this problem solving means, when the main edge direction of the interpolation target pixel belongs to the spatial domain, not only the pixel directly adjacent to the interpolation target pixel but also the pixel of the line adjacent to and above the interpolation target pixel is used to determine the edge type of the interpolation target pixel. Determine, and also adaptively interpolate according to the determined edge type. Therefore, according to an embodiment of the present invention, even if the frequency of the time axis increases, the image quality of the deinterlaced image can be prevented from falling, and the image of the image can be reconstructed in the vertical direction. You can improve the picture quality by improving. In addition, according to the embodiment of the present invention, it is possible to relatively alleviate an increase in the amount of calculation in improving the image quality of the deinterlaced image.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention. Since the embodiments described below are for the purpose of illustrating the technical idea of the present invention, the technical idea of the present invention should not be construed as being limited by the embodiments. Reference numerals added to the respective components in the description of the embodiment and the drawings are merely described for convenience of description.

1 is a block diagram showing the configuration of a deinterlacing apparatus according to an embodiment of the present invention. Referring to FIG. 1, the deinterlacing apparatus 100 includes a main edge direction detector 110, an upper and lower difference calculator 120, and an edge pattern recognizer 130. , And an interpolating unit 140.

The edge direction detector 110 determines a main edge direction by using pixels to be interpolated, that is, pixels adjacent to the interpolation target pixel in the temporal and spatial directions. More specifically, the edge direction detecting unit 110 determines whether the main edge direction belongs to a temporal domain (TD) or a spatial domain (SD), and determines the detailed direction of the main edge in the corresponding domain. . To this end, the edge direction detection unit 110 uses directional correlation between pixels to linearly interpolate the missing line. Examples of adjacent pixels used to determine the edge direction are pixels included in the spatial-temporal window shown in FIG. 2. In general, the pixel value of the pixel to be interpolated is represented by x ( i , j , k ), and as described below, since one embodiment of the present invention uses fuzzy rule-based image processing, interpolation is performed. The pixel value of the target pixel is represented by x _FER ( i , j , k ). Here, the variable i represents a column (ie, vertical line) number of the image, the variable j represents a (horizontal) line number, and the variable k represents a field number.

Referring to FIG. 2, in the edge direction detecting unit 110, pixels included in a spatial window as pixels adjacent to an interpolation target pixel, that is, the (j-1) th and (j + 1) th lines of the same kth field. Six pixels 211, 212, 213, 214, 215, and 216 and pixels included in the time window, that is, three pixels 221, 222, and 223 of the jth line of the (k-1) th field 220. And the main edge direction are determined using the three pixels 231, 232, and 233 of the j-th line of the (k + 1) -th field 230. Here, {u, d, r, l, p, n} of FIG. 2 represents {up, down, right, left, previous, next}, respectively. In order to measure the space-time correlation of the pixels included in the space-time window, the edge direction detector 110 calculates a change in six directions (indicated by dashed lines) using adjacent pixels. An example of an equation for calculating the change is shown in Equation 1.

The edge direction detecting unit 110 determines whether the main edge direction belongs to the time domain TD or the spatial domain SD and the correct direction of the main edge in the corresponding domain by using the calculation result according to Equation 1 above. do. The major edge direction can be determined using, for example, Equation 2.

Referring to Equation 2, the direction having the smallest difference among the difference values for the six directions is determined as the main edge direction. In Equation 2, C _{Φ, θ} represents a directional correlation measurement, which is a parameter indicating whether the main edge direction belongs to the spatial domain or the time domain, Φ (∈ {S D , TD }) and θ (∈ {45 ^o , 0 ^o , -45 ^o }) which indicates a specific direction in the domain.

According to the present invention, when the edge direction detection unit 110 determines that the main edge direction belongs to the time direction, that is, the parameter of the directional correlation measurement value C _{Φ, θ} calculated by Equations 1 and 2, When φ corresponds to TD, the input signal is transmitted directly to the interpolator 140. The interpolator 140 obtains pixel values of the interpolation target pixel using general linear interpolation. When the main edge direction belongs to the TD, the method of obtaining the pixel value of the interpolation target pixel in the interpolation unit 140 may be expressed by, for example, Equation (3).

On the other hand, if the edge direction detection unit 110, it is determined that the edge direction belonging to the spatial direction, that is, a parameter Φ of the directional correlation measurements (C _{Φ, θ)} calculated by the Equations (1) and (2) in SD If applicable, the edge type is further subdivided for the interpolation target pixel, and then interpolation is adaptively performed according to each type. To this end, according to an embodiment of the present invention, the upper and lower difference calculation unit 120 is a line higher or lower than the pixels included in the spatial window of FIG. 2 in the k-th field which is the same field as the interpolation target pixel. Using the pixels located at, the upper and lower differences in the main edge direction are further calculated. The upper difference may be calculated by subtracting the pixel value of the (j-1) -th line from the pixel value of the (j-3) -th line, that is, the difference between the pixel values with the adjacent upper line in the determined main edge direction. The lower difference may be calculated by subtracting the pixel value of the (j + 1) -th line from the pixel value of the (j + 1) -th line, that is, the difference between the pixel values with the adjacent lower line in the determined main edge direction. .

The edge type recognition unit 130 determines the subdivided edge type of the pixel to be interpolated using the upper difference and the lower difference. According to an embodiment of the present invention, an edge type of a pixel to be interpolated may be any one of 'simple edge', 'monotonic slope', or 'peak or valley'. Can be. The term 'simple edge' refers to a case where an edge is formed in a pixel to be interpolated in the main edge direction due to the characteristics of the upper difference and the lower difference (see FIG. 6D). 'Forging slope' refers to a case in which the pixel value monotonously increases or monotonically decreases in the direction of the main edge due to the characteristics of the upper difference and the lower difference (see FIG. 6E). 'Peak or valley' refers to the case where a peak is formed or a valley is formed in the pixel to be interpolated in the direction of the main edge due to the characteristics of the upper difference and the lower difference (see FIG. 6F).

More specifically, in the edge type recognition unit 130, whether the upper difference and the lower difference are positive (P) or negative (Negative, N), and / or the absolute values of the upper difference and the lower difference are respectively predetermined. It may be determined whether the edge type corresponds to a 'simple edge', 'forged slope', or 'peak or valley' based on whether it is greater than Big, B or Small (S). Here, the threshold is difficult to determine theoretically and can be determined as an appropriate value through experimentation. The threshold value when the upper difference and the lower difference is positive and the threshold value when the negative value may be the same, but may be different.

For example, when one of the absolute values of the upper difference and the lower difference is greater than the threshold and the other is smaller than the threshold, the edge type recognizer 130 sets the edge type of the pixel to be interpolated as 'simple edge'. Can be determined. If one of the absolute value of the upper difference and the lower difference is greater than the threshold and the other is smaller than the threshold value, the 'upper difference and the lower difference' is 'big negative (BN) and small negative (SN)', The first case of "big negative and small positive" (SP), "big positive (BP) and small negative", or "big positive and small positive (SP)", and "small negative and big negative", "small There may be a second case for negative and large positive numbers, 'small positive and large negative numbers', or 'small positive and large positive numbers'.

And when both the absolute value of the upper difference and the lower difference is greater or smaller than the threshold, the edge type recognition unit 130 determines the edge type of the pixel to be interpolated depending on whether the sign of the upper difference and the lower difference is the same. Forging slope 'or' peak or valley '. More specifically, when the sign of the upper difference and the lower difference is the same, it is determined as' forging inclination, 'In this case, the' upper difference and lower difference 'is the' big negative and big negative ',' small negative Small negative numbers', 'small positive numbers and small positive numbers', or' big positive numbers and big positive numbers'. If the sign of the upper difference and the lower difference is different, it is determined as 'peak or valley'. In this case, the upper and lower differences are 'large negative and large positive', 'small negative and small positive'. ',' Big positive and big negative ', or' small positive and small negative '.

Subsequently, referring to FIG. 1, the interpolator 140 adaptively obtains a pixel value based on the edge type of the interpolation target pixel, which is a determination result of the edge type determination unit 130. For example, when the determined edge type is 'simple edge', the interpolation unit 140 may be adjacent lines (ie, (j-1) th lines or (j + 1) th lines) located in the main edge direction. One of the pixel values of is taken as the pixel value of the interpolation target pixel. More specifically, in the case of the first case among the cases determined as 'simple edge', interpolation is performed with the pixel value in the main edge direction of the (j + 1) -th line, and is determined as 'simple edge'. In the case of the second case, interpolation is performed with pixel values in the main edge direction of the (j-1) th line. In the case where the determined edge type is 'forged gradient', the interpolator 140 sets the average of pixel values of adjacent lines positioned in the main edge direction as the pixel value of the interpolation target pixel.

When the last determined edge type is 'peak or valley', the interpolation unit 140 is located in the main edge direction (j-3), (j-1), (j + 1), and (j + 3) The pixel values of the interpolation target pixel can be obtained using the pixel values of the first line. For example, using Equation 4, interpolation may be performed using a value of j = 0, that is, a value of F (0). However, in Equation 4, F (-3), F (-1), F (1), and F (3) are the (j-3) th line, (j-1) th line, (( j + 1) th line and (j + 3) th pixel value.

According to an embodiment of the present invention, the upper and lower difference calculation unit 120 calculates the upper difference and the lower difference, and also with the calculated result to determine the edge type in the edge type recognition unit 130, and also the determined type In the interpolation unit 140, adaptively interpolating, a fuzzy rule-based edge-restoration algorithm may be used.

The fuzzy technique is nonlinear and knowledge-based, providing a suitable framework for the new method. 3 is a flowchart showing an image processing algorithm, ie, fuzzy image processing, using a fuzzy technique. Referring to FIG. 3, the fuzzy image processing is largely three stages, that is, the fuzzy process (Fuzzification, Θ), the operation operation on member values (E), and the defuzzy process (Defuzzification, Ψ). Consists of According to this fuzzy image processing, the output x _FER (i, j, k) for the input x (i, j, k) can be expressed by Equation 5. For reference, in FIG. 3, x _L (i, j, k) represents an output when linear interpolation is simply performed according to a conventional interpolation algorithm.

The approach according to an embodiment of the present invention is that any real world image is subjected to separate anti-aliasing lowpass filtering or image acquisition after lowpass filtering and decimation. It is based on the observation that it can be obtained from a high resolution image obtained from the system. And it is assumed that any ideal edge exists. This filtering operation modifies the values of adjacent pixels symmetrically or asymmetrically according to the position of the edge itself in the high resolution image. Thus, after the date, analyzing the pixel values of the low resolution image, the information about the position of the edge can be known at the resolution of the sub-pixel. Therefore, by using the position information of the edge, it is possible to interpolate the pixels of the low resolution image with higher accuracy for the high resolution image. Since the edges present in the low resolution image are generally derived from the steep edges of the high resolution image, the edges present in the low resolution image can be reconstructed into the steep edges. In order to achieve a satisfactory result, it is necessary to select the contour of the object that is important for the human senses, does not increase computation time, and is not affected by noise that may be included in the data.

In the fuzzy rule-based spatial domain linear average algorithm according to an embodiment of the present invention, a fuzzy rule is used to determine whether an interpolated pixel belongs to a strong edge or a type of edge. Use the gradient gradient value. The fuzzy gradient calculator is an example of the upper and lower difference calculator 120 as a means for calculating the fuzzy gradient value. According to an exemplary embodiment of the present invention, the fuzzy gradient value is determined by using pixels located in an upper line or a lower line than pixels included in the spatial window of FIG. 2 within the kth field, which is the same field as the interpolation target pixel. Compute the fuzzy gradients.

4 is a diagram illustrating positions of pixels of a neighbor window used by the fuzzy gradient calculator to obtain a fuzzy gradient value. Referring to FIG. 4, a neighbor window is used for each pixel x ( i , j , k ) (a pixel other than a boundary pixel) of the field image. Each neighboring window for (i, j, k) corresponds to one direction defined in the spatial window of FIG.

And the fuzzy gradients Γ ^U _ED x (i, j, k), C _{SD, ED} x (i, j, k), and Γ ^L _ED x (i, j, k) are as shown in Table 1 , Is defined as the difference between neighboring pixels. Each direction (Each Direction, ED) of the first column in Table 1 represents the relative angle to the vertical direction with respect to the pixel to be interpolated (see FIG. 4). In Table 1, the third column discloses a definition of a basic gradient for each direction and a formula for calculating the same, and the second and fourth columns disclose a definition and a formula of an upper gradient and a lower gradient, respectively. As a result, three fuzzy gradient values are defined for each of the three basic directions.

The three parameters C _{SD, 45} x (i, j, k), C _{SD, 0} x (i, j, k), and C _{SD, -45} x (i, j, k) are included in the base gradient. It is called gradient value. The three parameters Γ ^U ₄₅ x (i, j, k), Γ ^U ₀ x (i, j, k), and Γ ^U _-45 x (i, j, k) included in the upper gradient are the upper gradient values. The three parameters included in the lower gradient Γ ^D ₄₅ x (i, j, k), Γ ^D ₀ x (i, j, k), and Γ ^D _-45 x (i, j, k) are It is called gradient value.

In general, the Bob algorithm (intra-field linear interpolation algorithm) does not show any motion defects and has the advantage of minimizing the amount of computation. However, since the resolution in the vertical direction is 1/2 of the original image before the interpolation of the image, all the details cannot be expressed in the progressive scan image. To solve this problem in the intra-field linear interpolation algorithm, the algorithm according to the present invention uses the upper and lower gradient values in addition to the three basic gradient values to determine the pixel values of the interpolation target pixels. . In addition, one base gradient is not used for each direction, and an embodiment of the present invention uses both an upper gradient and a lower gradient for each direction. However, the upper and lower gradients used herein are in the same direction as the direction of the basic gradient (that is, the main edge direction determined by the edge direction detection unit 110).

Therefore, according to an embodiment of the present invention using a fuzzy rule-based edge reconstruction algorithm, the pixel value of the interpolation target pixel may be equal to the pixel value of the (j-1) th line and the (j + 1) th line immediately adjacent to the interpolation target pixel. The pixel values of the j-3) th line and the (j + 3) th line are also expressed. That is, as shown in FIG. 4, t is the pixel value of the (j-3) th line, u is the pixel value of the (j-1) th line, v is the pixel value of the (j + 1) th line, and When w is assigned to the pixel value of the (j + 3) th line, the pixel value x _FER (i, j, k) of the interpolation target pixel can be expressed using t, u, v, and w (t ∈ {UL, U, UR}, u∈ {ul, u, ur}, v∈ {dl, d, dr}, w∈ {DL, D, DR}).

The fuzzy image processing algorithm uses a membership function to determine the edge type of the interpolation pixel. The membership function is to classify and classify the characteristics of the interpolation target pixels by classifying the relative difference between adjacent pixel values in the main edge direction into several categories.

In accordance with an embodiment of the invention, the membership function is based on the difference in pixel values (eg, luminance) of the top and bottom pixels relative to the center pointer. One example of a purge set that can be used in an embodiment of the invention is shown in FIG. 5. In general, these terms are called fuzzy sets because "big", "small", "negative", and "positive" show only somewhat abstract characteristics in determining pixel values. It can be expressed as The fuzzy set can be represented by a membership function. Referring to FIG. 5, examples of membership functions include BN (fuge set big negative), SN (fuge set small negative), SP (fuge set small positive), and BP ( Fuzzy Set Big Positive. The horizontal axis of these functions represents all possible gradient values [-255, 255], and the vertical axis of these functions represents membership degree (∈ [0,1]). Membership degree indicates the degree to which a particular gradient value matches a predicate (eg, BP). If a particular gradient value has a membership degree of 1 for the fuzzy set BP, then that gradient value is certainly 'big positive'. The parameter set is selected as follows: Negative Big Bound (T _nb ), Negative Small Bound (T _ns ), Small Positive Bound (T _ps ), and Large Positive Bound ( Positve Big Bound, T _pb ). However, it is generally not easy to theoretically find the appropriate values for the parameter sets T _nb , T _ns , T _ps , and T _pb , so the values for these parameter sets can be determined experimentally. According to the simulation results, T _nb = -35, T _ns = -15, T _ps = 15, and T _pb = 35, but the embodiment of the present invention is not limited thereto.

The edge type determination unit 130 may be designed to use a difference between adjacent pixels, for example, a luminance difference. For example, suppose you are doing one-dimensional linear interpolation. FIG. 6 shows the effect of the edges. (A) is the initial data of high resolution, (b) the data after low pass filtering is performed on the data of (a), and (c) The data after decimation of the data in (b) is shown. When four consecutive pixel values are referred to as t, u, v, and w, respectively, output values x by conventional linear interpolation, as shown in Figs. 6D, 6E, and 6F. _L (indicated by ◇) is given by x _L = (u + v) / 2. However, as in the embodiment of the present invention, when the interpolation is adaptively performed according to the edge type, the output value x _FER (denoted by?) Is different from the output value by the conventional linear interpolation.

For example, as shown in (d) of FIG. 6, when the determined edge type corresponds to 'simple edge', the output value x _FER should be similar to one of the pixel value u adjacent to the left or the pixel value v adjacent to the right. do. Therefore, in the embodiment of the present invention, as shown in Fig. _6D , the output value x _FER is u or v. These results show that edges cannot be preserved by conventional linear interpolation algorithms. An interpolation algorithm for preserving edges can be implemented according to the rule represented by Equation 6.

In the case where the determined edge type corresponds to the 'forged slope' as shown in FIG. 6 (e), in order to preserve the forged gradient signal, the output value x _FER is (u + v) / 2 as in the prior art, and linear interpolation is suitable Do. However, when the determined edge type corresponds to 'peak or valley' as shown in Fig. 6 (f), the output value x _FER compensates the peak or valley portion at (u + v) / 2, which is a value obtained by linear interpolation. The compensation parameter λ must be taken into account. To sum up this, it is shown in equation (7).

FIG. 7 is a diagram illustrating differences between a process of obtaining an output value x _L and an output value x _FER according to an embodiment of the present invention using conventional linear interpolation when the edge type is peak or valley. Referring to FIG. 7, in the case of linear interpolation, the output value x _L has the same value as that of the adjacent pixel (ie, u or v). However, according to the embodiment of the present invention, by using four pixels adjacent to the interpolation target pixel in the main edge direction to improve the accuracy of interpolation, the difference as much as λ is compensated even though it is a slight difference. As can be seen with reference to FIG. 7, when the edge type is peak or valley, the interpolation target pixel is closer to the original image by being slightly larger or smaller by λ than the adjacent pixel value.

More specifically, consider a vertical pixel value, such as a brightness change, approximated by a cubic function of j. In FIG. 7, the symbol '○' means a pixel that actually exists. In addition, F (j) = α + βj + γj ² + δj ³ is a cubic function for j. Let F (0) be the value of the interpolation target pixel, and F (-3), F (-1), F (1), and F (3) are sample values of the original field. Here, F (-3), F (-1), F (1), and F (3) are known values and can be obtained by the following four equations. That is, F (-3) = α-3β + 9γ-27δ, F (-1) = α-β + γ-δ, F (1) = α + β + γ + δ, and F (3) = α + 3β + 9γ + 27δ. Using these equations, the output value x _FER (= F (0)) can be obtained using Equation 8 below.

And lambda can be defined by the following equation (9).

The above is summarized in Table 2.

As described above, in the process of deinterlacing an image using a conventional scanning method, by using a fuzzy-rule-based edge reconstruction algorithm as in the embodiment of the present invention, it is possible to realize better image quality than STELA, which is a conventional deinterlacing technique. have. In particular, according to an embodiment of the present invention, when the main edge direction belongs to the spatial domain, the edge types of pixels to be compensated are classified into simple edges, monotonic slopes, or peaks or valleys, and adaptively interpolated according to each type. By performing the above, the edge quality of the virtual vertical full resolution can be obtained.

1 is a block diagram showing the configuration of a deinterlacing apparatus according to an embodiment of the present invention.

2 shows a space-time window used to determine the edge direction.

3 is a flowchart showing fuzzy image processing.

4 is a diagram illustrating positions of pixels of a neighbor window used by the fuzzy gradient calculator to obtain a fuzzy gradient value.

5 is a diagram illustrating a membership function of a fuzzy set used in an embodiment of the present invention.

FIG. 6 shows the effect of the edges. (A) is the initial data of high resolution, (b) the data after low pass filtering is performed on the data of (a), and (c) Data after decimation of the data of (b), (d), (e), and (f) are diagrams showing edge types according to an embodiment of the present invention.

7 is a graph illustrating a process of obtaining lambda according to an embodiment of the present invention.

Claims (14)

A plurality of first pixels included in a previous field image and a next field image in a time direction with respect to a current field image including an interpolation target pixel of an input-parallel scan signal, and a space included in the current field image and spaced in the interpolation target pixel An edge direction detecting unit for determining a detailed direction together with whether the main edge direction of the interpolation target pixel belongs to a time domain or a spatial domain using a plurality of second pixels adjacent to each other in a direction; When it is determined that the main edge direction belongs to the spatial domain, the interpolation is performed using third pixels adjacent to the interpolation target pixel in the spatial direction and upward and downward in the main edge direction, respectively. An upper and lower difference calculation unit for calculating an upper difference and a lower difference for the target pixel; An edge type recognition unit for determining an edge type for the interpolation target pixel using characteristics of the upper difference and the lower difference; And If it is determined that the main edge direction belongs to the time domain, linear interpolation is performed using the first pixel adjacent to the main edge direction, and if it is determined that the main edge direction belongs to the spatial domain, the main edge And an interpolation unit for adaptively performing interpolation according to the edge type determined by the edge type recognizing unit using the second and third pixels adjacent in the direction to output a forward scan signal. The deinterlacing apparatus according to claim 1, wherein the edge direction detecting unit determines the main edge direction using the following formula (E-1).

(E-1) Where C _{TD , 45 °} , C _{TD , 0 °} , C _{TD , -45 °} , C _{SD , 45 °} , C _{SD , 0 °} , and C _{SD , -45 °} are each ┃pl-nr┃, ┃p n┃, ┃pr-nl┃, ┃ul-dr┃, ┃u-d┃, and ┃ur-dl┃, 'pl' is the value of the pixel adjacent to the location of the interpolation target pixel in the previous frame and 'nr' is the value of the pixel adjacent to the position of the interpolation target pixel in the next frame. The value of the pixel, 'p' is the value of the pixel located at the position (Previous) of the interpolation target pixel in the previous frame, 'n' is the value of the pixel located at the position (Next) of the pixel to be interpolated in the next frame, 'pr 'Is the value of the pixel adjacent to the right and right for the position of the interpolation target pixel in the previous frame, and' nl 'is the value of the pixel adjacent to the left and right with respect to the position of the interpolation target pixel in the next frame. 'Ul' is the value of the pixel adjacent to the upper and left side of the interpolation target pixel, 'dr' is the value of the pixel adjacent to the down and right side of the interpolation target pixel, and 'u' is the value of the interpolation The value of the pixel adjacent to the up side of the target pixel, 'd Is the value of the pixel adjacent to the down side of the interpolation target pixel, 'ur' is the value of the pixel adjacent to the upper and right side of the interpolation target pixel, and 'dl' is the lower left of the pixel to be interpolated. and Left) value of the adjacent pixel, Φ is a parameter indicating whether it is a time domain (TD) or a spatial domain (SD), and θ is a parameter indicating an angle with respect to a time direction or a vertical space direction. The de-interlacing apparatus according to claim 2, wherein when the? Corresponds to a time domain (TD), the interpolation unit performs interpolation using the following equation (E-2).

(E-2) Here, x _FER (i, j, k) indicates a pixel value of the interpolation target pixel located at the intersection of the i th column line and the j th horizontal line in the k th field image. The de-interlacing apparatus according to claim 2, wherein the upper and lower difference calculation units calculate an upper difference and a lower difference in the main edge direction by using the equation disclosed in the following table (T-1).

(T-1) Here, each direction ED indicates a relative angle with respect to the vertical direction, C _{SD, 45} x (i, j, k), C _{SD, 0} x (i, j, k), and C _{SD, -45} x (i, j, k) are the basic gradients, Γ ^U ₄₅ x (i , j, k), Γ ^U ₀ x (i, j, k), and Γ ^U _-45 x (i, j, k) are Upper Gradient, Γ ^D ₄₅ x (i, j, k) , Γ ^D ₀ x (i, j, k), and Γ ^D _-45 x (i, j, k) indicate the Lower Gradient, 'ul' and 'UL' are pixels adjacent to the upper and left sides of the interpolation target pixel, and the values of adjacent pixels, 'dr' and 'DR' are respectively adjacent to the lower and right sides of the interpolation target pixel. The values of pixels and adjacent pixels, 'u' and 'U', respectively, are upwards of the interpolation target pixel, and the values of adjacent pixels and adjacent pixels, 'd' and 'D' are respectively below the interpolation target pixel ( Pixels adjacent to Down), 'ur' and 'UR' are adjacent to the upper and right pixels of the interpolation target pixel, and 'dl' and 'DL' are adjacent The pixels adjacent to the left and left of the interpolation target pixel and values of the pixels adjacent thereto are shown. The method of claim 4, wherein the edge type recognizer determines whether the upper difference and the lower difference are positive or negative, or an absolute value of each of the upper difference and the lower difference is predetermined. And an edge type of the interpolation target pixel is determined by determining whether it is larger (Big, B) or smaller (Small, S). The method of claim 5, wherein the edge type of the pixel to be interpolated is one of 'simple edge', 'forged slope', and 'peak or valley', (The upper difference, the lower difference) is (BN, SN), (SN, SP), (BP, SN), (BP, SP), (SN, BN), (SN, BP), (SP, BN ), Or (SP, BP), the edge type recognition unit determines the edge type of the interpolation target pixel as 'simple edge', When (the upper difference, the lower difference) corresponds to (BN, BN), (SN, SN), (SP, SP), or (BP, BP), the edge type recognizing unit of the interpolation target pixel Determine the edge type as 'Forged Slope', When (the upper difference, the lower difference) corresponds to (BN, BP), (SN, SP), (BP, BN), or (SP, SN), the edge type recognition unit of the interpolation target pixel A deinterlacing device, characterized in that the edge type is determined as 'peak or valley'. Here, 'BN' indicates that the upper or lower difference is negative and its absolute value is greater than the threshold, and 'SN' indicates that the upper or lower difference is negative and its absolute value is less than the threshold. 'SP' indicates that the upper difference or lower difference is positive and its absolute value is less than the threshold, and 'BP' indicates that the upper difference or lower difference is positive and its absolute value is greater than the threshold. Point. 7. The deinterlacing apparatus according to claim 6, wherein the interpolation unit performs interpolation using Equation (E-3) or (E-4) when Φ corresponds to the spatial domain SD.

(E-3)

(E-4) Here, x _FER (i, j, k) indicates a pixel value of the interpolation target pixel located at the intersection of the i th column line and the j th horizontal line in the k th field image, 't' is the value of the pixel located in the main edge direction among UL, U, UR, 'u' is the value of the pixel located in the main edge direction among ul, u, ur, and 'v' is dl, d, The value of the pixel located in the main edge direction among dr, and 'w' indicates the value of the pixel located in the main edge direction among DL, D, DR, λ represents 0.0625 × (−t + u + v−w). A plurality of first pixels included in a previous field image and a next field image in a time direction with respect to a current field image including an interpolation target pixel of an input-parallel scan signal, and a space included in the current field image and spaced in the interpolation target pixel Determining a detailed direction along with whether the main edge direction for the interpolation target pixel belongs to a time domain or a spatial domain using a plurality of second pixels adjacent in the direction; When it is determined that the main edge direction belongs to the spatial domain, the interpolation is performed using third pixels adjacent to the interpolation target pixel in the spatial direction and upward and downward in the main edge direction, respectively. Calculating an upper difference and a lower difference for the target pixel; Determining an edge type for the interpolation target pixel using characteristics of the upper difference and the lower difference; And If it is determined that the main edge direction belongs to the time domain, linear interpolation is performed using the first pixel adjacent to the main edge direction, and if it is determined that the main edge direction belongs to the spatial domain, the main edge And interpolating adaptively according to the edge type determined by the edge type recognition unit using the second and third pixels adjacent to each other in a direction to output a progressive scan signal. 9. The deinterlacing method according to claim 8, wherein in the step of determining the direction, the main edge direction is determined using the following formula (E-5).

(E-5) Where C _{TD, 45 °} , C _{TD, 0 °} , C _{TD, -45 °} , C _{SD, 45 °} , C _{SD, 0 °} , and C _{SD, -45 °} are respectively ┃pl-nr┃, ┃p n┃, ┃pr-nl┃, ┃ul-dr┃, ┃u-d┃, and ┃ur-dl┃, 'pl' is the value of the pixel adjacent to the location of the interpolation target pixel in the previous frame and 'nr' is the value of the pixel adjacent to the position of the interpolation target pixel in the next frame. The value of the pixel, 'p' is the value of the pixel located at the position (Previous) of the interpolation target pixel in the previous frame, 'n' is the value of the pixel located at the position (Next) of the pixel to be interpolated in the next frame, 'pr 'Is the value of the pixel adjacent to the right and right for the position of the interpolation target pixel in the previous frame, and' nl 'is the value of the pixel adjacent to the left and right with respect to the position of the interpolation target pixel in the next frame. 'Ul' is the value of the pixel adjacent to the upper and left side of the interpolation target pixel, 'dr' is the value of the pixel adjacent to the down and right side of the interpolation target pixel, and 'u' is the value of the interpolation The value of the pixel adjacent to the up side of the target pixel, 'd Is the value of the pixel adjacent to the down side of the interpolation target pixel, 'ur' is the value of the pixel adjacent to the upper and right side of the interpolation target pixel, and 'dl' is the lower left of the pixel to be interpolated. and Left) value of the adjacent pixel, Φ is a parameter indicating whether it is a time domain (TD) or a spatial domain (SD), and θ is a parameter indicating an angle with respect to a time direction or a vertical space direction. 10. The method of claim 9, wherein when Φ corresponds to the time domain TD, interpolation is performed using Equation (E-6) in the step of outputting a forward scan signal by performing the interpolation. De-interlacing method.

(E-6) Here, x _FER (i, j, k) indicates a pixel value of the interpolation target pixel located at the intersection of the i th column line and the j th horizontal line in the k th field image. The deinterlacing method according to claim 10, wherein in the calculating of the upper difference and the lower difference, the upper difference and the lower difference are calculated in the main edge direction by using the formula disclosed in Table (T-2).

(T-2) Here, each direction ED indicates a relative angle with respect to the vertical direction, C _{SD, 45} x (i, j, k), C _{SD, 0} x (i, j, k), and C _{SD, -45} x (i, j, k) are the basic gradients, Γ ^U ₄₅ x (i , j, k), Γ ^U ₀ x (i, j, k), and Γ ^U _-45 x (i, j, k) are Upper Gradient, Γ ^D ₄₅ x (i, j, k) , Γ ^D ₀ x (i, j, k), and Γ ^D _-45 x (i, j, k) indicate the Lower Gradient, 'ul' and 'UL' are pixels adjacent to the upper and left sides of the interpolation target pixel, and the values of adjacent pixels, 'dr' and 'DR' are respectively adjacent to the lower and right sides of the interpolation target pixel. The values of pixels and adjacent pixels, 'u' and 'U', respectively, are upwards of the interpolation target pixel, and the values of adjacent pixels and adjacent pixels, 'd' and 'D' are respectively below the interpolation target pixel ( Pixels adjacent to Down), 'ur' and 'UR' are adjacent to the upper and right pixels of the interpolation target pixel, and 'dl' and 'DL' are adjacent The pixels adjacent to the left and left of the interpolation target pixel and values of the pixels adjacent thereto are shown. 12. The method of claim 11, wherein in the edge type recognition step, it is determined whether the upper difference and the lower difference are positive (P) or negative (N), or the absolute value of each of the upper difference and the lower difference is And an edge type of the interpolation target pixel is determined by determining whether it is larger than a predetermined threshold (Big, B) or smaller (Small, S). The method of claim 12, wherein the edge type of the pixel to be interpolated is one of 'simple edge', 'forged slope', and 'peak or valley', (The upper difference, the lower difference) is (BN, SN), (SN, SP), (BP, SN), (BP, SP), (SN, BN), (SN, BP), (SP, BN ), Or (SP, BP), the edge type recognition unit determines the edge type of the interpolation target pixel as 'simple edge', When (the upper difference, the lower difference) corresponds to (BN, BN), (SN, SN), (SP, SP), or (BP, BP), the edge type recognizing unit of the interpolation target pixel Determine the edge type as 'Forged Slope', When (the upper difference, the lower difference) corresponds to (BN, BP), (SN, SP), (BP, BN), or (SP, SN), the edge type recognition unit of the interpolation target pixel A method of deinterlacing, characterized in that determining the edge type to 'peak or valley'. Here, 'BN' indicates that the upper difference or lower difference is negative and its absolute value is greater than the upper threshold, and 'SN' indicates that the upper difference or lower difference is negative and its absolute value is smaller than the threshold. 'SP' indicates that the upper or lower difference is positive and its absolute value is less than the threshold, and 'BP' indicates that the upper or lower difference is positive and its absolute value is greater than the threshold. . The method of claim 13, wherein when Φ corresponds to the spatial domain SD, outputting a forward scanning signal by performing the interpolation may be performed using the following equation (E-7) or (E-8). Deinterlacing apparatus, characterized in that for performing.

(E-7)

(E-8) Here, x _FER (i, j, k) indicates a pixel value of the interpolation target pixel located at the intersection of the i th column line and the j th horizontal line in the k th field image, 't' is the value of the pixel located in the main edge direction among UL, U, UR, 'u' is the value of the pixel located in the main edge direction among ul, u, ur, and 'v' is dl, d, The value of the pixel located in the main edge direction among dr, and 'w' indicates the value of the pixel located in the main edge direction among DL, D, DR, λ represents 0.0625 × (−t + u + v−w).

Images