KR101513395B1 - A motion adaptive deinterlacing system - Google Patents

Abstract

The present invention relates to a motion adaptive de-interlacing system, which is applied with a motion detection capable of reducing failures of motion detection and considers weight values and motion sensitivity. The motion adaptive de-interlacing system according to the present invention comprises: a motion detection unit detecting a motion type of an object based on the pixels between the input image fields, and detecting a low motion of an object based on block.; a weight determination unit determining a weight, which is an interpolation coefficient, based on the detection result by the motion detection unit, and selecting an interpolation method according to the weight; and, an interpolation execution unit interpolating the missing pixel according to the interpolation method determined in the weight determination unit.

Description

[0001] The present invention relates to a motion adaptive deinterlacing system,

The present invention relates to a deinterlacing technique for converting an interlaced scan format into a progressive scan format, and more particularly, to a motion adaptive deinterlacing system that applies motion detection to reduce motion detection failure and considers weighting and motion sensitivity. will be.

Typical TV systems, such as video broadcast, PAL, SECAM, and NTSC, employ an interlaced format to halve the bandwidth of recent video transmission. This is because the reference frame rate is clear for slow motion distribution. However, the conventional interlace format has a problem that noise including interline flicker, jaggedness, and line crawling occurs for an object having a high horizontal frequency. In addition, the development of modern display systems such as ultra-high definition television (HDTV), PC monitor, LCD, PDP and the like are required to display the whole image immediately.

Deinterlacing technology is a conversion technique that converts an interlaced scan format to a progressive scan format. Conventional deinterlacing methods are classified into four methods such as an intra-field, an inter-field, a motion adaptive, and a motion compensated method. The intra-field method uses pixels of the current field to fill a lost line, which is suitable for video sequence work with a low spatial frequency, but generates flickering artifacts (noise) in a high frequency region.

The interfield method is used for a high correlation between adjacent fields for improving video quality. However, the interfield method provides a high quality in a still region but provides a noise in a moving region.

The motion adaptation (MA) method combines the advantages of the intra-field method and the inter-field method, and is the most popular de-interlacing method due to its low complexity and high video quality. It is disclosed in Korean Patent Registration No. 10-0813986 (Mar. As disclosed, after sensing the motion region, spatial and temporal interpolation is applied depending on whether motion is present or not. This motion adaptation method forms a system as shown in Fig. 14, and the interpolation step is performed by the following equation (1).

Here, PM is the motion likelihood values, (i, j) represents the pixel location, n is the current field, X _n (i, j) denotes a pixel value at position (i, j), D _n (i, j) represents the last interpolated pixel at the same position.

The Motion Compensation (MC) method calculates block or object motion within adjacent fields. The MC method achieves better performance than other methods, but has the problem of requiring high computational cost.

Korean Registered Patent Publication No. 10-0553814B1, 2006. 02. 14, 7 page 27 lines to 29 lines.

It is an object of the present invention to provide a motion adaptive de-interlacing system and a motion-adaptive de-interlacing method, De-interlacing system.

Another object is to accurately detect the motion in the low motion by calculating the motion sensitivity based on the pixel value, including the motion detection part, and calculating the absolute difference based on the block.

Another object of the present invention is to determine a weight based on the calculated motion sensitivity and the sum of absolute differences using a weight determining unit, and to select an appropriate interpolation method based on the determined weight.

Another object is to re-determine whether or not motion by applying an absolute difference sum with respect to a low motion sensitivity area, further including an exception determination unit.

Still another object is to execute interpolation including spatial interpolation, temporal interpolation and spatial interpolation and temporal interpolation according to the calculated weight including the interpolation executing part.

According to an aspect of the present invention, there is provided a motion adaptive deinterlacing system including a motion detection unit detecting a motion type of an object based on pixels between input image fields and detecting a low motion of the object based on a field block, And an interpolation execution unit for interpolating the lost pixels according to the interpolation method determined by the weight determining unit. The interpolation method may further include interpolating the interpolated pixel according to the interpolation method.

In addition, in the motion adaptive deinterlacing system according to the present invention, the motion detection unit may include a pixel-based sensing unit for calculating pixel values between image fields, calculating motion sensitivity (MI) with the calculated pixel values, And a block-based sensing unit for calculating a sum of block-based absolute differences (SAD) for sensing.

Further, in the motion adaptive deinterlacing system according to the present invention, when the motion sensitivity is smaller than 1 (? ₁ ), the interpolation selector is regarded as a still area or stopped scene of the video, Interpolation) is selected, and intra-field interpolation is selected when the motion sensitivity is greater than 2 (τ ₂ ), and the spatial and temporal interpolation is selected when the motion sensitivity is between τ ₁ and τ ₂ .

In the motion adaptive deinterlacing system according to the present invention, when the weight is 0, the weight determining unit determines the sum of the absolute differences (SAD) by considering the sum of three absolute differences (SAD) ) Is greater than the third threshold value (? ₃ ), an exception determination unit ( ₁₃ ) for correcting the weight value to 1 is further included.

As described above, the motion adaptive deinterlacing system according to the present invention has an effect of greatly improving the deinterlacing performance by applying the improved motion detection technique based on pixels and blocks.

In addition, the motion sensitivity of the object can be determined by calculating the motion sensitivity based on the pixel value, and the absolute difference can be calculated based on the block, thereby accurately detecting the motion in the low motion of the object.

Further, by determining the weight based on the calculated motion sensitivity and the sum of the absolute differences, it is possible to select the optimum interpolation method according to the determined weight value.

In addition, it is possible to prevent the selection error of the interpolation method in advance by checking whether motion is applied by applying the sum of absolute differences in the low motion sensitivity region.

In addition, spatial interpolation, temporal interpolation, and temporal interpolation combined with the temporal interpolation are performed according to the calculated weight values, so that the advantage of each interpolation method can be maximally applied.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the entire configuration of a motion adaptive deinterlacing system according to the present invention; FIG.
2 is a block diagram showing a detailed configuration of a motion detection unit in a motion adaptive deinterlacing system according to the present invention;
3 is a pixel layout diagram for explaining a general motion detection concept in a motion adaptive deinterlacing system according to the present invention;
4 is a conceptual diagram of pixel-based motion detection using a FIR filter in a motion-adaptive deinterlacing system according to the present invention.
5 is a conceptual diagram of block-based motion detection using SAD in a motion-adaptive deinterlacing system according to the present invention.
6 is a configuration diagram showing a detailed configuration of a weight determining unit in a motion adaptive deinterlacing system according to the present invention;
7 is a diagram illustrating a concept of applying a weight in a motion adaptive deinterlacing system according to the present invention.
8 is a diagram showing a concept of a direction vector of the MELA algorithm in a motion adaptive deinterlacing system according to the present invention.
9 to 13 are graphs showing test results for checking the performance of the motion adaptive deinterlacing system according to the present invention.
14 shows a configuration of a general motion adaptive interlacing system;

Hereinafter, a detailed description will be given of the motion adaptive deinterlacing system according to the present invention.

FIG. 1 is a block diagram of a motion adaptive deinterlacing system according to the present invention. The motion adaptive deinterlacing system includes a motion detection unit 10, a weight determination unit 20, and an interpolation execution unit 30.

The motion detection unit 10 senses the motion type of the object based on the pixels between input image fields, detects the low motion of the object based on the field block, and detects the motion of the object based on the pixel-based sensing unit 11 And a block-based sensing unit 12.

The pixel-based sensing unit 11 calculates a pixel value between image fields, and calculates a motion sensitivity (MI) using the calculated pixel value. In the present invention, the pixel-based motion detection is based on two or more fields, Acquiring information, detecting a motion based on a pixel, and calculating an adjacent pixel value based on a pixel to be interpolated, the pixel value applying a finite impulse response (FIR) filter.

That is, in the case of general pixel-based motion detection, the motion sensitivity (MI-Motion Intensity) of each pixel is calculated by applying the following equation 2 with respect to the 5-field pixel shown in FIG. The maximum value of the calculated motion sensitivity is defined as the final motion sensitivity.

In this case, there arises a problem that the moving object can not be accurately detected due to the noise inside the object. In the pixel-based sensing unit 11 according to the present invention, as shown in FIG. 3, And each pixel value is calculated using Equation (4), each motion sensitivity is defined by Equation (5), and the maximum value of each motion sensitivity is defined as a final motion sensitivity .

Where n is the current field and X is the interpolated pixel and the pair of (n-2, n), (n-1, n + 1) and (n, n + 2) , And A to H are pixel values in the field.

The block-based sensing unit 12 calculates a block-based absolute difference sum (SAD) as shown in FIG. 5 to detect a low motion of a moving object, and the block- The sum of absolute differences SAD is calculated in the fields (n-1, n), (n, n + 1), and (n-1, n + 1) using the following equation (7).

The weight determining unit 20 determines a weight, which is an interpolation coefficient, according to the detection result in the motion detection unit 10, selects an interpolation method according to the weight, and, as shown in FIG. 6, (21) and an exception determination unit (22).

The interpolation selector 21 calculates the weight W for determining the ratio at the time of mixed interpolation using the sum of the motion sensitivities and the absolute differences (SAD) calculated in the motion detection unit, using the following equation (8) 7, when the motion sensitivity is smaller than 1 (τ ₁ ), the corresponding pixel is regarded as a still region or stopped scene of the video, so that inter-field interpolation (temporal interpolation) is selected, When the motion sensitivity is greater than 2 (τ ₂ ), intra-field interpolation is selected, and when the motion sensitivity is between τ ₁ and τ ₂ , the spatial and temporal interpolation are jointly selected.

If the sum of the absolute differences (SAD) is greater than or equal to the third threshold value τ ₃ (SAD) in consideration of the sum (SAD) of the three absolute differences calculated by the block- ), The weight is set to be 1.

That is, when the weight value calculated by the interpolation selector 21 is 0, it is regarded that there is no motion pixel. However, some of the pixels that are regarded as static regions due to low MI can actually have small movements.

The interpolation executing unit 30 interpolates the lost pixels according to the interpolation method determined by the weight determining unit 20 and the interpolation executing unit 30 according to the present invention performs spatial interpolation, temporal interpolation, interpolation, and interpolation in which the spatial interpolation and the temporal interpolation are combined,

The spatial interpolation (intra-field interpolation) applies a modified edge-based line average algorithm (MELA-modified edge-based line average) (9), and the pixel correlation of the edge-based line average (ELA) is calculated by the following equation (10): " (9) " Using the direction vector and the pixel correlation, spatial interpolation is performed using the following equation (11).

Where P is a diagonal, Q is an inverse diagonal, V is a vector in the vertical direction, and the operator ∧ is an AND operator.

In the present invention, temporal interpolation (inter-field interpolation) uses a simple algorithm for the previous (n-1) and later (n + 1) pixels at the same position. That is, there is a high correlation between the pixel values of adjacent fields if the pixel is stationary motion or background, which can lead to anomalous execution if motion detection is performed correctly and using only such simple temporal interpolation. Therefore, in the present invention, temporal interpolation performs temporal interpolation using the following equation (12).

Experimental contents and results for evaluating the performance of the motion adaptive deinterlacing system according to the present invention are as follows.

In order to evaluate the performance of the motion adaptive deinterlacing system according to the present invention, experiments were performed with various reference CIF video sequences including dynamic video in low motion. The reference values were τ ₁ = 20, τ ₂ = 70, τ ₃ = 100 < / RTI > 9 (a) and 9 (b) show the results when the interpolation selector 21 and the exception detector 22 are applied. In FIG. 9 (a), the ball is moving down, and the ball exists in the (n-1) and (n + 1) fields but does not exist in the fields of (n-2), (n) and Do not. If there is only one moving object, pixel-based motion description can often mis-detect motion.

Figure 10 shows one of the worst cases. The letters move from right to left, and almost every pixel is recognized as a background. Therefore, I chose a temporal filter that should be a lot of movement. In order to improve such a situation, the exception determination unit 22 is applied, and the results of the motion direction shown in Figs. 9 (b) and 10 (b) are shown.

Table 1 shows PSNR comparisons and subjective quality comparisons of FIGS. 11, 12 and 13. It can be seen that the deinterlacing method according to the present invention has better performance than the well-known LA, ELA, MELA, vertical temporal filter method and motion adaptive method. In addition, the proposed method increases the PSNR by about 0.95dB. Particularly subjective performance shows better results in low motion video than high motion video like Akiyo, Mother & daughter, and container. This part is representative in the low motion area. It can be seen that while the other methods are blurred, the proposed method shows a clean calibration result.

[Table 1]

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but may be implemented by various motion adaptive deinterlacing systems within the scope of the present invention.

10: Motion detection part
11: Pixel-based sensing unit
12: Block-based sensing unit
20: weight determining unit
21: Interpolation selector
22:
30: interpolation execution unit
50: Motion adaptive deinterlacing system

Claims (10)

1. A deinterlacing system for converting an interlaced scan format to a progressive scan format,
A motion detection unit detecting a motion type of an object based on pixels between input image fields and detecting a low motion of the object based on a field block;
A weight determining unit for determining a weight, which is an interpolation coefficient, according to the detection result in the motion detection unit, and selecting an interpolation method according to the weight;
And an interpolation executing unit for interpolating the lost pixels according to the interpolation method determined by the weight determining unit,
Wherein the motion detection unit comprises:
A pixel-based sensing unit for calculating pixel values between image fields and calculating motion sensitivity (MI) with the calculated pixel values;
And a block-based sensing unit for calculating a sum of block-based absolute differences (SAD) to detect a low motion of the moving object.
delete The method according to claim 1,
Wherein the pixel-
A motion estimation method, comprising: obtaining motion information based on at least two fields, detecting motion based on pixels, calculating adjacent pixel values based on interpolated pixels, and applying the finite impulse response (FIR) (4). ≪ / RTI >
&Quot; (4) "

Where n is the current field and X is the interpolated pixel and the pair of (n-2, n), (n-1, n + 1) and (n, n + 2) , And A to H are pixel values in the field.
The method of claim 3,
Wherein the pixel-
The motion sensitivity of each pixel is calculated by the following equation (5) by applying the calculated pixel values, and the final motion sensitivity (MI) is calculated to be the largest of the motion sensitivities of the pixels Adaptive deinterlacing system.
&Quot; (5) "

5. The method of claim 4,
Wherein the block-
The sum of absolute differences (SAD) is calculated in the fields of (n-1, n), (n, n + 1) and (n-1, n + 1) The motion adaptive de-interlacing system comprising:
&Quot; (7) "

6. The method of claim 5,
The weight determining unit
And an interpolation selector for calculating a weight W for determining the ratio at the time of mixed interpolation using the sum of the motion sensitivity and the absolute difference (SAD) calculated in the motion detection unit using the following equation (8) The motion adaptive de-interlacing system comprising:
&Quot; (8) "

Where τ ₁ and τ ₂ are empirically set values.
The method according to claim 6,
Wherein the interpolation selection unit comprises:
If the motion sensitivity is less than 1 (τ _1), the pixel is regarded as a still scene or a still region of the video inter-than field interpolation (time interpolated) the selection, and the motion is 2 (τ ₂₎ Sensitivity And when the motion sensitivity is between τ ₁ and τ ₂ , the spatial and temporal interpolation is selected in combination.
The method according to claim 6,
Wherein the weight determining unit includes:
When the sum of absolute differences (SAD) is larger than the third threshold value (? ₃ ) in consideration of the sum of three absolute differences (SAD) calculated by the block-based sensing unit when the weight is 0, And an exception determination unit configured to correct the motion adaptive deinterlacing to the motion adaptive deinterlacing system.
The method according to claim 1,
The interpolation execution unit
Performing spatial interpolation, temporal interpolation, and interpolation in which the spatial interpolation and the temporal interpolation are combined,
The spatial interpolation
Applying the modified edge-based line average (MELA-modified line average), the three directional vectors in the vertical (90 °), diagonal (63 °) and reverse diagonal (117 °) (9), and calculates the pixel correlation of the edge-based line average (ELA) using the following equation (10), and using the direction vector and the pixel correlation, The motion adaptive deinterlacing system comprising:
&Quot; (9) "

&Quot; (10) "
C _-1 = | U _-1 -L ₁ |,
C ₀ = | U ₀ -L ₀ |,
C ₁ = | U ₁ -L _-1 |
&Quot; (11) "

Where P is a diagonal, Q is an inverse diagonal, V is a vector in the vertical direction, and the operator ∧ is an AND operator.
10. The method of claim 9,
The interpolation execution unit
Wherein the temporal interpolation is performed using the following equation (12).
&Quot; (12) "

Images