KR100471186B1 - Deinterlacing apparatus and method for interpolating of image data by region based - Google Patents

Abstract

The present invention relates to a deinterlacing apparatus and method for interpolating image data on an area basis. The deinterlacing apparatus of the present invention generates the divided image data by smoothing the input image data, using the gradient estimation, and the watershed algorithm. The spatial direction and temporal interpolation are processed by detecting the direction of the edge using the correlation of the divided regions. As a result, the distortion is less than that of the interpolation method using the directional detection of the edge by the difference between pixels, and the interpolation performance is particularly excellent in a severely curved image.

Description

Deinterlacing apparatus and method for interpolating image data based on region {DEINTERLACING APPARATUS AND METHOD FOR INTERPOLATING OF IMAGE DATA BY REGION BASED}

The present invention relates to a deinterlacing apparatus for interpolating image data and a method thereof, and more particularly, to a deinterlacing apparatus for interpolating image data by examining correlation between pixels on an area basis and a processing method thereof.

De-interlacing is a technique of converting interlaced scan image data into progressive scan image data in an image data processing method. In interlaced scanning image data, one frame screen is composed of two field screens, and a separate deinterlacing apparatus is required to deinterlace it.

Prior arts relating to such deinterlacing apparatus and methods are described, for example, in 'Deinterlacing apparatus and method' of Korean Patent Laid-Open Publication No. 2001-068516 (published: 2001.7.23), and Korean Patent Publication No. 2002-026042. (Publishing date: 2002.4.6) 'Deinterlacing apparatus and method using motion compensation type interpolation' and 'Motion compensation conversion circuit of image signal' of JP-A-12-261768 (published date: September 22, 2000) Etc. are disclosed.

Korean Patent Laid-Open Publication No. 2001-068516 discloses an edge orientation detection method using a difference value between pixels, a median filtering method for noise removal, and a soft switch output method of temporal interpolation and spatial interpolation in a space-time deinterlacing system. have. According to such a deinterlacing apparatus and method, an image quality is excellent by additionally performing median filtering after interpolation.

Korean Patent Laid-Open Publication No. 2002-026042 discloses an interpolation method for compensating a motion by using a bidirectional motion vector and median filtering, and is easy to implement and has excellent outline preservation ability.

Japanese Laid-Open Patent Publication No. 12-261768 discloses a method of detecting a motion vector to compensate for motion, and then obtaining a new interpolation value by weighting time, space, and space-time interpolation values according to error values. According to this, deterioration of the inherent image quality with respect to the motion compensation process can be eliminated.

However, the above-described techniques have a problem of interpolation in the direction of an edge obtained by using a difference value between pixels, so that the edge direction is incorrectly obtained and the interpolation effect is deteriorated in an image having a high distortion.

References to conventional deinterlacing techniques are also disclosed below.

References 1. H.Y. Lee, J.W. Park, T.M. Bae, S.U. Choi, and Y.H. Ha, "Adaptive scan rate up-conversion system based on human visual characteristics," IEEE Consumer Electronics, vol. 46, no. 4, nov. 2000.

References 2. R. Li, B. Zeng, and M. L. Liou, "Reliable motion detection / compensation for interlaced sequences and its applications to deinterlacing," IEEE Circuits Syst. Video Technology, vol. 10, no. 1, pp. 23-29, 2000.

Therefore, in the deinterlacing apparatus, various methods are applied to interpolate image data, for example, when deinterlacing or enlarging an image in a vertical direction. For example, according to Reference 1, a new pixel is generated using the average value of these pixels by using two pixels in the vertical direction of the vertical line of the pixel to be generated, or several vertical lines are generated in the upper and lower lines of the pixel to be generated. The pixels are used to interpolate new pixels using a filter such as polyphase. However, the interpolation method according to the reference 1 is simple to implement the technology, but there is a problem of damaging the edge (edge).

Alternatively, according to Reference 2, a correlation is investigated by using an absolute value of the difference between pixels of the upper and lower lines of the pixel to be generated, and new pixels are generated by averaging two pixels in the most correlated direction. . According to this method, it is possible to interpolate while maintaining the edge of the video data. However, this method has the disadvantage of distorting the edges in the image with a lot of curvature because the horizontal edge or the change in pixel value is severe because interpolation is performed while maintaining the edge while calculating the correlation with the difference of pixel values.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problem, and to provide a deinterlacing apparatus for segmenting image data and detecting and interpolating edge directions of pixels to be interpolated using the divided regions.

In addition, another object of the present invention is to solve the above-described problem, and to implement a deinterlacing method for segmenting image data and detecting and interpolating edge directions of pixels to be interpolated using the divided regions.

To this end, the present invention implements a method of interpolating a pixel based on a region, a method of obtaining correlation between pixels to be interpolated based on a region, and a method of interpolating using a region based deinterlacing method in a space-time deinterlacing apparatus.

According to an aspect of the present invention for achieving the above object, a de-interlacing device, a memory unit having a plurality of line memories for storing the pixels of the peripheral pixels and the front and rear fields of the pixel to be interpolated from the image data input from the outside And an area divider and peripheral pixels that receive the peripheral pixels from the memory unit to generate image data having a plurality of divided regions, and detect and output edge directionality of the pixel to be interpolated from the divided image data. And a deinterlacing unit which receives the divided image data and spatially interpolates using the detected edge directionality.

The region dividing unit may include a smoothing unit which receives the peripheral pixels from the memory unit and removes a discontinuous image, and an gradient estimating unit which estimates a slope by receiving image data from which the discontinuous image is removed from the peace unit; And a directional detection unit corresponding to the estimated gradient and dividing the input image data of the neighboring pixels into a plurality of areas by using a fractional algorithm, and a directional detector detecting the directionality of the edges of the divided areas.

The deinterlacing unit receives pixels in the front and back fields from the memory unit and interpolates temporally, and outputs the spatiotemporal interpolated image data by combining the spatial and temporal interpolated image data. In this case, the de-interlacing unit may combine the spatial and temporal interpolated image data by applying a specific weight corresponding to the presence or absence of motion.

According to another aspect of the present invention for achieving the above object, there is provided a memory unit for receiving image data input from the outside, an area divider for dividing the image data into a plurality of areas, and a space and the divided image data. A deinterlacing apparatus comprising a deinterlacing unit for temporally interpolating, wherein a deinterlacing method for interpolating the image data of the deinterlacing apparatus comprises: image data including peripheral pixels of pixels to be interpolated from the memory unit and pixels in a front and back field Inputting the pixels to smooth the neighboring pixels, estimating the inclination, and generating divided image data having a plurality of regions; and between the pixel to be interpolated and the peripheral pixels for each region from the divided image data. Detecting edge directionality, said sword Interpolating spatially in the most correlated direction among the obtained directionalities, interpolating temporally by detecting and estimating the degree of motion by receiving the pixels in the front and rear fields, and corresponding the degree of motion of the spatially and temporally interpolated image data. Interpolating in time and space, filtering using the space-time interpolated interpolation value and two pixels in the vertical direction of the pixel to be interpolated, and generating the space-time interpolated image data.

Therefore, according to the present invention, the image data is divided into a plurality of regions, and the directionality of the edges is detected from the divided image data of the divided regions and spatially interpolated.

DETAILED DESCRIPTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a detailed configuration of an area-based space-time deinterlacing apparatus according to the present invention.

Referring to the drawings, the space-time deinterlacing apparatus 100 includes a novel area divider 140. And a memory unit 110 including a plurality of line memories 112 to 122, and a deinterlacing unit 160 interpolating spatially and temporally.

The memory unit 110 includes, for example, six line memories, line memories 112 to 118 for storing peripheral pixels of the interpolated pixel, and line memories 120 to store front and rear field pixels. 122).

The region dividing unit 140 receives the surrounding pixels to generate a plurality of divided image data, a smoothing part 142, a gradient estimating unit 144, and a watershed algorithm. A processing unit 146 and a directionality detection unit 148.

The smoothing unit 142 is provided with, for example, an average, morphology or median filter to smooth the input discontinuity of the image data.

The inclination estimator 144 is provided with, for example, a sobel, which is a differential operator for contour detection, a robert, a cubic, or a morphology, and estimates the inclination of the smoothed image data. do.

The fractional processor 146 generates image data having divided regions using a fractional algorithm. References to the watershed algorithm are disclosed below, and since the watershed algorithm is well known to those skilled in the art, a detailed description thereof will be omitted.

References 3. L. Vincent, "Morphological grayscale reconstruction in image analysis: applications and efficient algorithms," IEEE Image Processing, vol. 2, no 2, pp. 176-201, April, 1993.

References 4. M.C. Kim, J.G. Choi, D.H. Kim, H. Lee, M. H. Lee, C.T. Ahn, and Y.S. Ho, "A VOP generation tool: automatic segmentation of moving objects in image sequences based on spatio-temporal information," IEEE Circuits Syst. Video Technology, vol. 9, no. 8, pp. 1216-1226, 1999.

The directional detector 148 detects the directionality of the edge by comparing the original image data input to the memory unit 110 with the divided image data to calculate the directionality of the pixels to be interpolated and surrounding pixels. At this time, the directional detection checks whether the regions between the pixels are the same in various directions using the upper and lower lines (eg, two or four upper and lower lines) of the pixel to be interpolated. In the embodiment of the present invention, as shown in FIGS. 6A to 6G, the directional levels D1 to D7 are detected in seven directions. In addition, the direction detecting unit 148 includes a counter (not shown) for determining a correlation between directions, and calculates and stores a count value for each direction according to the direction detection. For example, if the same directionality is detected in one area, the count value is added. Specifically, an algorithm for calculating a count value for directional detection using four upper lines is disclosed in Equation 1 below.

...

Where k is a pair of five pixels used to calculate one direction to detect the directionality between the surrounding pixels of the top four lines of the pixel to be interpolated, i, j are the vertical, horizontal position and R of the pixel The number of levels of the divided regions is shown. In addition, C denotes a constant indicating whether or not regions obtained from five pixel pairs in each direction are the same, except that the middle pixel pair is defined as 2 by weighting.

Therefore, after obtaining area information using the divided area data, the direction to be interpolated is detected in each area information. In this case, the direction having the most count value is determined as the direction to interpolate, and when the direction has the same count value as the maximum value in two or more directions, 1, 2, ..., n- Priority is given in the order of 1 and n directions to determine the level D of the direction.

Where D represents the level of the direction to be determined, for example if there are only seven directions, the range of D is determined to be between 1 and 7, in particular 4 represents the vertical direction.

As shown in Equation 3 below, only when the count value according to the direction is at least an appropriate threshold T or more, the direction is recognized as the most correlated direction and interpolation is performed. Otherwise, it is preferable to interpolate in the vertical direction.

Where I _{i-1, j} is Denotes a pixel of the line immediately above the pixel to be interpolated, and the threshold T is experimental data. The criterion of the threshold is, for example, when using four lines for directional determination, experimentally 15 obtains the most desirable result according to the present invention.

The deinterlacing unit 160 further includes a spatial interpolation unit 162 for processing an area-based spatial interpolation method, a time interpolation unit 164 for processing with a general temporal interpolation method, and spatial and temporal interpolation. It includes a space-time interpolation unit 166 to process.

The spatial interpolation unit 162 receives image data divided from the area dividing unit 140 and pixels around the rotor 110 and spatially interpolates in the direction having the highest directional correlation.

The temporal interpolation unit 164 generally includes a motion detection and moving estimation unit (not shown) for interpolating using a motion vector, and uses the upper and lower fields of the front and rear field pixels and the current field using them. Motion detection and motion estimation from the surrounding pixels of the lines.

The spatiotemporal interpolator 166 interpolates the interpolation values output from the spatial and temporal interpolators 162 and 164 by applying appropriate weights and combines them. For example, when there is a lot of motion, a spatial interpolation result is applied with a as 1, and when there is no motion, a interpolation result is applied with a as 0. In other words, as shown in Equation 4 below, the deinterlacing unit 160 applies a weight based on the degree of motion to the interpolation values processed by the spatial and temporal interpolation units 162 and 164 to interpolate the space time.

Here, A is a spatially interpolated value and B is a temporally interpolated value. In detail, B is an average value of the front and rear fields, as shown in Equation 5 below.

As described above, the deinterlacing apparatus 100 of the present invention calculates an area-based spatial deinterlacing pixel value using an interpolated value with directionality and an intermediate value of two pixel values in the vertical direction. The region-based temporal deinterlacing method uses pixels of the front and rear fields and pixels adjacent to the top and bottom of the current field by using the motion detection / motion estimation unit as in the general method. As described above, the space-time interlacing method obtains the pixel information in the space in the above-described region-based space deinterlacing method.

2 is a flowchart illustrating a deinterlacing method of interpolating image data based on an area according to the present invention. This procedure is processed by the deinterlacing apparatus 100.

Referring to the drawing, the deinterlacing apparatus 100 performs a processing routine for generating the image data input to the memory unit 110 in step S200 into image data having a plurality of divided regions in the region dividing unit 140. do. In step S210, the direction to be interpolated is detected, and then in step S220, an interpolation processing routine is performed.

In detail, as illustrated in FIG. 3, the divided image data generation processing routine S200 inputs image data including peripheral pixels of pixels to be interpolated and pixels of a front and back field from the memory unit 110 in step S202. In operation S204, the area divider 140 smoothes the peripheral pixels through the smoother 142. In step S206, the slope is estimated. Subsequently, in step S208, the divided video data is generated and output.

As shown in FIG. 4, the interpolation processing routine S220 determines a direction having the highest correlation among the detected directions for the pixels to be interpolated in step S222, and determines a space and Interpolate in time. In step S226, a motion degree, that is, a motion vector, is analyzed and space-time interpolated in order to combine spatial and temporal interpolated interpolation values. In this case, the space-time interpolation applies weights according to Equation 4 described above.

Subsequently, the median filtering is performed by using the space-time interpolated interpolation value and two pixels in the vertical direction of the pixel to be interpolated. In operation S230, image data interpolated in time and space is generated and output.

5 is a view for explaining the detection of the directionality of the image data based on the divided region according to the present invention.

Referring to the drawing, the image data 170 output by the region dividing unit 140 is divided into a plurality of divided regions 170a to 170c. Peripheral pixels of the pixel 172 to be interpolated in each region detect the directionality of edges with two (174, 176) or four lines among the upper and lower lines, and calculate a correlation to determine the most suitable direction. Interpolate spatially.

6A to 6G are diagrams for describing directionality detection using four lines according to an exemplary embodiment of the present invention.

Referring to the drawing, four upper and lower lines y-3, y-1, y + 1 and y + 3 lines are used to detect the directivity of the pixels 182 to be interpolated with the neighboring pixels. The directional levels D1 to D7 are detected in the seven directions 184 of the region-based edges.

The correlation is determined to interpolate in the direction having the highest count value by adding the corresponding count value if the areas between the pixels in the seven directions or the various directions are equal to each other. At this time, as shown in the above equation, only when the count value is greater than or equal to the appropriate threshold value T, interpolation is performed in the corresponding direction.

And the experimental results according to the embodiment of the present invention are shown in FIG.

Referring to the drawings, (a) is the interpolated image data by the MELA method of the reference 1 (b) and the deinterlacing method according to the present invention of the existing deinterlacing method using the image data of the sequential scanning method Video data (c) is shown.

Therefore, the present invention divides the image data into a plurality of regions by using the watershed algorithm, detects the directionality of the pixels to be interpolated in the divided region and interpolates to each region, thereby maintaining the image while maintaining the edge as shown in the drawing. By preventing distortion, better image data can be obtained.

As described above, the conventional interpolation methods mainly use the method of using the pixel difference, while the present invention implements a method for finding the direction of the edge based on the segmented image in the spatial interpolation method. That is, first, the input image data is divided into a plurality of regions, and the direction of the edge is detected and interpolated using the boundary between each.

As described above, the conventional deinterlacing method interpolates in the edge direction obtained using the difference value between pixels, while the deinterlacing method of the present invention first distinguishes the regions of pixels and uses the correlation between the regions, thereby deinterlacing apparatus and method based on region. To provide. As a result, the distortion caused by incorrectly finding the edge direction is small, and the effect is excellent in the severely curved image.

1 is a block diagram showing a detailed configuration of an area-based spatiotemporal deinterlacing apparatus according to the present invention;

2 is a flowchart showing a deinterlacing procedure for interpolating image data according to the present invention;

FIG. 3 is a flowchart showing the detailed procedure of generating the area-divided image data shown in FIG. 2;

4 is a flowchart showing the detailed procedure of the interpolation step shown in FIG.

5 is a view for explaining the detection of the directionality of the image data based on the divided region according to the present invention;

FIG. 6 is a diagram for explaining directionality detection using four lines according to an embodiment of the present invention; FIG. And

FIG. 7 is a diagram illustrating experimental results of image scanning in a sequential scanning method using a conventional MELA method and an interpolation method of the present invention.

Explanation of symbols on the main parts of the drawings

Deinterlacing apparatus 110: memory unit

112 to 122: line memory 140: area divider

142: smoothing unit 144: slope estimation unit

146: water fountain processing unit 148: directionality detection unit

160: deinterlacing unit 162: spatial interpolation unit

164: space-time interpolation unit 166: time interpolation unit

Claims (6)

In a deinterlacing device: A memory unit including a plurality of line memories for storing peripheral pixels of the pixel to be interpolated from the image data input from the outside and pixels of the front and back fields; A region dividing unit which receives the peripheral pixels from the memory unit to generate image data having a plurality of divided regions, and detects and outputs an edge directionality of the pixel to be interpolated from the divided image data; And a deinterlacing unit which receives the peripheral pixels and the divided image data and interpolates spatially using the detected edge directionality. The method of claim 1, The area divider; A smoothing unit which receives the peripheral pixels from the memory unit and removes the discontinuous image; An inclination estimator for estimating an inclination by receiving image data from which the discontinuous image is removed from the peace unit; A water content processor corresponding to the estimated gradient and dividing the image data of the input neighboring pixels into a plurality of areas using a water order algorithm; And a directional detector configured to detect directionalities of edges of the divided regions, respectively. The method of claim 1, The deinterlacing unit; Receiving pixels in the front and back fields from the memory unit and interpolating in time; And interpolating the spatial and temporally interpolated image data to output the interpolated image data in time and space. The method of claim 3, wherein The deinterlacing unit; And decomposing the spatial and temporally interpolated image data by applying a specific weight corresponding to the presence or absence of motion. A deinterlacing apparatus comprising a memory unit for receiving image data input from an external device, a region dividing unit for dividing the image data into a plurality of regions, and a deinterlacing unit for interpolating the divided image data in a spatial and temporal manner. A deinterlacing method for interpolating the image data of a deinterlacing device, comprising: Receiving image data including peripheral pixels of the pixel to be interpolated and pixels in the front and rear fields from the memory unit, smoothing the peripheral pixels, estimating the inclination, and generating divided image data having a plurality of regions; ; Detecting edge directionality between the pixel to be interpolated and the peripheral pixels for each region from the divided image data; And spatially interpolating in the direction with the highest correlation among the detected directionalities. The method of claim 5, wherein After the spatial interpolation step, receiving pixels in the front and rear fields to detect and estimate a degree of motion to interpolate in time; Interpolating the spatial and temporally interpolated image data in time and space according to the degree of movement; Filtering using the space-time interpolated interpolation value and two pixels in the vertical direction of the pixel to be interpolated; And generating the interpolated image data in space time.