KR20040070487A - Image processing apparatus and method using frame-rate conversion - Google Patents

Abstract

PURPOSE: An image processing apparatus using frame-rate conversion and a method therefor are provided to remove a motion vector discrimination error by inter-field interval difference and perform the frame-rate conversion. CONSTITUTION: The first and second image block extracting units(210,220) extract blocks with a certain size from an inputted field-unit interface image. A motion vector extracting unit(240) compares the blocks extracted from successively inputted two images, and calculates a motion vector. A motion filter(260) discriminates whether motion exists using a motion vector value of pixels within a certain range. If the motion exists, the motion filter(260) passes the motion vector value. If the motion does not exist, the motion filter(260) compulsorily changes the motion vector to '0' and outputs the '0'. An FRC(Frame Rate Converter)(280) performs the frame rate conversion of the motion vector values outputted from the motion filter(260), and outputs an image.

Description

Image processing apparatus and method using frame-rate conversion

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a frame-rate converter (FRC) using a motion estimation technique. More particularly, the present invention relates to reducing and converting memory usage during field-based image processing in a frame-rate converter. The present invention relates to a motion filter of a field-to-field frame-rate converter and a method of implementing the same.

In general, an image processing method of a high-definition display device such as a digital television or a PDP uses progressive scanning. This method displays video frames on a frame-by-frame basis, as if the film is projected on the screen.

1 is a block diagram of an image signal processing part of a general digital display device. The configuration includes a de-interlacing unit 100, a 1-frame delay unit 110, a first pixel block 120, a second pixel block 130, and a sum of absolute difference map 140. , A motion vector extractor 150, a frame rate converter (hereinafter referred to as FRC 160), and an interlacing 170.

The deinterlacing unit 100 converts an input image in a field unit into a frame unit.

The 1-frame delay unit 110 delays the image converted in units of frames, thereby making it possible to compare the previous frame (hereinafter referred to as P frame) and the current frame (hereinafter referred to as C frame).

The first pixel block 120 is a C × C pixel block located at a predetermined position in a C frame of the currently input image. ) And extract.

The second pixel block 130 determines and extracts a C × C pixel block located at the predetermined position in the P frame.

The SAD map 140 compares the difference SAD between the block in the P frame extracted from the first pixel block 120 and the block in the C frame extracted from the second pixel block 130 and stores the difference value.

The motion vector extractor 150 determines the displacement having the smallest value among the values stored by the SAD map 140 as the movement direction and size of the current block.

The FRC 160 performs frame rate conversion.

The interlacing unit 170 converts the frame rate-converted image into a field unit image which is the size of the original input.

In FIG. 1, in order to clearly perform FRC, the currently input image (C frame image) and the previously input image (P frame image) should be the same pixel unit image. However, since the interlaced input image is composed of two fields, odd and even, there is a pixel difference, and the current input image and the previously input image have a difference of 1 pixel up and down. This can be a problem when comparing two fields to detect vector motion. For this reason, the vertical interpolation is performed through the deinterlacing 100 to prevent the pixel difference between the two images. The deinterlaced image is stored in the memory (not shown) and the pixel blocks of the predetermined position are compared with the pixel blocks of the same position extracted from the previous frame delayed by the 1-frame delay unit 110. The block comparison between the previous frame and the current frame is performed to estimate where the previous image is currently moved within a predetermined search range, thereby allowing the motion vector extraction unit 150 to determine the motion vector. The predetermined block size used is a square block of C × C. This is true for interlaced display formats where the horizontal has twice the number of pixels as vertical, so a block size of C x C / 2 can be used, but square blocks have been de-interlaced by doubling the number of vertical pixels through de-interlace. To extract the motion vector. The extracted motion vector is applied to the FRC algorithm to create a new frame, and the interlacing unit 170 outputs half the number of vertical pixels to match the display format.

In the above-described conventional image processing apparatus, the amount of memory is increased to store an image and the amount of calculation is also increased in proportion to the increase of memory by initially performing deinterlacing on the input image. In addition, the last interlacing requires another memory capacity. This increase in memory and amount of computation not only expands the hardware to implement this, but can also cause problems with heat generation and error in signal processing. In addition to this problem, the image which has passed through the deinterlace and the interlace as in the conventional image processing configuration is not the same as the original image, resulting in image quality degradation.

The technical problem to be achieved by the present invention is to perform FRC conversion after eliminating the motion vector discrimination error due to the gap between fields so as to simplify the hardware and increase the computational efficiency by eliminating the de-interlacing process for the input interlaced image To provide an image processing apparatus and method using a, FRC conversion.

1 is a block diagram of a general digital image signal processing apparatus.

2 illustrates one embodiment of an image processing apparatus of the present invention.

3 is a flow chart illustrating in detail the method of motion filtering.

4 illustrates an example of an input original image.

FIG. 5 illustrates a partial result of the image of FIG. 4 after processing including the deinterlacing and interlacing functions as shown in FIG. 1.

FIG. 6 shows an output result of an image generated due to a motion vector processing error when deinterlacing is omitted for the image of FIG. 4.

FIG. 7 illustrates the image output result when the deinterlacing is omitted and the motion vector error is compensated by applying the motion filter algorithm of the present invention.

In order to solve the above problems, an image processing apparatus for frame rate conversion (FRC) includes a block extracting unit for extracting a block having a predetermined size from an interlace image of an input field unit; A motion vector calculator configured to calculate a motion vector by comparing respective blocks extracted from two input images sequentially; A motion filter for determining the presence or absence of motion using the motion vector values of pixels within a predetermined range, passing the motion vector value determined to be motion as it is, and forcibly converting the motion vector value determined to be no motion to “0” and outputting the motion vector value; And an FRC converter that outputs an image by performing frame rate conversion on the motion vector values output from the motion filter.

The block extractor may include C × C / 2 ( It is desirable to determine blocks of the form

The block extractor further includes a delay block for delaying a previously inputted field image by one field length, the first block extractor extracting a block therefrom, and a second block extractor extracting a block from a currently input field image. It is preferable to.

The image processing apparatus using the FRC transform compares each pixel extracted by the first block extractor and each pixel extracted by the second block extractor to obtain a sum of absolute difference (SAD) map that obtains a difference. It is preferable to further include.

Preferably, the motion vector extractor extracts a motion vector of an input image from the SAD map.

The motion filter may include a motion discrimination unit configured to determine whether or not there is motion by using motion vector values of pixels with respect to an image within a predetermined mask range; And a motion compensation unit for passing the motion vector value determined to have been motion by the motion discriminating unit as it is, and forcibly converting the motion vector value determined to have no motion to " 0 ".

The motion discrimination unit sets a predetermined mask block, converts the motion vector value calculated by the motion vector calculation unit into an absolute value, and applies the mask block so that the absolute value of the motion vector is within a predetermined pixel size. It is desirable to determine whether or not the number is less than a predetermined number.

Preferably, the motion compensator converts the absolute values of the motion vectors in the mask range to "0" when the motion vector absolute values in the mask block are less than or equal to a predetermined number of pixels. .

Preferably, the motion compensator outputs the motion vector as it is when the absolute value of motion vectors in the mask block is less than or equal to a predetermined pixel size.

The predetermined pixel size is 1, and the predetermined number is the number of pixels covered by the mask block.

In order to solve the above problems, the image processing method using the FRC transform, extracting a block having a predetermined size from the interlaced input image; Comparing each of the blocks extracted from the successive successive interlaced input images and extracting a motion vector therefrom; Determining the presence or absence of motion by using motion vector values of pixels within a predetermined range; When it is determined that there is motion, passing the motion vector value within the predetermined range as it is, and forcibly converting the motion vector value within the predetermined range determined to have no motion to " 0 "; And performing frame rate conversion on the motion vector values output according to the presence or absence of motion.

The block extracting step uses C x C / 2 ( It is preferable to extract a block of a format.

The block extracting step may include delaying a previously input field image by one field length and extracting a first block from the delayed image; And extracting a second block from a currently input field image.

The image processing method using the FRC transformation preferably further includes comparing each pixel of the first block with pixels of the second block corresponding thereto to obtain a difference.

And extracting a motion vector of the input image from the difference between the pixels of the first block and the second block, and storing the result.

The motion determining step may include setting a predetermined mask block; Calculating an absolute value of the motion vector value; And applying the mask block to determine whether or not the absolute value of the motion vector is less than or equal to a predetermined pixel size within the range.

In the motion compensation step, when the absolute value of the motion vectors in the mask block is less than or equal to a predetermined pixel size, the absolute value of the motion vectors in the corresponding mask range is converted to “0” and output.

Preferably, the motion compensation step outputs a motion vector within the mask range as it is when the absolute value of the motion vectors in the mask block is less than or equal to a predetermined pixel size.

The predetermined pixel size is 1, and the predetermined number is the number of pixels covered by the mask block.

Preferably, the mask block is in a 3x3 format and the predetermined number is nine.

In the image processing apparatus using the FRC transform to solve the above problem, the motion vector value with respect to the motion vector value extracted from the interlaced input image due to the line shift between the odd (odd) or even field (even) A motion filter for determining whether to; And a motion vector compensation unit forcibly converting the motion vector value to “0” and outputting the motion vector value as it is, if the motion vector value is due to the line shift, and outputting the motion vector value as it is otherwise. It is characterized by.

In order to solve the above problems, an image processing method using FRC transform includes: determining whether a motion vector value is due to a line shift between a hole or an even field with respect to a motion vector value extracted from interlaced input images; And forcibly converting the motion vector value to “0” and outputting the motion vector value as it is, when the motion vector value is due to the line shift for FRC conversion. It is done.

In order to perform frame rate conversion on the interlaced input image, the input image must be deinterlaced and interlaced at the beginning and the end, respectively. As a result, hardware has increased and design difficulties have arisen. In the present invention, instead of eliminating such deinterlacing and interlacing operations and the resulting hardware, the present invention employs an algorithm that yields the same frame rate conversion effect.

By simply removing the deinterlacing and interlacing, the lines are shifted from each other because the two images to be input and compared are each an even field and an odd field.

Since there is no motion in still images, it is recognized that there is substantially no motion vector. However, since the interlaced input image has a difference of 1 pixel even if there is no motion between the previous input field image and the current input field image, the interpolated image should be interpolated from one of the two field images. In this case, it is not a problem if only one image is consistently selected. However, when both field images are randomly interpolated and the image is interpolated, the output image is shifted by one pixel. Accordingly, the present invention does not perform the deinterlacing and interlacing operations conventionally required for FRC conversion of an image (video), but uses a simple motion filter to solve the problem.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

2 illustrates one embodiment of an image processing apparatus of the present invention. The configuration includes a 1-field delay unit 200, a first image block extractor 210, a second image block extractor 220, a sum of absolute difference map 230, and a motion vector. An extractor 240, a storage 250, a motion filter 260, a motion vector compensator 270, and a frame rate converter (hereinafter referred to as FRC) 280 are included.

The 1-field delay unit 200 delays an image input in units of fields, and is to make it possible to compare a field of a previously input image with a field of an image currently being input.

The first image block extractor 210 extracts a block for extracting a motion vector from the currently input image. This block takes the form of a C x C / 2 pixel block ( Is determined as As in the previous technique, since the vertical length is shorter than the frame-by-frame image by deinterlacing, the length in the vertical direction can be reduced by half.

The second image block extractor 220 extracts a block for extracting a motion vector from a previously input image. The shape and position of the block are the same as the blocks extracted by the first image block extractor 210.

The SAD map 230 compares the difference (SAD) of the pixels in the image block previously inputted with the current input image block extracted by the first image block extractor 210 and the second image block extractor 220, respectively. Store the difference.

The motion vector extractor 240 extracts the displacement having the smallest value among the values stored by the SAD map 230 as a motion vector value that is a moving direction and a magnitude of the current block.

The storage unit 250 stores the motion vector values extracted by the motion vector extractor 240.

The motion filter 260 is for screening motion vector values in the storage unit 250 that are incorrectly determined as motion vectors based on their absolute values. That is, for the motion vector value extracted from the interlaced input images, it is determined whether the motion vector value is due to a pixel position difference due to line shift between an odd or even field.

The motion vector compensator 270 coerces the value of the incorrectly determined motion vector determined from the motion filter 260 to "0", and outputs the motion vector as it is.

The frame rate converter (hereinafter referred to as FRC) 280 performs frame rate conversion on the motion vector value output from the motion vector compensator 270.

Referring to the operation of Figure 2 as follows.

The 1-field delay unit 200 delays the input interlaced image by one field unit. As a result, a rectangular block of pixels of CxC / 2 (e.g., C = 32) can be set (extracted) from the previously inputted field delayed image and the current inputted image, and compared with each other in block units It becomes possible.

The SAD map 230 compares each pixel of the block set from the currently input image and the block set from the previously input delayed image and obtains the respective gap difference.

Using the difference, the motion vector extractor 240 extracts the motion vector MV (vx, vy) from the image of the block, and the result is stored in the storage 250.

The motion filter 260 is a motion vector value obtained from the storage unit 250 due to inherent characteristics of even (even) field image and odd (hole) field image (input order may be reversed) sequentially input. This is to prevent the gap from being interpreted as having 'motion'.

The motion filter 260 sets a mask having a predetermined size A × B (for example, A and B may be set to 3 respectively). After calculating the absolute value of the motion vector value for the pixels covered by the mask, it is determined whether or not the absolute value of the motion vector within the mask block range is less than or equal to a predetermined number of pixels.

In this case, the predetermined pixel size is 1 (gap difference between the even and odd fields), and the predetermined number is the number of pixels covered by the mask block (preferably 9 according to A × B = 3 × 3 format).

When the motion vector absolute values of the pixels in the mask block range are less than or equal to 9 or more from the motion vector compensator 270 motion filter 260, it is determined that there is no motion, and the absolute values of the motion vectors of the pixels in the mask range are determined. Coerced to "0" to output. In addition, when the motion vector absolute values of the pixels in the mask block range are less than or equal to 9, the motion filter 260 determines that the motion has actually occurred, and outputs the motion vector in the mask range as it is.

Here, the size of the mask block can be arbitrarily determined.

The absolute value is obtained from the motion vector value above because only information on the moving range is required regardless of the direction. It is determined whether the absolute value corresponding to each coordinate of the motion vector value is smaller than the predetermined thresholds th1 and th2 to obtain an incremented count value. Herein, the predetermined thresholds are preferably 2 respectively. The magnitude comparison between the motion vector absolute value and the predetermined threshold value of the C × C / 2 block is performed for all cells in the mask.

Motion filtering is performed in the case of a still picture. The input still interlaced image (image) has a value of vy = 1 only on the vertical axis due to the comparison of the even and odd fields having a difference of 1 pixel even though the motion is "0". Fields interpolated between them arbitrarily select vy as either 0 or 1. Therefore, the straight line shifted by 1 pixel can be remarkably observed from the output still image.

The motion vector compensator 270 is intended to prevent the distortion of the image during output by making vy consistently " 0 " in this case.

3 is a flow chart illustrating in detail the method of motion filtering.

First, let MV (vx, xy) be a result of extracting a motion vector of pixels constituting the block from blocks of a predetermined size (C × C / 2) of two field images sequentially input.

These motion vectors are stored in storage and then motion filtered one by one.

To start motion filtering, a mask block of size A × B is first set (step 300) to check the possibility of a motion vector.

The motion vector of one pixel within the A by B mask ( Take the absolute value of step (310). This may be expressed as in Equation 1 below.

The absolute value substitution of the motion vector value is for calculating a moving range irrespective of the directionality of the vector value, so as to prevent the possibility of erroneously judging an image without motion as an image with motion.

A predetermined threshold at which each of the absolute values of It is determined whether or not to exceed (step 320). Where the predetermined threshold Is the number of pixels larger than 1 pixel, that is, 2 respectively.

The absolute value of the motion vector If less, the count is increased by one (step 330). Incrementing the count is performed for cases where the absolute value of the motion vector has a value of one pixel or less.

In operation 340, it is determined whether all of the pixels within the range that the mask block can cover are performed (step 340). When the mask size is 3 x 3, steps 320 to 330 are performed for all nine pixels.

From the motion vector of the pixels that the mask can cover It is determined whether fewer cases are greater than or equal to another threshold value th3 (step 350). This is to check whether the motion vector value of all the pixels in the mask block range is composed of values of 0-1.

If the motion vector value of the pixels in the mask does not exceed the threshold th3, part of it means that the absolute value of the motion vector is 2 or more, which means that there was motion in the mask. Accordingly, the value of the motion vector MV (vx, vy) of the predetermined block C × C / 2 of the input image passes through the motion filter while maintaining the original value (step 360).

If the motion vector value of the pixels in the mask is greater than or equal to the threshold th3, most or all pixels in the mask are less than or equal to the motion vector absolute value of 1, which means that there was no motion in the mask. Therefore, when comparing interlaced images with even and odd fields, all motion vector values of the corresponding block are forced to prevent straight line misalignment during output due to the motion vector value of 1 pixel even when there is no motion. 0 "(step 370).

The steps up to here are the motion filtering steps.

The filtered motion vector values of steps 360 and 370 after the motion filtering are completed are frame rate converted (FRC) and output (step 380).

Looking at the hardware of the present invention described above it can be seen that the following effects occur:

1. By reducing the block size to be processed in half (C x C-> C x C / 2), we save 50% of the required memory and 50% of the corresponding computation.

2. Since it does not perform de-interlacing and interlacing functions, the configuration is simpler, resulting in a reduction in computation amount and an increase in computation speed.

In spite of such hardware effects, the reason why the deinterlacing function could not be omitted until now is that when the deinterlacing is omitted, the error of not finding the correct position of the image occurs due to the difference between the pixels of the odd field and the even field. However, in the present invention, by providing a motion filter algorithm that solves this problem, the motion vector value is forced to "0" for the part that appears to move by 1 pixel when the deinterlacing is not performed on the still image. This prevents line misalignment.

4 illustrates an example of an input original image.

FIG. 5 illustrates a partial result of the image of FIG. 4 after processing including the deinterlacing and interlacing functions as shown in FIG. 1.

FIG. 6 shows an output result of an image generated due to a motion vector processing error when deinterlacing is omitted for the image of FIG. 4. The phenomenon that a line shift | deviates from the circle mark part of this figure can be found.

FIG. 7 illustrates the image output result when the deinterlacing is omitted and the motion vector error is compensated by applying the motion filter algorithm of the present invention. In this figure, the line shift shown in Fig. 6 cannot be found anywhere.

According to the present invention, by using a motion filter algorithm in the frame rate conversion processing of an image, it is possible to reduce the hardware configuration and the amount of computation, thereby increasing the cost and increasing the computational efficiency.

Claims (22)

In the image processing apparatus for frame rate conversion (FRC), A block extracting unit extracting a block having a predetermined size from an interlace image of an input field unit; A motion vector calculator configured to calculate a motion vector by comparing respective blocks extracted from two input images sequentially; A motion filter for determining the presence or absence of motion using the motion vector values of pixels within a predetermined range, passing the motion vector value determined to be motion as it is, and forcibly converting the motion vector value determined to be no motion to “0” and outputting the motion vector value; And And an FRC converter configured to output an image by performing frame rate conversion on the motion vector values output from the motion filter. The method of claim 1, wherein the block extraction unit, From the input image, C x C / 2 ( Image processing apparatus using the FRC transform, characterized in that it determines the block of the () form. The method of claim 1, wherein the block extraction unit, And a delay block for delaying a previously input field image by one field length, the first block extractor extracting a block therefrom, and a second block extractor extracting a block from a currently input field image. Image processing unit using FRC conversion. The method of claim 3, The method further includes a sum of absolute difference (SAD) map that compares each pixel extracted by the first block extractor and each pixel extracted by the second block extractor to obtain a difference. Image processing device. The method of claim 4, wherein the motion vector extraction unit And a motion vector of the input image is extracted from the SAD map. The method of claim 1, wherein the motion filter, A motion discrimination unit for determining the presence or absence of motion using the motion vector values of the pixels with respect to an image within a predetermined mask range; And The image using the FRC transform, which includes a motion compensator for passing the motion vector value determined by the motion discrimination unit as it is, and forcibly converting and outputting the motion vector value determined as non-motion. Processing unit. The method of claim 6, wherein the motion determination unit, When a predetermined mask block is set, the motion vector value calculated by the motion vector calculating unit is converted into an absolute value, and the absolute value of the motion vector is within a predetermined pixel size within the range by applying the mask block. An image processing apparatus using FRC conversion, characterized in that it is determined whether or not it is abnormal. The method of claim 7, wherein The motion compensator converts the absolute values of the motion vectors within the mask range to "0" when the motion vector absolute values in the mask block are less than or equal to a predetermined pixel size. An image processing apparatus using FRC conversion. The method of claim 7, wherein And the motion compensator outputs the motion vectors as they are when the motion vector absolute values in the mask block are less than or equal to a predetermined pixel size. The method according to claim 8 or 9, And the predetermined pixel size is 1 and the predetermined number is the number of pixels covered by the mask block. In the image processing method using the FRC transform, Extracting a block having a predetermined size from the interlace input image; Comparing each of the blocks extracted from the sequentially successive interlaced input images and extracting a motion vector therefrom; Determining the presence or absence of motion by using motion vector values of pixels within a predetermined range; When it is determined that there is motion, passing the motion vector value within the predetermined range as it is, and forcibly converting the motion vector value within the predetermined range determined to have no motion to " 0 "; And And performing frame rate conversion on the motion vector values output according to the presence or absence of motion. 12. The method of claim 11, wherein the block extraction step From the input image, C x C / 2 ( ) A method of processing an image using an FRC transform, characterized by extracting a block of a format. The method of claim 11, wherein the block extraction step, Delaying a previously input field image by one field length and extracting a first block from the delayed image; And And extracting a second block from the currently input field image. The method of claim 13, And comparing each pixel of the first block with the pixels of the second block corresponding to each pixel to obtain a difference thereof. The method of claim 14, Extracting the motion vector of the input image from the difference between the pixels of the first block and the second block, and storing the result. The method of claim 11, wherein the motion determination step, Setting a predetermined mask block; Calculating an absolute value of the motion vector value; And And determining whether or not the absolute value of the motion vector is less than or equal to a predetermined pixel size within the range by applying the mask block. The method of claim 16, The motion compensation step uses an FRC transform, when the absolute value of motion vectors in the mask block is less than or equal to a predetermined pixel size, the absolute value of motion vectors in the corresponding mask range is converted to " 0 " Image processing method. The method of claim 16, The motion compensation step may further include outputting a motion vector within a corresponding mask range when the motion vector absolute values in the mask block are less than or equal to a predetermined pixel size. The method of claim 17 or 18, And the predetermined pixel size is 1, and the predetermined number is the number of pixels covered by the mask block. 20. The image processing method according to claim 19, wherein the mask block is in a 3x3 format and the predetermined number is 9. In the image processing apparatus using FRC conversion, A motion filter for determining whether the motion vector value is due to odd or even field misalignment with respect to the motion vector value extracted from the interlace input images; and And a motion vector compensator for coercively converting the motion vector value to “0” and outputting the motion vector value as it is, if the motion vector value is due to the line shift, and outputting the motion vector value as it is otherwise. An image processing apparatus using FRC conversion. In the image processing apparatus using FRC conversion, Determining whether the motion vector value is due to a line shift between an odd or even field, with respect to the motion vector value extracted from the interlace input images; And For FRC conversion, if the motion vector value is due to the line misalignment, forcibly converts the motion vector value to "0" and outputs it; otherwise, outputting the motion vector value as it is. Image processing method for FRC conversion.