KR100987922B1 - Frame based reference frame fate distortion optimization using Motion Blur/Deblur post filter - Google Patents

Abstract

The present invention relates to a method of generating a reference frame of a video codec for implementing motion compensation through motion prediction.

As a result of shooting with video equipment, each frame of the video is fixed in a number of frames due to external factors such as the camera's frames per second (FPS) and shutter speed and the movement or shaking of the shooting equipment, or the movement of the object being shot. Motion-Blur occurs. When motion prediction is performed on such an image, a difference image occurs when motion blurring occurs in the reference image and motion blurring does not occur in the input image, or when there is no motion blurring in the reference image and motion blurring exists in the input image. Larger values result in less efficient compression.

In the present invention, additional reference images are created by intentionally performing a motion blur and motion deblur filter on a reference frame, and motion estimation is performed between the original reference image and the added reference images. Compression efficiency is increased by selecting a reference image with less parallax image data as the final reference frame.

Coding, Decoding, Video, Motion Compensation, Motion Prediction, Reference Video

Description

Frame based reference frame fate distortion optimization using Motion Blur / Deblur post filter for motion compensation and decoding system using motion compensation technique using selective reference image

The present invention relates to a method of generating a reference frame of a video codec for implementing motion compensation through motion prediction. More particularly, the present invention relates to a motion or object of an image input apparatus such as a camera in addition to a conventional reference image. By creating a reference image with motion blur noise generated by motion and the like and removing the reference image, and selecting a reference image with better efficiency among these reference images, motion compensation can be achieved. The present invention relates to a method for increasing the compression efficiency of a video codec.

In general, a video encoder to which a motion compensation method is applied through motion prediction is as shown in FIG.

A subtractor 10 for obtaining a difference signal between an input image and a motion compensated image signal, a transform unit 20 for converting the difference signal obtained by the subtractor 10 according to a predetermined function; A quantization unit 30 for quantizing the image signal converted by the conversion unit 20, an encoding unit 40 for encoding the image information quantized in the quantization unit 30, and a quantization unit ( A motion compensation means for performing motion compensation by inversely processing the quantized video signal at 30),

The motion compensating unit includes an inverse quantization unit 50 for inverse quantization of the quantized video signal in the quantization unit 30 and an inverse transformer for inverting the converted signal in the transformer 20. 60), an adder 70 for generating a reconstructed image from the image converted by the inverse transform unit 60 by adding the motion compensated image, the frame memory 80, and a previous image or an inputted future image. A motion predictor 90 for predicting motion from a reference image to generate a motion vector, and a motion compensator 100 for compensating motion using the motion vector generated from the motion predictor 90. It is configured to include.

The input image may be classified into an intra-frame and an inter-frame, and the interframe selects a motion image by selecting a before or after image of the currently input image as a reference frame. Motion Estimation is performed to enhance the compression efficiency by encoding the residual frame in the order of Transform, Quantization, and Entropy Coding.

When the next image needs a reference image, a recontructed frame is generated through inverse quantization and inverse transform before entropy coding, and used as a reference image.

In the case of H.264, a deblocking filter is configured after the inverse transform unit 60 to obtain a clearer reference image.

In the case of MPEG4 part2, the transform unit 20 uses DCT (Discrete Cosine Transform) for transform and uses VLC (Variable Length Coding) for entropy coding of the encoder 40.

In case of H.264 / AVC, Integer Transform is used for transformation and CAVLC or CABAC is used for entropy coding. It also uses the Deblocking filter.

 In general, as a result of shooting with a video device, each frame of the video is fixed to a large number of frames due to external frames such as the camera's frames per second (FPS), shutter speed, and the movement or hand movement of the shooting device, or the movement of the object to be photographed. Motion-Blur occurs.

When motion prediction is performed on such an image, a difference image occurs when motion blurring occurs in the reference image and motion blurring does not occur in the input image, or when there is no motion blurring in the reference image and motion blurring exists in the input image. Larger values result in less efficient compression.

Therefore, in the present invention, a number of additional reference images are performed by intentionally performing a motion blur and a motion deblur filter on a reference frame, and motion estimation is performed between the original reference image and the added reference images. In this case, the compression efficiency is increased by selecting the reference image with less difference image data as the final reference frame.

To this end, the present invention,

In a video encoder for performing motion prediction using a reference image,

Originally, one or more motion blur filters and motion blur filters for generating additional reference video signals according to the motion degree and angle of the reference video, and frame memory configured for each of the motion blur filter and the motion blur filter And reference image generation control means for generating a reference image by selecting a motion blur filter and a motion deblur filter according to a blur mode determined by a preset program mode, and for the generated reference images and the original reference images. A motion prediction and mode selection control means for selecting a blur mode to be applied to the motion compensation by performing motion prediction and calculating the cost of the motion prediction process, and a motion for performing motion compensation according to the blur mode selected from the motion prediction and mode selection controller. Characterized in that it comprises a compensation means.

Such a motion prediction and compensation process of the encoder of the present invention,

A blur mode selection process of selecting a motion blur filter and a deblur filter according to a blur mode determined by a preset program mode to generate an additional reference image according to the degree and angle of movement with respect to the original reference image, and a selected motion blur filter And a filtering process of generating reference images by the deblur filter, a process of predicting motion on the original reference image and the reference images generated through the filtering process, and comparing the costs generated after the motion prediction with a minimum value. And a motion compensation process of compensating for the motion by selecting the blur mode.

On the other hand, the present invention,

In a video decoder for performing motion compensation using a reference image,

Filter selection control means for extracting the blur mode information from the decoded video signal and selecting and controlling a motion blur filter or a motion blur filter according to the extracted blur mode information to generate a reference picture, and one for generating a reference picture. Compensation of motion according to the above-described motion blur filter and motion deblur filter, frame memory for each of the motion blur filter and motion blur filter, motion motion information extracted from the generated stream, and generated reference image. Characterized in that it comprises a motion compensation means to.

The motion compensation process of the decoder of the present invention,

Extracting a blur mode from the decoded video signal, generating a reference image through a motion blur filter or a deblur filter according to the extracted blur mode information, and compensating for motion according to the generated reference image Characterized in that made.

The present invention effectively reduces bit generation by further reducing temporal redundancy by adding or subtracting the motion blur phenomenon present in most of the reference images used in motion prediction and compensation used to reduce temporal redundancy in video codecs, thereby effectively reducing the bit rate. Increases efficiency

The present invention is not limited to the MPEG4 and H.264 / AVC examples described above, and is applicable to video codecs that perform all inter predictions. The present invention has various algorithms in entropy coding such as Wavelet in addition to DCT in conversion. Applicable for both codecs.

First, referring to the embodiment of FIG. 2, the video compression encoding apparatus to which the motion compensation method using the selective reference image of the present invention is applied will be described with the same reference numerals.

A subtractor 10 for obtaining a difference signal between an input image and a motion compensated image signal, a transform unit 20 for converting the difference signal obtained by the subtractor 10 according to a predetermined function; A quantization unit 30 for quantizing the image signal converted by the conversion unit 20, an encoding unit 40 for encoding the image information quantized in the quantization unit 30, and a quantization unit ( A motion compensation means for performing motion compensation by inversely processing the quantized video signal at 30),

The motion compensating unit includes an inverse quantization unit 50 for inverse quantization of the quantized video signal in the quantization unit 30 and an inverse transformer for inverting the converted signal in the transformer 20. 60 and an adder 70 for generating a reconstructed image from the image converted from the inverse transform unit 60 by adding the motion compensated image, the frame memory 80, and the inverse transform unit through the inverse transform unit 60. Originally, one or more motion blur filters 110 and motion blur filters 120 for generating additional reference video signals according to the degree and angle of motion of the reference video, and the motion blur filter 110 and motion di Generates a reference image by selecting a blur frame memory 130 and a deblur frame memory 140 configured for each of the blur filters 120 and a motion blur filter and a motion deblur filter according to a predetermined mode to generate a reference image Motion prediction is performed on both the reference images received from the fisherman 150, the blur frame memory 130 and the deblur frame memory 140, and the original reference images received from the frame memory 80. And calculating the cost of the motion prediction process to perform motion compensation according to the mode selected from the motion prediction and mode selection controller 90 'and the motion prediction and mode selection controller 90'. It is configured to include a motion compensation unit (100 ').

Referring to the operation of the present invention in detail as follows.

As also stated in the prior art,

The interframe of the input image selects the image before or after the current input image as a reference image, performs motion prediction, and encodes the residual frame in the order of transform, quantization, and entropy coding. This increases the compression efficiency.

When the next image needs a reference image, a recontructed frame is generated through inverse quantization and inverse transform before entropy coding, and used as a reference image.

 At this time, in the present invention, in the process of predicting motion and compensating the motion, the compression efficiency may be reduced by motion blurring or debluring which may appear in the current video and the reference video, and thus the motion blur and the motion blur may be added to the reference video. The purpose of the present invention is to increase the compression efficiency by intentionally creating an additional reference image by performing motion de-filtering, and selecting a reference image having a small difference image data as a final reference frame by predicting a motion including the original reference image.

The reference image generation controller 150 selects the motion blur filter 110 and the motion blur filter 120 according to the program mode set therein to generate the blur mode.

That is, the n motion blur filters 110 and the m motion blur filters 120 are selected to generate (n + m) blur modes.

The image reconstructed through the inverse transform unit 60 is transferred to the motion blur filter 110 and the motion blur filter 120 through the frame memory 80, and the motion blur filter 110 and the motion de-selected as described above. The filter is filtered through the blur filter 120 to generate a plurality of reference images.

Thereafter, the motion prediction and mode selection unit 90 ′ stores and stores the blur frame memory 130 and the de blur frame memory filtered by the frame memory 80, the motion blur filter 110, and the motion blur filter 120. A motion is predicted for each reference image transmitted from 140, and a cost of each execution step is calculated to select an optimal mode.

3 illustrates a motion prediction process using the reference image as described above.

The motion mode selection process of selecting a motion blur filter and a motion deblur filter for generating additional reference images according to the degree and angle of movement with respect to the original reference image, and the motion blur filter and the motion deblur filter according to the selected motion mode. A motion mode having a minimum value by comparing a filtering process of generating reference images, a process of predicting motion on the original reference image and reference images generated through the filtering process, and a cost incurred after the motion prediction It includes a motion compensation process to compensate for the movement.

In blur mode generation, the reference image generation controller 150 sets the motion size (length, intensity), the direction of movement, and the type of filter with respect to the original reference image, and sets n modes in the motion blur filter 100. The m motion mode filter 120 sets m modes.

The movement size refers to the length or intensity of the movement. The movement size defines the number of filter taps to be used in the filter, and the movement direction refers to the direction of the filter tap such as horizontal, vertical or diagonal (360 degrees omnidirectional). The type of filter information indicates whether to use the uniform filter according to the filter tab or the PSF (Point Spread Function) filter.

An image with each mode blur is added to the set n modes, and an image with each mode blur is removed from the m modes.

The motion prediction and mode selection controller 90 'performs n + m + 1 motion predictions along with the existing reference image, calculates the cost of each execution step, and selects a low-cost mode as an optimal mode. Do it.

4 is a block diagram of a video compression decoding apparatus to which a motion compensation method using the selective reference image of the present invention is applied.

A decoder 200 that performs entropy decoding on the input stream, an inverse quantizer 210 that performs inverse quantization on the image signal decoded by the decoder 200, and an inverse quantized image The inverse transform unit 220 performing the transform process in the encoding process in reverse with respect to the signal, an adder 230 for generating a reconstructed image by adding a motion compensated image, and a video signal decoded by the decoder 200. The filter selection controller 240 for extracting the blur mode information from the blur mode information and selecting and controlling the motion blur filter 250 or the motion blur filter 260 using the extracted information to generate a reference image; And a plurality of frame memory 270 for each of the motion blur filter 250 and the motion deblur filter 260, and the motion blur filter 250 and the motion deblur filter 260, respectively. , Each It is configured to include a motion compensation unit 280 to compensate for the motion from the reference image transmitted from the frame memory 270.

The decoder of the present invention is characterized in that the motion compensation is performed by extracting the blur mode information input through the encoding process from the encoder of the present invention and generating a reference image through the extracted information. .

The process includes extracting a blur mode from the decoded video signal, generating a reference image through a motion blur filter or a deblur filter according to the extracted blur mode information, and compensating for motion according to the generated reference image. This includes the process of doing so.

The decoder decodes the input stream by performing entropy decoding on the decoder 200, and inversely quantizes and inverse transforms the input stream to form a reconstructed image from the motion compensated image.

At this time, the filter selection controller 240 extracts the blur mode information input from the decoded video signal.

The blur mode information exists inside the decoder in the form promised with the encoder, and selects whether to perform motion blur filtering or motion deblur filtering from the extracted blur mode information, and the motion size (length, intensity), motion direction, Obtain information about the type of filter to determine the filter to apply.

From this information, the filter selection controller 240 performs motion blur filtering or motion deblur filtering on the decoded reference image to obtain a new reference image to which motion blur noise is added or removed, and provides it to the motion compensation unit 280. .

The motion compensator 280 performs motion compensation by using the reference image.

1 is a block diagram showing the configuration of a general video encoder.

2 is a block diagram showing a configuration of a video compression encoding apparatus to which a motion compensation technique using the present invention's selective reference image is applied.

3 is a flowchart of a method of determining a selective reference image for motion compensation according to the present invention;

Figure 4 is a block diagram showing the configuration of a video compression decoding apparatus to which the motion compensation method using the present invention selective reference image.

Claims (7)

In a video encoder to which a motion compensation method using motion prediction is applied, One or more motion blur filters and motion deblur filters for generating additional reference picture signals from the original reference picture, a frame memory configured for each of the motion blur filter and the motion deblur filter, and a preset program mode Reference image generation control means for generating a reference image by selecting a motion blur filter and a motion deblur filter according to the blur mode determined by the step, and performing motion prediction on both the generated reference images and the original reference images. A motion prediction and mode selection control means for selecting a blur mode to be applied to the motion compensation by calculating the cost of the motion prediction process, and motion compensation means for performing motion compensation according to the blur mode selected from the motion prediction and mode selection controller. Using a selective reference image characterized in that Video compression encoding apparatus to which a motion compensation technique. The apparatus of claim 1, wherein the blur mode defined in the reference image generation control means includes motion size, direction of motion, and type of filter. In the motion prediction and motion compensation process of the compression encoder for predicting motion and compensating the motion according to the predicted motion, A blur mode selection process of selecting a motion blur filter and a deblur filter according to a blur mode determined by a preset program mode to generate an additional reference image according to the degree and angle of movement with respect to the original reference image, and a selected motion blur filter And a filtering process of generating reference images by the deblur filter, a process of predicting motion on the original reference image and the reference images generated through the filtering process, and comparing the costs generated after the motion prediction with a minimum value. And a motion compensation process of compensating for motion by selecting a blur mode. 4. The method of claim 3, wherein in the blur mode selection process, the blur mode includes information on the size of the motion, the direction of the motion, and the type of the filter. In the video decoder for performing motion compensation, Filter selection control means for extracting the blur mode information from the decoded video signal and selecting and controlling a motion blur filter or a motion blur filter according to the extracted blur mode information to generate a reference picture, and one for generating a reference picture. Compensation of motion according to the above-described motion blur filter and motion deblur filter, frame memory for each of the motion blur filter and motion blur filter, motion motion information extracted from the generated stream, and generated reference image. An apparatus for compressing and decoding a video to which a motion compensation technique using a selective reference image is applied, comprising a motion compensation means. The apparatus of claim 5, wherein the blur mode information includes motion size, direction of motion, and type of filter. 7. In the compression and decoding apparatus to which the motion prediction and compensation technique is applied, One or more motion blur filters and motion deblur filters are configured for a reference image for the original motion prediction to generate additional reference images, and motion prediction is performed on the generated reference image and newly added reference images, respectively. Calculates the cost of the motion prediction process, selects the motion prediction value to be applied to the motion compensation, performs the motion compensation, compresses and encodes the video, Comprising one or more motion blur filters and motion deblur filters, extracting filter information on the motion blur or motion deblur from the compressed coded image, and extracting the motion blur filter or the motion deblur filter according to the extracted information. The video compression encoding apparatus and the decoding apparatus applying the motion compensation method using the selective reference image, characterized in that for generating a reference image to compensate for the motion to decode the video.

Images