KR101072458B1 - High-speed image encorder based on motion vector reference map and method used in the same - Google Patents

Abstract

The present invention relates to a fast image encoder based on a motion vector reference map and a method used therein, wherein a motion vector of a macroblock of a first size of an input image is obtained for each reference image, and a macroblock of a first size is obtained from a first image. The SAD (Sum of Absolute Differences) between the pixel value in the area of the block of the second size and the pixel value in the area moved by the motion vector of the macroblock of the first size is divided into blocks of the second size smaller than the size. A motion vector reference map in which the reference image is calculated for each reference image, the reference image is allocated to each block of the second size, and the encoding mode is determined using the generated motion vector reference map. As a result, the coding complexity can be reduced without coding loss.

Description

High-speed image encoder based on motion vector reference map and its method {{HIGH-SPEED IMAGE ENCORDER BASED ON MOTION VECTOR REFERENCE MAP AND METHOD USED IN THE SAME}

The present invention relates to a fast image encoder based on a motion vector reference map and a method thereof, and more particularly, to a fast image encoder based on a motion vector reference map which reduces coding complexity without loss of coding, and to selecting a fast reference image used therein. It is a way.

Recently, as hardware is advanced and consumers want high-definition service, ITU-T and MPEG have jointly defined a video coding standard with high-performance compression efficiency called H.264 / AVC. The new technology of the H.264 / AVC video standard has a compression structure similar to the previously defined H.263, MPEG-4, etc., but maximizes the compression efficiency.

1 shows a simplified functional block diagram of an encoder part of the H.264 / AVC video standard. When the input current image is an intra frame, H.264 / AVC performs an entropy encoding process on the quantization coefficients obtained through DCT transformation and quantization, based on the difference between the prediction pixel determined as an optimal mode through the intra prediction process and the current image. The data is sent to the decoder.

The coded image is stored in the reference frame image memory by adding the intra prediction pixel and the image obtained by IDCT and inverse quantization. In the case of the inter frame, the difference between the prediction image obtained by the motion prediction / compensation of the input image and the reference image and the difference between the prediction image determined in the intra mode and the prediction pixel determined as the optimal mode are different from the current input image. Entropy coding is performed through DCT transformation and quantization. The encoded image is stored in the reference image memory by combining the image obtained through IDCT and inverse quantization and the motion compensated image as well as the intra frame.

In a typical image encoder, instead of encoding a whole image unit, a picture element or a pixel of a predetermined size block (e.g., M pixels in the horizontal direction and N pixels in the vertical direction (usually represented by M × N)) Encoding is performed in units of sets of pixels, and a block size is generally 16 × 16 or 8 × 8. A 16x16 block is called a macroblock and an 8x8 or 4x4 block is called a block.

 While the conventional technique uses one previously decoded image as a reference image with a minimum of 16 × 16 blocks as a reference image, motion compensation is performed and then encoded in 8 × 8 block units, whereas macroblocks of H.264 / AVC are variable. Compensate for motion using blocks. Such a motion compensation technique using a variable block divides the macroblock size from 16 × 16 to 4 × 4 to reflect the characteristics of the image, thereby increasing coding efficiency but requiring high complexity. This complexity is due to the use of rate-distortion optimization (RDO), which requires a large amount of computation to determine the mode of the macroblock for variable block motion compensation.

H.264 / AV supports five inter modes and three intra modes for the macroblock mode, of which SKIP, 16 × 16, 8 × 16, 16 × 8, SUB8 × 8 (or 'P8 × 8'). Denotes an inter mode and is used for macroblock encoding through motion prediction. The SUB8x8 mode is determined to be one of 8x8, 8x4, 4x8 and 4x4 in each 8x8 block (see FIG. 3). The inter mode prediction method divides an image into blocks of various sizes as illustrated in FIG. 3, and uses motion compensation to predict a motion from a previously encoded image and generate a current block. If the block size of motion compensation is reduced, more accurate prediction can be achieved. However, since the coding efficiency decreases because the amount of motion vector information to be encoded for each block increases, it is very important to determine an optimal block size among blocks of various sizes.

H.264 / AVC also provides Intra4 × 4, Intra8 × 8, and Intra16 × 16 for intra mode prediction, and Intra8 × 8 mode for luminance blocks is provided only in the Fidelity Range Extensions (FRExt) profile. Intra mode predicts the current macroblock using the boundary pixels of the previously encoded valid neighboring macroblocks.

As described above, it is very important to select a mode having the best coding efficiency among inter modes and intra modes. H.264 / AVC considers the number of bits that occur with distortion to determine the optimal mode among various encoding modes. To do this, we use a cost function based on the Lagrangian function. The rate-distortion optimization technique calculates the rate-distortion cost for each macroblock mode. In inter mode, the motion vector and the reference picture must be determined before calculating this value. 2 shows a process for finding a mode with a conventional minimum rate-distortion. Referring to FIG. 2, macroblocks, which are basic blocks of an inter mode, are divided into 16 × 16, 16 × 8, 8 × 16, and SUB8 × 8 partitions. SUB8x8 can be further subdivided into 8x8, 8x4, 4x8 and 4x4 modes independently for each 8x8 block. For each of the divided partition blocks, Equation 1 is used to determine a motion vector having a minimum value and a reference picture.

Jmotion is a function to determine the optimal motion vector and reference image, and SADmode is the current input image block s and motion prediction / compensation for each partition block. It is obtained by using a block r and a difference obtained from a reference image decoded in the process and stored in the frame memory. H and V are the height and width of each partition block, and mx and mv are the motion vectors. R ( MV , REF ) is the number of bits needed to encode the motion vector MV and the reference image REF , and λ motion is a Lagrangian coefficient, which is dependent on the quantization coefficient to match the order of the distortion value. The prediction modes determined by the inter mode and the intra mode determine and encode a mode in which the Jmode value is minimum as an optimal mode for macroblock encoding using Equation 3.

Where λ mode is the square of λ motion . H and V are block sizes for all blocks, and M is all predictable prediction modes. R ( s, rM , M ) is the number of bits that actually occur when encoded in the mode corresponding to M, and SSDmode is calculated as the square of the difference between the current input and the decoded image rM . In order to predict the minimum rate-distortion for the motion prediction in the inter mode, not only the image just created but also the images created in up to 16 bidirectional directions are used as reference images. 4 shows a process of motion estimation and compensation using N reference images previously made.

As described above, H.264 / AVC uses motion prediction using multiple reference pictures to increase the coding efficiency of the interblock. If the multi-reference video motion prediction is performed, accurate prediction is possible, and thus the temporal redundancy can be further reduced. The higher the number of reference pictures, the higher the PSNR and the lower bit-rate can be obtained. However, as the number of reference pictures increases, the computational complexity also increases linearly in proportion because the rate-distortion based mode decision method is applied to all reference pictures.

An object of the present invention is to provide a fast video encoder based on a motion vector reference map with reduced coding complexity without loss of coding.

Another object of the present invention is to provide a fast reference picture selection method in which coding complexity is reduced without coding loss.

The objectives of the present invention include a DCT / quantizer for compensating and encoding a motion of an input image, an entropy encoder for entropy encoding an image encoded by the DCT / quantizer, and a motion vector reference map. An encoding controller configured to determine an encoding mode of a quantization unit, and a storage unit configured to store the motion vector reference map, wherein the encoding controller obtains a motion vector for a macroblock of a first size of an input image for each reference image. And dividing the macroblock of the first size into blocks of a second size smaller than the first size, and moving the pixel block included in the block area of the second size and the motion vector of the macroblock of the first size. A SAD between pixel values in a region is calculated for each reference image, and the block size of the second size is based on the calculated difference values. Reference image a motion vector assigned reference movement, characterized in that for generating a reference vector map can be accomplished by a high speed image coder based on the map.

In addition, the above objectives, in the method for selecting a reference image to be used to compensate for the motion of the input image, the method comprising: obtaining a motion vector for each macroblock of the first size of the input image for each reference image, Dividing a macroblock of a first size into a block of a second size smaller than the first size, and moving the pixel value in the area of the block of the second size and the motion vector of the macroblock of the first size Calculating a SAD between pixel values in, for each reference image, generating a motion vector reference map to which a reference image is allocated for each block of the second size, based on the calculated difference values, and the motion vector reference map Can be achieved by a fast reference image selection method based on a motion vector reference map including selecting a reference image. .

The fast vector image encoder and the fast reference image selection method based on the motion vector reference map according to an embodiment of the present invention have the effect of reducing the coding complexity without coding loss by reducing the number of reference images used when the encoding mode is selected. do.

In addition, the fast vector image encoder and the fast reference image selection method based on the motion vector reference map according to an embodiment of the present invention reduce coding complexity by reducing the number of reference images used when selecting a coding mode, thereby reducing encoding complexity by about 38%. The effect is to reduce the degree.

In addition, the fast vector image encoder and the fast reference image selection method based on the motion vector reference map according to an embodiment of the present invention reduce the number of reference images used when selecting an encoding mode while maintaining the current standard encoding standard. Coding complexity is reduced by about 38% without coding loss.

1 is a schematic functional block diagram of a conventional standard video encoder;
2 is a flowchart illustrating a method of determining an encoding mode in a conventional standard video encoder.
3 is a diagram for explaining a block size division partition of a conventional inter mode;
4 is a diagram for explaining temporal motion prediction using a conventional multi-reference image;
5 is a functional block diagram of an encoder according to an embodiment of the present invention;
6 is a flowchart illustrating a method of generating a motion vector reference map according to an embodiment of the present invention;
7 is a flowchart illustrating a method of determining an encoding mode according to an embodiment of the present invention;
8 is a diagram for describing a method of determining a reference image for motion prediction and compensation of a variable block according to an embodiment of the present invention; and
9 is a diagram for describing a fast reference image determination method for motion prediction according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In describing the following specific embodiments, various specific details are set forth in order to explain and understand the invention in more detail. However, one of ordinary skill in the art can understand that the present invention can be used without these various specific details. In some cases, it is mentioned in advance that parts of the invention which are commonly known in the description of the invention and which are not highly related to the invention are not described in order to prevent confusion in explaining the invention without cause.

5 is a functional block diagram of an encoder according to an embodiment of the present invention. Referring to FIG. 5, an encoder according to an embodiment of the present invention includes a DCT / quantization unit 101, an inverse quantization / inverse DCT unit 103, an entropy encoding unit 105, a deblocking filter 107, and a screen. It includes an internal predictor 109, a motion vector detector 111, a motion compensator 113, a weight predictor 115, an encoding controller 117, and a storage unit 119 for storing a motion vector reference map. .

The DCT / quantization unit 101 compensates and encodes the motion of the input image. Specifically, the DCT / quantization unit 101 performs a mathematical process called a discrete cosine transform (DCT) on an input image to calculate a DCT coefficient value representing a frequency component and discretely calculate the calculated DCT coefficient value. Quantize by mapping to a representative value. According to an alternative embodiment of the present invention, the image input to the DCT / quantization unit 101 may be input through a predetermined preprocessing process. The preprocessing may include format conversion, frame reordering, and noise removal.

The inverse quantization / inverse DCT unit 103 performs inverse quantization and inverse DCT operations on an image encoded by the DCT / quantization unit 101 to make a prediction screen for motion compensation.

The entropy coding unit 105 entropy codes the video coded by the DCT / quantization unit 101. Specifically, the entropy encoder 105 performs a process of reducing the number of transmission bits as a whole by expressing high frequency information with short codes and other information with long codes.

 The deblocking filter 107 is a filter that reduces block distortion generated when encoding an image.

The intra prediction unit 109 performs an intra frame (intra frame or intra prediction) prediction operation, and may specifically predict a current pixel value from a plurality of adjacent pixels.

The motion vector detector 111 detects a motion vector for a predetermined block size. The block size is determined to be as optimal as possible. According to an alternative embodiment of the present invention, the encoding control unit 117 determines the optimal block size, and the motion vector detection unit 111 determines the motion vector with respect to the determined block size. Detect.

The motion compensator 113 performs motion compensation on the screen to be encoded based on the motion amount (motion vector) of the subject existing in the screen.

The weight predictor 115 multiplies the signal subjected to the motion compensation by an adaptive variable weight coefficient instead of a constant coefficient to generate a prediction signal.

The encoding control unit 117 performs an operation of optimizing the amount of information generated by selecting a method (that is, an encoding mode) of encoding a frame (picture) unit or a smaller unit of an image or selecting a parameter thereof. To this end, the encoding control unit 117 controls the operation of the other components. The encoding control unit 117 according to an embodiment of the present invention supports variable block size motion compensation, generates a motion vector reference map, and uses the same to determine an encoding mode.

While the motion compensation unit 113 may make more accurate prediction as the block size of the motion compensation is smaller, the code amount for the motion vector information may increase because the motion vector information should be encoded for each block. In order to solve this problem, a video coding scheme such as H.264 / AVC uses a method ('variable block size motion compensation') that uses an optimal one among motion compensation block sizes, and the present invention also uses variable block size motion compensation. do.

For example, in the present invention, five inter modes and three intra modes supported by H.264 / AVC are used, and the encoding control unit 117 utilizes a motion vector reference map to obtain the best coding rate among these modes. You can select the mode. Five inter modes are SKIP, 16x16, 16x8, 8x16, P8x8, where P8x8 mode is 8x8, 8x4, 4x8, 4 within each 8x8 block. It is determined by one of × 4. In the inter mode, motion compensation is performed by dividing an image into blocks of any one of the above five types, and then generating a current block by predicting motion from a previously encoded image. Three intra modes are Intra4x4, Intra8x8, and Intra16x16. In the intra mode, the current macroblock is predicted using the boundary pixels of previously encoded valid neighboring macroblocks.

Specifically, the encoding control unit 117 uses a method called a bitrate-distortion optimization technique, in which a mode having a minimum value by calculating bitrate-distortion values for each macroblock mode is encoded by the current macroblock. Determine as the best mode to make.

The encoding control unit 117 according to an embodiment of the present invention enables high-speed encoding by selecting using a motion vector reference map when selecting a reference image used when calculating a bit rate-distortion cost.

For example, the encoding controller 117 obtains a motion vector of a macroblock of a first size of the input image for each reference image, and converts a macroblock of a first size into a block of a second size smaller than the first size. The SAD between the pixel value in the region of the block of the second size and the pixel value in the region moved by the motion vector of the macroblock of the first size is calculated for each reference image and based on the calculated difference values. A motion vector reference map is generated by allocating a reference image for each block of two sizes.

In a preferred embodiment of the present invention, the macroblock of the first size is 16 × 16 macroblocks (MB) and the block of the second size is 4 × 4 blocks. According to the present embodiment, a motion vector is searched and stored for each reference picture in units of 16 × 16 macroblocks. To this end, under the control of the encoding control unit 117, the motion vector detecting unit 111 detects the motion vector for each reference image in units of 16 × 16 macroblocks, and stores the storage unit 119 or a separate storage unit (not shown). ). After detecting the motion vector for each reference image, the encoding control unit 117 divides the above-described 16 × 16 macroblock into 16 4 × 4 blocks, and each divided 4 × 4 block obtains each reference obtained above. The SAD between the pixel values in the region moved by the motion vector of the 16x16 macroblock per image is calculated. Subsequently, the encoding control unit 117 generates a motion vector reference map by allocating the number of each reference picture having the smallest SAD value for every 4x4 block. The motion vector reference map generated as described above is stored in the storage unit 119. Then, when motion prediction for variable blocks of various sizes is performed, a motion vector reference map made for 4 × 4 block sizes belonging to each variable block is performed. Is used. As a result, the number of applications of the reference image per variable block can be reduced, and encoding can be performed at high speed.

Meanwhile, although each of the components illustrated in the embodiment of FIG. 5 is shown as being separated from each other, this is merely exemplary, and some of these components are implemented by being integrated with each other in software and / or hardware. May be implemented separately from each other in software and / or hardware.

6 is a flowchart illustrating a method of generating a motion vector reference map according to an embodiment of the present invention, FIG. 7 is a flowchart illustrating a method of determining an encoding mode according to an embodiment of the present invention, and FIG. A diagram for describing a method of determining a reference image for motion prediction and compensation of a variable block according to an embodiment of the present invention.

Now, with reference to each of these drawings or simultaneously, a method of selecting a fast reference image based on a motion vector reference map according to an embodiment of the present invention will be described. A fast reference image selection method based on a motion vector reference map according to an embodiment of the present invention may generate a motion vector reference map and select a reference image at high speed by using the same.

Referring to FIG. 6, a method of generating a motion vector reference map is exemplarily illustrated. According to the present method, first, a motion vector (MV) for each reference image of a 16 × 16 macroblock (MB) is obtained and stored. (S601), SAD is calculated between the pixel value of each of the 4x4 blocks constituting the 16x16 macroblock and the pixel value in the region moved by the motion vector of the 16x16 macroblock for each reference image obtained in step S601. (S603). Thereafter, steps S601 and S603 are performed on all reference images (S605). When steps S601 and S603 are completed for all reference images (S605: YES), a motion vector reference map is created by allocating a reference image having the smallest SAD value for every 4x4 block (S607).

Referring to FIG. 7, a method of determining an encoding mode according to an embodiment of the present invention is exemplarily illustrated. According to the present method, a screen to be currently encoded (in this specification, 'screen,' image ', Or, terms such as 'frame' will be used interchangeably with each other unless there is no benefit to distinguish them from each other. If it is an intra frame, step S711 is performed. If it is an inter frame, step S703 is performed. In operation S711, as an operation of determining an intra mode, a method of determining a conventional intra mode may be used.

The case of an inter frame will be described. In the case of the inter frame, with reference to the motion vector reference map generated by the motion vector reference map generation method illustrated in FIG. 6, the 16x8 and 8x16 macroblocks are allocated to the map. A motion vector using only reference images is obtained (S703). Subsequently, with reference to the motion vector reference map, each bit rate-distortion value RD for the P8x8 block is obtained (S705). 8 (a), (b), (c) and (d) exemplarily illustrate operations of steps S703 and S705. Thereafter, the bit rate-distortion value RD is also calculated for other encoding modes (S707), and an encoding mode having the smallest RD value among the calculated values is determined as an optimal macroblock mode (S709).

9 is a diagram for describing a method of determining a fast reference image for motion prediction according to an embodiment of the present invention. Referring to FIG. 9, a method of sequentially performing a motion vector reference map generation and an encoding mode is illustrated. Is shown.

Referring to FIG. 9, when the current image is an intra frame, an operation of determining an intra mode is performed (S921: YES). In the case of an inter frame (S903: N0), an operation vector reference map is performed by performing steps S905 to S911. In operation S913 to S917, the RD value for each mode is calculated, and the mode having the smallest RD value is determined as an optimal macroblock.

Specifically, a motion vector for a 16 × 16 macroblock is obtained for each reference image Ref (S905 and S909), and the motion vector obtained in S905 and S909 for each of the 4 × 4 blocks constituting the 16 × 16 macroblock. The SAD value for the pixel values in the regions shifted by the vector is obtained using S907. A motion vector reference map is generated by allocating the number of the reference image having the smallest SAD value among every 4 × 4 blocks among the SAD values obtained in S907 (S911). Then, using the motion vector reference map, the motion vectors for each of the 16x8 and 8x16 blocks are obtained only in the allocated reference image (S913), and the RD values through the same process for each of the P8x8 blocks are obtained. Obtained (S915). Thereafter, the RD value for each mode is calculated to determine the mode having the minimum RD value as the optimum macroblock mode (S919).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

101: DCT / quantization unit 103: inverse quantization / inverse DCT
105: entropy encoding unit 107: deblocking filter
109: intra prediction unit 111: motion vector detection unit
113: motion compensation unit 115: weight prediction unit
117: encoding control unit 119: motion vector reference map storage unit

Claims (12)

A DCT / quantizer for compensating and encoding the motion of the input image;
An entropy encoding unit for entropy encoding an image encoded by the DCT / quantization unit;
An encoding control unit for generating a motion vector reference map and determining an encoding mode of the DCT / quantization unit using the motion vector reference map; And
And a storage unit which stores the motion vector reference map.
The encoding control unit,
A motion vector of a macroblock of a first size of an input image is obtained for each reference image, and the macroblock of the first size is divided into blocks of a second size smaller than the first size, and the Sum of Absolute Differences (SADs) between pixel values in an area and pixel values in an area moved by a motion vector of the macroblock of the first size are calculated for each reference image and based on the calculated difference values. A motion vector reference map based fast image encoder, characterized in that for generating a motion vector reference map to which a reference image is allocated for each block of size. The method of claim 1,
And the macroblock of the first size is a 16 × 16 macroblock (MB). The method of claim 1,
And the second sized block is a 4x4 block. The method of claim 1,
The encoding control unit,
And a motion vector reference map is generated by assigning the reference image having the smallest SAD value to the block of the second size. The method of claim 1,
The encoding control unit,
Determine whether the motion compensation is inter frame compensation (inter frame) or intra frame compensation (intra compensation), and in the case of inter frame compensation, the motion vector reference map is generated in units of macroblocks. Based fast image encoder. The method of claim 1,
The encoding control unit,
And a motion vector reference map based on the motion vector reference map when predicting a motion of a variable block between the macroblock of the first size and the block of the second size. The method according to claim 6,
The encoding control unit,
The fast vector encoder based on the motion vector reference, wherein the motion is predicted by using a reference picture allocated to a block of a second size constituting the variable block. A method of selecting a reference picture used to compensate for and encode a motion of an input picture,
Obtaining a motion vector for each macroblock of the first size of the input image for each reference image;
Dividing the macroblock of the first size into blocks of a second size smaller than the first size;
Calculating sum of absolute differences (SAD) for each reference image between a pixel value in an area of the block of the second size and a pixel value in an area moved by the motion vector of the macroblock of the first size;
Generating a motion vector reference map in which a reference image is allocated to each block of the second size based on the calculated difference values; And
And selecting a reference image using the motion vector reference map. The method of claim 8,
And a macroblock of the first size is a 16 × 16 macroblock (MB). The method of claim 8,
And a second sized block is a 4x4 block. The method of claim 8,
The generating of the motion vector reference map may include:
And a motion vector reference map is generated by allocating the reference image having the smallest SAD value to the block of the second size. The method of claim 8,
The generating of the motion vector reference map may include:
Determine whether the motion compensation is inter frame compensation (inter frame) or intra frame compensation (intra compensation), and in the case of inter frame compensation, the motion vector reference map is generated in units of macroblocks. Based Fast Reference Image Selection Method.

Images