KR100900316B1 - Method and Apparatus for Intra mode prediction searching for compression of real time video signal - Google Patents

Abstract

The present invention provides a method for determining a search threshold by comparing a rate-distortion cost of a video signal macroblock received from an external source, comparing the rate-distortion cost of a received macroblock with a rate-distortion cost of an adjacent macroblock, Determining an inter mode having the minimum error sum by measuring the error sum for each inter mode of the received macroblock, and intra mode predictive search corresponding to the determined minimum error sum and the search threshold of the inter mode; It may provide an intra mode prediction search method for real-time video compression comprising the step of determining.

H.264, Intra Prediction Search, Inter Prediction Search

Description

Method and Apparatus for Intra mode prediction searching for real time video signal

1 is a view schematically showing the structure of a video compressor to which the present invention is applied;

2 is a diagram briefly showing a procedure of a video compressor to which the present invention is applied.

3 is a diagram schematically illustrating a procedure for predicting a conventional screen motion compared with the present invention.

4 is a diagram for explaining a macroblock to be applied in the H.264 system.

5 is a diagram schematically illustrating a configuration of a motion estimator for screen motion prediction according to an exemplary embodiment of the present invention.

6 is a diagram schematically illustrating a sequence for predicting screen motion according to an embodiment of the present invention.

7 is a diagram illustrating an intra mode search method based on an inter mode according to an embodiment of the present invention.

8 is a view showing a type of variable block of the inter mode in the H.264 scheme.

9 is a diagram illustrating a prediction search mode of an intra mode in the H.264 scheme.

10 is a view for explaining one practical embodiment when the present invention is applied.

<Explanation of symbols for the main parts of the drawings>

101: block unitization unit 103: block DCT (Discrete cosine transform) unit

105: quantizer 107: variable length encoder

109: motion compensator 111: inverse quantization and IDCT unit (inverse-DCT)

113: motion estimation unit 115: multiplexer

117: addition unit 119: subtraction unit

The present invention relates to an intra mode predictive searching method and apparatus for real time video signal compression.

H.264 is one of the compression methods currently used as a real-time video signal compression method. H.264 is the most recently standardized video compression coding scheme, which shows higher performance in terms of rate-distortion (RD) than previous video coding schemes, but also has a disadvantage of significantly increasing the complexity of the encoder. .

H.264 is a method of compressing the entire video screen in real time by dividing the screen into macro blocks and compressing the macro blocks by performing both the inter mode prediction search and the intra mode prediction search. .

7 and 8 illustrate a type of a variable block for determining an inter mode and a mode defined for searching for an intra mode in the present H.264 scheme.

As shown in FIG. 7, the inter mode includes seven types, and in order to determine the final inter mode, the mode having the minimum value is determined by measuring the seven rate-distortion costs in each macro block.

Intra mode has three types of block sizes of 4x4, 8x8, and 16x16, as shown in FIG. 8, and four modes (predictive directions) for I16x16MB (Intra 16x16 Luminance Macro Block) ), And there are nine modes in I4x4MB. In addition, there are four modes for the chroma component (Chrominance) 8x8 block.

Even in this intra mode search, the final intra mode search selects the mode having the least residual data as the final intra mode.

As described above, in the H.264 method, since all predictive search of the inter mode and the intra mode must be performed, the configuration of the encoder becomes very complicated, and as a result, there is a disadvantage in that power consumption and operation speed become slow.

Accordingly, various methods for solving such complexity have been discussed.

An object of the present invention is to provide an intra mode predictive search method and apparatus for real-time video signal compression.

In addition, the present invention improves the performance of H.264 video compression system in terms of bit rate and compression speed by speeding up intra mode prediction search among inter-mode prediction and intra-mode prediction techniques. There is this.

In order to achieve the above objects, according to an aspect of the present invention, the step of measuring the rate-distortion cost of the video signal macro block received from the outside, pre-segmented adjacent to the rate-distortion cost of the received macro block Comparing a rate-distortion cost of the macroblock to determine a search threshold, determining an inter mode having a minimum error sum by measuring an error sum for each inter mode of the received macroblock, and determining the determined inter An intra mode predictive search method for real-time video compression may be provided that includes determining an intra mode predictive search based on a minimum error sum of a mode and the search threshold.

In an exemplary embodiment, in the determining of the intra mode predictive search, the intra mode predictive search may be omitted when the determined minimum error sum of the inter mode exceeds the search threshold. In the determining of the intra mode prediction search, if the minimum error sum of the determined inter mode is less than or equal to the search threshold, the intra mode is predicted and searched corresponding to the inter mode having the minimum error value. You can do

In the intra mode prediction search, when the inter mode having the minimum error sum is 16x16, the intra mode prediction search predicts and searches the DC mode of I16MB (Intra 16x16 Macro Block Luminance) and the DC (directionless) mode of Intra 8x8 Chroma (Chrominance). The mode may be omitted, further comprising predicting and searching for an optimal intra mode. In addition, when the inter mode having the minimum error sum is 16x8, the method further includes predicting and searching for the optimal intra mode by omitting the I16MB DC mode, the HOR mode, and the intra 8x8 Chroma DC mode, and omitting the I4MB mode. It may be characterized by.

In addition, when the inter mode having the minimum error sum is 8x16, the method further includes predicting and searching for the optimal intra mode by omitting the I16MB DC mode, the VER mode, and the intra 8x8 Chroma DC mode, and omitting the I4MB mode. It may be characterized by.

In addition, when the inter mode having the minimum error sum is an 8x8 sub-block, the prediction search is performed in the DC mode of I16MB and the DC mode of the intra 8x8 Chroma. If the sub-block mode is 8x4, the DC of I4MB is used. And predictive search in HOR mode, predictive search in I4MB DC mode when the sub-block mode is any one of 8x8 and 4x4, and predictive search in I4MB DC mode and VER mode when the sub-block mode is 4x8. It may be characterized by further comprising the step.

According to another aspect of the present invention, by measuring the rate-distortion cost of the video signal macroblock received from the outside, comparing the rate-distortion cost of the received macroblock with the rate-distortion cost of the adjacent macro block A threshold determining unit determining a search threshold, an inter mode calculating unit measuring an error sum for each inter mode of the received macroblock, and a final inter having a minimum error sum by comparing the measured error sums for each inter mode A final inter mode determiner for determining a mode and an intra mode determiner for determining an intra mode predictive search according to the minimum error sum of the determined final inter mode and the search threshold; A predictive search apparatus can be provided.

In an exemplary embodiment, the intra mode determiner may omit the intra mode predictive search when the minimum error sum of the determined final inter mode exceeds the search threshold. The apparatus may further include an intra mode calculator configured to predict and search an intra mode corresponding to the final inter mode having the minimum error value.

The intra mode calculator may further include a 16x16 inter mode search predictor that predicts and searches an optimal intra mode by performing a predictive search in a DC mode of I16MB and a DC mode of an intra 8x8 Chroma, and omitting the I4MB mode. The intra mode calculator may further include a 16x8 inter mode search predictor configured to predict and search an optimal intra mode by performing a predictive search in a DC mode and a HOR mode of an I16MB, and a DC mode of an intra 8x8 Chroma, and omitting the I4MB mode. Can be.

The intra mode calculation unit may further include an 8x16 inter mode search predictor that predicts and searches an optimal intra mode by performing a predictive search using a DC mode of 16 MB, a VER mode, and a DC mode of an intra 8x8 Chroma, and omitting the I4 MB mode. Can be.

In addition, the intra mode calculation unit 8x8 sub-block for predictive searching in at least one mode of DC mode of I16MB, DC mode of Intra 8x8 Chroma, DC and HOR mode of I4MB, DC mode of I4MB, DC mode and VER mode of I4MB. The method may further include a (sub-block) inter mode search predictor.

Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view schematically showing the structure of a video compressor to which the present invention is applied.

Referring to FIG. 1, the video compressor includes a block unit unit 101, a block cosine transform unit DCT 103, a quantizer 105, a variable length encoder 107, a motion compensator 109, and an inverse. It includes a quantization and IDCT unit (inverse-DCT) 111, a motion estimator 113, a multiplexer 115, and further includes a subtractor 119 and an adder 117.

The block unit unit 101 is responsible for dividing the video signal input from the outside in units of frames in units of blocks. This is to set the macro block used for predicting the video signal in the motion estimator 113.

The block DCT (discrete cosine transform) unit 103 performs a discrete cosine transform (DCT) on a signal obtained by subtracting the block unit 101 and the motion compensator 109 from the subtractor 119. In charge. The discrete cosine converter is responsible for converting pixels into frequencies.

The quantizer 105 is responsible for quantizing the signal converted by the block DCT unit 103 into a digital signal, and the variable length encoder 107 converts the quantized signal from the quantizer 105 into a variable length encoding ( variable length coding) to generate a compressed stream for transmission to the outside.

The inverse quantization and IDCT unit (Inverse-DCT) 111 plays a role of re-quantizing and inverse DCT transforming the quantized signal in the quantizer 105 to reproduce the original frame.

The motion estimator 113 adds the video input signal received by the block unitization unit 101 and the signal received by the inverse quantization and the previous frame reproduced by the IDCT unit 111 and the motion compensation unit 109. This part estimates and predicts the motion by using the sum signal. The motion estimator 113 is responsible for performing an intra mode prediction search and an inter mode prediction search.

The motion compensator 109 combines the block unitization unit 101 and the subtraction unit 119 to subtract the compensated motion from the original frame.

The multiplexer 115 is the part to multiplex to transmit the compressed video stream.

2 is a diagram briefly showing a procedure of a video compressor to which the present invention is applied.

2, a video signal is first input to a video compressor (step 201). Thereafter, the input video signal is blocked, and motion is predicted in the frame based on the blocked video signal (step 203). For such motion prediction, a prediction search using an H.264 inter mode and an intra mode may be used.

After the prediction of the motion is completed, the discrete cosine transformer transforms the screen (step 205), and quantizes and encodes the converted screen to convert it to a dark-axis stream signal (step 207). When all of these steps are completed, the video signal is transmitted to the outside in a axial stream by multiplexing (step 209).

3 is a diagram schematically showing a procedure for predicting a conventional screen motion compared with the present invention.

In the screen compression using the H.264 method, the screen is divided into several macro blocks (MBs) for inter mode prediction and intra mode prediction. Therefore, first, the rate-distortion (R-D) cost of the inter mode is measured to determine the optimal inter mode, and intra mode prediction is performed according to the determined inter mode.

Thus, referring to Figure 3, first measure the rate-distortion cost for a 16x16 macroblock (301), then measure the rate-distortion cost for a 16x8 macroblock (303), and then the rate for the 8x16 macroblock- The distortion cost is measured (305) and then the rate-distortion cost for the 8x8 macroblock is measured (307).

The measurement of the rate-distortion cost according to each macro block is because each variable block formed in the inter mode has a 16x16 skip mode, a 16x16 mode, an 8x16 mode, a 16x8 mode, and an 8x8 mode. In particular, 8x8 mode, 8x4 mode, 4x8 mode, and 4x4 mode are present again as sub modes of the 8x8 mode.

The rate-distortion costs are all measured for each variable block possible in the inter mode, and then the optimal inter mode having the lowest value is determined by comparing the rate-distortion costs of each block (step 309).

Then, it is determined whether the prediction method is a Frext (Fidelity Range Extensions) method (step 311), and if the Frext method is performed, the prediction of the Luma (Luminance) 16x16 method is performed during intra mode prediction (step 313), and then the Luma 4x4 method is used. Predictive search (step 315), Chroma (Chrominance) 8x8 method predictive search (step 317) and Luma 8x8 method predictive search (step 319) are all performed to compare the respective search results to determine the optimal intra mode. (Step 321).

On the other hand, if the prediction method is not Frext method, Luma 16x16 prediction search (step 323), Luma 4x4 prediction prediction (step 325) and Chroma 8x8 prediction prediction (step 327) are performed during intra mode prediction. The predictive search values are compared to determine the optimal intra mode (step 329).

The final mode is then determined using each optimal mode of intra mode and inter mode (step 331), which is the final mode of the macroblock in which compression is performed (step 333).

In this case, a large amount of computation is required for the inter mode search, and in addition to the intra mode search, the computational amount is very large, and thus, there are many problems in real-time video compression.

4 is a diagram illustrating a macro block applied in the H.264 method.

Referring to FIG. 4, the position of each macro block in the full screen frame is described. Here, if MBkl is the macroblock 400 currently being processed, the processing of the macroblock in the frame is sequentially processed in the direction from the upper left to the lower right, so that a neighboring macro that occurs when processing the MBkl 400 block is processed. There may be three blocks on top of the processing block and one on the left. That is, the left block 407 having the (k, l-1) coordinate, the upper right block 405 having the (k-1, l + 1) coordinate, and the upper block having the (k-1, l) coordinate are It is. When processed in this manner, it is possible to compress the screen by comparing the values of neighboring macro blocks that have been processed in advance.

5 is a diagram schematically illustrating a configuration of a motion estimator for predicting screen motion according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the motion estimator 500 according to the present invention includes a threshold determiner 503, an inter mode calculator 505, a final inter mode determiner 507, an intra mode determiner 509, and an intra. The mode calculator 511 and the output unit 513 are included.

The threshold value determination unit 503 is responsible for determining the threshold value by using the rate distortion cost of the neighboring macro block, which is previously calculated, from the input macro block signal 501.

Details of this threshold determination will be described again with reference to FIG. 6.

The inter mode calculator 505 is responsible for calculating the inter mode of the input macro block signal 501. The inter mode calculator 505 calculates an error sum for each inter mode described in FIG. 8. This inter mode calculation method will be described in detail with reference to FIG. 6.

The final inter mode determiner 507 is responsible for determining an inter mode having a minimum error sum using the error sums of the inter modes calculated by the inter mode calculator 505.

The intra mode determiner 509 is responsible for determining whether to calculate the intra mode by comparing an error sum of the optimal inter mode determined by the inter mode determiner 507 with the threshold determined by the threshold value determiner 503.

The intra mode calculator 511 calculates the intra mode when the intra mode determiner 509 determines that the intra mode is to be calculated. The method of calculating the intra mode in the intra mode calculator 511 will be described in detail with reference to FIG. 7.

The output unit 513 outputs the compressed signal to the outside by performing intra mode and inter mode prediction searching on the macro block signal input by the above method.

6 is a diagram schematically illustrating a procedure for predicting screen motion according to an exemplary embodiment of the present invention.

Referring to Fig. 6, first, a search threshold is determined (step 601). The search threshold is an important part of deciding whether to do intra mode predictive search in a later step.

These search thresholds

[Equation 1]

Tkl = min RD Costij {RD cost (m, n) | (m, n) ∈ Neighbors},

Can be obtained from the formula

Tkl means search threshold and RD Cost means rate-distortion cost.

In addition, (m, n) means an index value for a neighboring block, and kl means a macroblock to be currently processed.

The position on the frame of each (M, n) value and kl value is described in FIG. 4. As described above with reference to FIG. 4, since the macro block is configured on the screen, when the search threshold is determined using the neighboring block, a sum of absolute error (SAD) according to the variable block of the inter mode of the current macro block is then determined. Calculate each.

First, measure the error sum for the 16x16 macroblock (step 603), then measure the error sum for the 16x8 macroblock (step 605), then measure the error sum for the 8x16 macroblock (step 607). The error sum for the 8x8 macroblock is then measured (step 609).

After measuring all the error sums for each variable block, the inter mode of the minimum error sum is determined (step 611).

It is then determined whether the minimum error sum has a value greater than the search threshold (step 613), and if the minimum error sum is not greater than the search threshold, the prediction search in intra mode is skipped (step 619), otherwise the higher step Intra mode is searched based on the minimum error sum inter mode determined in operation S615.

Then, the final mode is selected (step 617), which is determined as the mode of the final macroblock (step 621).

7 is a diagram illustrating an intra mode search method based on an inter mode according to an embodiment of the present invention.

Referring to FIG. 7, the reference step 615 of FIG. 6 is described in detail. First, an optimal inter mode is checked (step 701). This confirmation of the optimal inter mode is a step determined in reference step 611 of FIG.

Then, if the optimal inter mode step is 16x16, first perform a directionless (DC) mode of In16 16x16 Macro Block luminance (I16MB), then perform a DC mode of 8x8 Chroma (step 705). I4MB is skipped (SKIP) (step 707) to search for the optimal intra mode (step 733).

On the other hand, if the optimal inter mode step is 16x16, first perform a directionless (DC) mode of I16MB (step 703), then perform DC mode of 8x8 Chroma (step 705), and skip I4MB (SKIP). (Step 707) to search for an optimal intra mode (step 733).

On the other hand, if the optimal inter mode step is 16x8, first perform the directionless (DC) mode and the horizontal (HOR) mode at step I16MB (step 709), and then perform the DC mode of 8x8 Chroma (step 711). I4MB is skipped (SKIP) (step 713) to search for the optimal intra mode (step 733).

On the other hand, if the optimal inter mode step is 8x16, first perform I16MB of DC (directionless) mode and VER (vertical) mode (step 715), then perform 8x8 Chroma DC mode (step 727). I4MB is skipped (SKIP) (step 719) to search for the optimal intra mode (step 733).

Finally, if the optimal inter mode step has a submode of 8x8, first perform I16MB of DC (directionless) and VER (vertical) modes (step 715), then 8x8 Chroma of DC mode. (Step 727), the sub-block of the 8x8 mode is determined (step 725) and performed in DC mode and HOR mode in I4MB in 8x4 mode (step 727), and in I4MB in 8x8 mode or 4x4 mode. Mode (step 729), and in the 4x8 mode, performs the DC mode and the HOR mode (step 731).

After performing each mode, the optimal intra mode is searched for (step 733).

Comparing the intra mode prediction search in this manner with the conventional intra mode prediction search described in FIG. 3, it can be seen that the prediction mode of the intra mode is significantly reduced according to each inter mode. As a result, the amount of computation is reduced compared to the conventional method, and thus it may be very efficient for compressing a real-time video signal.

10 is a view for explaining an actual embodiment when the present invention is applied.

Referring to FIG. 10, first, when one frame 1000 of a video signal to be compressed is input, the frame 1000 is a frame 1010 divided into several macro blocks.

In this case, in the h.264 method, each macroblock is not divided into the same macroblock by the inter prediction method, and the size of the variable described in FIG. 7 is changed according to the luminance or the luminance of the image present in the video signal. Predictive search into blocks.

The inter prediction search variable blocks may be divided into the variable blocks described with reference to FIGS. 1011 and 1013 as described above with reference to FIG. 8, and the prediction mode of the intra mode is changed according to each block.

In the conventional method, as shown in the flowchart of FIG. 3, all the intra mode searching methods are performed regardless of the type of each block. However, in the present invention, as shown in the flowchart of FIG. The intra mode is determined in different ways according to the search threshold and the sum of the errors according to each variable block. That is, the intra prediction search is performed differently for the reference numeral 1011 block and the reference numeral 1013.

For example, the reference numeral 1011 has an 8x8 subblock 1021, and according to the 8x8 subblock path of FIG. 7, intra prediction only in the case of I16MB: DC mode, 8x8Chroma: DC mode, and I4MB: DC mode. Do a search.

In addition, the reference numeral 1013 has an 8x16 block 1023, and intra prediction may be performed only in the I16MB: DC mode, the HOR mode, and the 8x8Chroma: DC mode along the 8x16 path of FIG.

In this case, only three types of intra mode prediction searches are required for blocks 911 and 1013. However, if the conventional method is used, all of the modes described in FIG. 9 must be performed. You must perform a search. In the above example, it can be seen that the amount of calculation is reduced compared to the conventional method, which is very efficient for compressing a real-time video signal.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

According to the present invention, an intra mode predictive search method and apparatus for real-time video compression can be provided.

In addition, the present invention improves the performance of an H.264 video compression system in terms of bit rate and compression rate by speeding up intra mode prediction search among inter-mode prediction and intra-mode prediction techniques. Can be.

Claims (14)

In the intra mode prediction search method for real-time video compression, Determining a search threshold for the macro block received from the outside; Calculating an error sum (SAD) according to an inter mode of the received macroblock; Determining an inter mode having a minimum error sum as a final inter mode, Determining whether to perform an intra mode predictive search for the received macroblock by comparing the error sum of the last inter mode with the search threshold; Including; The search threshold is calculated according to the following formula, intra mode predictive search method for real-time video compression. Search threshold (Tkl) = min RD Costij {RD cost (m, n) | (m, n) ∈ Neighbors} Where kl represents a received macroblock, (m, n) represents an index value for a neighboring macroblock adjacent to the received macroblock, and RD Cost represents a rate-distortion cost. The method of claim 1, In the step of determining whether to perform an intra mode prediction search, Intra-mode predictive search method for real-time video compression, characterized in that the intra mode prediction search is not performed if the error sum of the last inter mode does not exceed the search threshold. The method of claim 1, In the step of determining whether to perform an intra mode prediction search, And performing an intra mode prediction search according to the final inter mode when the sum of errors of the last inter mode exceeds the search threshold. The method of claim 3, If the last inter mode is 16x16 The intra mode predictive search is a prediction search in a DC mode of an intra16x16 macro block luminance (I16MB) and a DC mode of an intra 8x8 chroma (Chrominance), and an I4MB mode is omitted. Mode prediction search method. The method of claim 3, If the last inter mode is 16x8 The intra mode predictive search is a prediction mode of the I16MB DC mode and HOR mode, intra 8x8 Chroma DC mode, characterized in that I4MB mode is omitted, intra mode predictive search method for real-time video compression. The method of claim 3, If the final inter mode is 8x16 The intra mode predictive search is a prediction mode of the I16MB DC mode and VER mode, intra 8x8 Chroma DC mode, characterized in that I4MB mode is omitted, intra mode predictive search method for real-time video compression. The method of claim 3, If the last inter mode is an 8x8 sub-block The intra mode predictive search is a predictive search in a DC mode of I16MB, a DC mode of an intra 8x8 Chroma. When the sub-block mode is 8x4, the predictive search is performed in a DC and a HOR mode of I4MB, and the sub-block mode is 8x8 and 4x4. In either case, the prediction mode is searched in the DC mode of I4MB, and the prediction mode is searched in the DC mode and the VER mode of I4MB when the sub-block mode is 4x8. In the intra mode prediction search device for real-time video compression, A threshold determination unit that determines a search threshold for the macro block received from the outside; An inter mode calculator for calculating an error sum (SAD) according to the inter mode of the received macro block; A final inter mode determiner configured to determine an inter mode having a minimum error sum as a final inter mode; An intra mode determiner which determines whether to perform an intra mode prediction search on the received macro block by comparing the error sum of the last inter mode with the search threshold; Including; The search threshold is calculated according to the following formula, intra mode predictive search device for real-time video compression. Search threshold (Tkl) = min RD Costij {RD cost (m, n) | (m, n) ∈ Neighbors} Where kl represents a received macroblock, (m, n) represents an index value for a neighboring macroblock adjacent to the received macroblock, and RD Cost represents a rate-distortion cost. The method of claim 8, The intra mode determiner Intra-mode predictive search for real-time video compression, characterized in that the intra mode predictive search is not performed if the error sum of the last inter mode does not exceed the search threshold. The method of claim 8, The intra mode determiner And performing an intra mode prediction search according to the final inter mode when the sum of errors of the last inter mode exceeds the search threshold. The method of claim 10, The intra mode calculation unit, If the last inter mode is 16x16 The intra mode prediction search is a prediction search in the DC mode of the Intra 16x16 Macro Block Luminance (I16MB) and the DC mode of the Intra 8x8 Chroma (Chrominance), and the I4MB mode is omitted. Mode predictive retrieval device. The method of claim 10, The intra mode calculation unit, If the last inter mode is 16x8 The intra mode predictive search is an intra mode predictive search apparatus for real-time video compression, characterized in that predictive search in the DC mode and HOR mode of the I16MB, DC mode of the intra 8x8 Chroma, omitting the I4MB mode. The method of claim 10, The intra mode calculation unit, If the final inter mode is 8x16 The intra mode predictive search is a prediction mode of the I16MB DC mode and VER mode, intra 8x8 Chroma DC mode, characterized in that I4MB mode is omitted, intra mode predictive search device for real-time video compression. The method of claim 10, The intra mode calculation unit, If the final inter mode is an 8x8 sub-block, at least one of DC mode of I16MB, DC mode of Intra 8x8 Chroma, DC and HOR mode of I4MB, DC mode of I4MB, DC mode of I4MB and VER mode 8x8 sub-block inter mode search prediction unit for performing prediction search in one mode Intra prediction prediction device for real-time video compression further comprising a.

Images