JP2010509800A - Video encoding / decoding method and apparatus using motion vector tracking - Google Patents

Abstract

  A video encoding method and apparatus using motion vector tracking is disclosed. The video encoding method and apparatus determine a corresponding region of a reference picture using motion information of a current picture, and determine a corresponding region of another reference picture using motion information of the determined corresponding region of the reference picture. A plurality of reference pictures to be used for prediction of the current picture are determined through a motion vector tracking process to be determined, and a prediction value of the current picture is generated through a weighted sum of the determined plurality of reference pictures. It is characterized by that.

Description

  The present invention relates to video predictive encoding and decoding, and more specifically, determines a plurality of reference pictures by continuously tracking a motion vector path of a current picture, and determines a plurality of determined plurality of pictures. The present invention relates to a video encoding method and apparatus for predictively encoding a current picture using a reference picture, and a decoding method and apparatus thereof.

  When encoding a moving picture, compression is performed by removing spatial redundancy and temporal redundancy in the video sequence. In order to remove temporal redundancy, another picture located in front of or behind the current coded picture is used as a reference picture, and a reference picture area similar to the current coded picture area is used. A search is performed to detect a motion amount between corresponding regions of a picture that is currently encoded and a reference picture, and a predicted video obtained by performing a motion compensation process based on the motion amount and a video that is currently encoded Encode the difference.

  A video picture is coded in one or more slice units. Here, one slice may include only one macroblock, and one entire picture may be encoded into one slice. H. According to the H.264 standard, a video picture is an I (Intra) slice that is encoded only by prediction within the screen, a P (Predictive) slice that is encoded by prediction using a video sample of one reference picture, And B (Bi-predictive) slices encoded by prediction using video samples of two reference pictures.

  Conventionally, in MPEG-2 or the like, bi-directional prediction using one picture immediately before the current picture and one picture appearing immediately after as a reference picture has been performed. H. In H.264 / AVC (Advanced Video Coding), the concept of bi-directional prediction is extended, and it is not limited to the picture immediately before and immediately after the current picture, and any two pictures can be used as reference pictures regardless of before and after. Available. In this way, a picture predicted using any two pictures as reference pictures regardless of before and after is defined as a bi-predictive picture (hereinafter referred to as “B picture”).

  FIG. 2 is a diagram for describing a prediction process of a block included in a current picture encoded as a B picture by H.264 / AVC. FIG. H. According to H.264 / AVC, a block in a B picture uses two reference pictures A and B in the same direction as in MB 1 or two reference pictures in different directions as in MB 2 B and C are used, an area sampled in two different areas within the same reference picture A, such as MB 3, or any reference picture B or D, such as MB4 or MB5 Predicted using only

  In general, the encoding efficiency of video data coded as a B picture is even higher than the encoding efficiency of video data coded as an I picture or P picture. This is because the B picture has prediction data similar to the current video data compared to the P picture that uses one reference picture to use two reference pictures and the I picture that uses prediction in the screen. In addition to a high possibility of being generated, in the case of a B picture, in order to use an average value of two reference pictures as prediction data, even if an error occurs between pictures located before and after in time, time This is because the average value flattened in the direction is used as the predicted value, and there is a case where the coding distortion hardly occurs visually as if a kind of low frequency filtering is performed.

  Compared to the P picture, if a larger number of reference pictures are used so that a B picture using two reference pictures has a larger encoding efficiency, the encoding efficiency can be improved. Regardless, when motion prediction and compensation are performed for each reference picture, the conventional video compression standard proposal uses only up to two reference pictures due to the limit of the amount of calculation.

  The present invention has been devised to solve the above-described problems, and uses more reference pictures for prediction of the current block by tracking the motion vector path of the reference picture of the current block. Thus, it is an object of the present invention to provide a video encoding method and apparatus for improving the encoding efficiency, and the decoding method and apparatus.

  In order to solve the above technical problem, the video coding method according to the present invention is used for prediction of the current block by tracking the motion vector path of the corresponding region of the reference picture referenced by the current block. Determining a corresponding region of a plurality of reference pictures; generating a predicted block of the current block by obtaining a weighted sum of the corresponding regions of the plurality of reference pictures; and the current block and the predicted block; And a step of encoding the difference.

  A video encoding apparatus according to the present invention tracks a motion vector path of a corresponding area of a reference picture referenced by a current block, thereby determining a corresponding area of a plurality of reference pictures used for prediction of the current block A determination unit; a weighted prediction unit that generates a prediction block of the current block by obtaining a weighted sum of corresponding regions of the plurality of reference pictures; and an encoding that encodes a difference between the current block and the prediction block And a section.

  According to the video decoding method of the present invention, the prediction mode information included in the input bitstream is read, and the prediction mode of the current block to be decoded is determined. For the current block predicted using the corresponding area, the corresponding area of the reference picture indicated by the motion vector of the current block included in the bitstream and the motion vector path of the corresponding area of the reference picture are tracked. And determining a corresponding area of a plurality of reference pictures used for prediction of the current block, and generating a predicted block of the current block by obtaining a weighted sum of the corresponding areas of the plurality of reference pictures Performing the generated prediction block and the current stream included in the bitstream. And the difference value of the block and the predicted block is added, characterized in that it comprises a step of decoding the current block.

  The video decoding apparatus according to the present invention interprets prediction mode information included in an input bitstream and determines a prediction mode of a current block to be decoded, and a plurality of discs as a result of the determination. For the current block predicted using a corresponding area of a reference picture, a corresponding area of a reference picture indicated by a motion vector of the current block included in the bitstream, and a motion vector path of the corresponding area of the reference picture By tracking a current picture block, a reference picture determination unit that determines a corresponding area of a plurality of reference pictures used for prediction of the current block, and a weighted sum of the corresponding areas of the plurality of reference pictures. A weighted prediction unit for generating a prediction block of the block, the generated prediction block, and Tsu said equipped Doo stream current block and the difference value between the prediction block is added, characterized in that it comprises a decoding unit to decode the current block.

  According to the present invention, prediction efficiency and encoding efficiency can be improved by performing predictive encoding using a larger number of reference pictures.

H. 2 is a diagram for describing a prediction process of a block included in a current picture encoded as a B picture by H.264 / AVC. FIG. FIG. 6 is a diagram illustrating an example of a process of determining a plurality of reference pictures used for prediction of a current picture by a video encoding method according to the present invention. FIG. 10 is a diagram illustrating another example of a process of determining a plurality of reference pictures used for prediction of a current picture by the video encoding method according to the present invention. FIG. 10 is a diagram illustrating still another example of a process of determining a plurality of reference pictures used for prediction of a current picture by the video encoding method according to the present invention. 1 is a block diagram illustrating a video encoding apparatus according to the present invention. FIG. 6 is a block diagram illustrating a specific configuration of a motion compensation unit 504 in FIG. 5. H. 2 is a diagram illustrating blocks of various sizes used in H.264 / MPEG-4AVC variable block size motion prediction. FIG. It is the image which illustrated an example of the image by which variable block motion prediction was carried out. FIG. 10 is a diagram illustrating a process for determining a corresponding area of another reference picture that is referenced by a corresponding area of a reference picture divided based on a motion block boundary by the video encoding method according to the present invention. FIG. 6 is a diagram for explaining another example of a process for determining a corresponding area of another reference picture to which a corresponding area of a reference picture divided based on a motion block boundary is referred by the video encoding method according to the present invention; is there. FIG. 10 is a diagram for explaining a process of calculating a weight given to a corresponding region of a reference picture in the video encoding method according to the present invention. 5 is a flowchart illustrating a video encoding method according to the present invention. It is the block diagram which showed the structure of the video decoding apparatus by this invention. 5 is a flowchart illustrating a video decoding method according to the present invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  The video encoding method according to the present invention uses a motion vector of a reference picture indicated by a motion vector of a current picture, and continuously tracks a corresponding area of another reference picture, so that a plurality of images used for prediction of the current picture are used. One reference picture is determined, and a prediction value of the current picture is generated through a weighted sum of the determined reference pictures. First, a process of determining a plurality of reference pictures applied to the video coding method and apparatus according to the present invention will be described with reference to FIGS.

  FIG. 2 is a diagram illustrating an example of a process of determining a plurality of reference pictures used for prediction of the current picture by the video encoding method according to the present invention.

General motion prediction is performed on a block (hereinafter referred to as “current block”) 21 to be encoded of the current picture, the corresponding region 22 of the reference picture 1 most similar to the current block 21, and the current block It is assumed that the motion vector MV ₁ indicating the position difference between the reference area 21 and the corresponding area 22 of the reference picture 1 is determined. Further, it is assumed that the current block 21 is a motion block that refers to only one reference picture included in the P picture. However, the present invention is not limited to a motion block having one motion vector, as shown in FIG. 2, but by tracking each motion vector with respect to a motion block in a B picture having two motion vectors. Can also be applied.

Referring to FIG. 2, a motion vector MV ₁ generated as a result of motion prediction for the current block 21 indicates an area in the reference picture 1 that has the least error from the current block 21. According to the prior art, the value of the corresponding region 22 of the reference picture 1 is determined as the predicted value of the current block 21, and a residue that is the difference between the determined predicted value and the original pixel value of the current block 21 is encoded. Turn into.

The video encoding method according to the present invention uses not only the corresponding area of the reference picture indicated by the motion vector of the current block but also the motion information of the corresponding area of the reference picture, as in the prior art, Corresponding regions of other reference pictures used for prediction are used for prediction of the current block. Referring to FIG. 2, the motion vector MV _{2 included} in the corresponding region 22 of the reference picture 1 corresponding to the current block 21 is used, and the corresponding region 23 of the reference picture 2 used for prediction of the corresponding region 22 of the reference picture 1. To decide. Further, by using the motion vector MV ₃ of reference pictures 2 corresponding region 23, to determine the corresponding area 24 of the reference picture 3, which is used to predict the reference picture 2 corresponding region 23. The motion vector MV _n of the corresponding area 25 of the reference picture n-1 is used to determine the corresponding area 26 of the reference picture A used for prediction of the corresponding area 25 of the reference picture n-1. As will be described later, the process of continuously tracking the corresponding area of the reference picture pointed to by the motion vector of the current block and the corresponding area of the other reference picture pointed to by the motion vector of the corresponding area of the reference picture is limited to the intra-predicted block. Or a reference picture in which the size of the corresponding region included in the intra prediction block is equal to or greater than a predetermined critical value.

As described above, according to the present invention, not only the corresponding area 22 of the reference picture 1 indicated by the motion vector MV ₁ of the current block 21 but also the reference indicated by the motion vector MV ₂ of the corresponding area 22 of the reference picture 1 as in the conventional technique. The motion vector path of the corresponding area of the reference picture indicated by the motion vector of the current block 21 is tracked, such as the corresponding area 24 of the reference picture 3 indicated by the motion vector MV3 of the corresponding area 23 of the picture 2 and the corresponding vector 23 of the reference picture 2. The prediction block of the current block 21 is generated by multiplying the corresponding areas of the plurality of reference pictures determined by performing a predetermined weighting and adding them.

FIG. 3 is a diagram illustrating another example of a process of determining a plurality of reference pictures used for prediction of a current picture by the video encoding method according to the present invention. In FIG. 3, it is assumed that I ₀ is an I picture, P ₁ and P ₅ are P pictures, and B ₂ and B ₃ are B pictures. Hereinafter, the present invention, the process will be described for determining a plurality of reference pictures corresponding area used for prediction of the current block 31 provided in B ₂ picture.

Referring to FIG. 3, it is assumed that the current block 31 included in the B ₂ picture has two motion vectors MV ₁ and MV ₂ as a result of general motion prediction. When the current block to be encoded has two motion vectors, such as the current block 31 included in the B ₂ picture, the motion vector path of the corresponding region of the reference picture is tracked for each motion vector. The corresponding region of the plurality of reference pictures is determined by performing the process. That is, the motion information of the corresponding region 33 of the P ₁ picture indicated by the first motion vector MV ₁ of the current block 31 is used, and the correspondence of the other reference picture I ₀ used for the prediction of the corresponding region 33 of the P ₁ picture. Region 34 is determined. In the case of an I ₀ picture, the tracking process is interrupted because it is an I picture composed only of intra-predicted blocks, and the corresponding region 34 does not have motion information.

Similarly, when the second motion vector MV ₂ is indicated _{B 3} picture corresponding region 32 of the current block 31, the _{B 3} picture is a B-picture, the corresponding area 32 has two motion vectors. Again, one of the two motion vectors corresponding area 32 of the B ₃ picture has to track the left motion vector, B ₃ corresponding region 41 of P ₁ picture is used to predict the picture of the corresponding region 32 , The corresponding area 42 of the I ₀ picture used for the prediction of the corresponding area 41 of the P ₁ picture is determined. Also, of the two motion vectors corresponding area 32 of the B ₃ picture has to track the right motion vectors, determining a corresponding area 38 of the P ₅ picture is available of the B ₃ picture prediction of a corresponding region 32 . As described above, the tracking process using the right motion vector among the two motion vectors included in the corresponding region 32 of the B ₃ picture is the same as the intra prediction picture having no motion information like the I ₀ picture. Or the reference picture in which the size of the area included in the intra-predicted block of the corresponding area is equal to or larger than a predetermined size is performed.

  As described above, the corresponding areas 32, 33, 34, and 38 of the plurality of reference pictures determined by tracking the motion vector of the current block 31 are respectively multiplied by predetermined weights, and then added to each other. Generate a prediction block.

  FIG. 4 is a diagram illustrating still another example of a process of determining a plurality of reference pictures used for prediction of the current picture by the video encoding method according to the present invention. FIG. 4 is similar to the motion vector tracking process described above with reference to FIG. 3 except that only a picture encoded before the current picture is used to reduce the delay during video encoding. . H. In H.264 / AVC, two reference pictures in any direction can be used without being limited to the picture immediately before or after the current picture, so that only the forward picture is used as shown in FIG. In some cases, predictive encoding is performed.

Referring to FIG. 4, by tracking the _two motion vectors MV ₁ and MV ₂ of the current block 43, the corresponding regions 44, 45, 46, 47, 48, 49, 50, 51 can be determined.

  As described above, the video coding method and apparatus according to the present invention uses not only the corresponding area of the reference picture indicated by the motion vector of the current block but also the motion information included in the corresponding area of the reference picture, Corresponding regions of other reference pictures used for region prediction are used for prediction of the current block. When the corresponding area of the current block or reference picture has two motion vectors, the corresponding area of the reference picture is determined by tracking each motion vector.

  FIG. 5 is a block diagram illustrating a video encoding apparatus according to the present invention. In the following, for the convenience of explanation, H.C. The video coding apparatus according to the H.264 / AVC will be mainly described, but the video coding apparatus according to the present invention can be applied to other video coding schemes using motion prediction and compensation. Referring to FIG. 5, the video encoding apparatus 500 includes a motion prediction unit 502, a motion compensation unit 504, an intra prediction unit 506, a conversion unit 508, a quantization unit 510, a rearrangement unit 512, an entropy coding unit 514, and an inverse. A quantization unit 516, an inverse transformation unit 518, a filter 520, a frame memory 522, and a control unit 525 are provided.

  The motion prediction unit 502 divides the current picture into blocks of a predetermined size, and within the predetermined search area range of the reference picture restored after being previously encoded and stored in the frame memory 522, Motion prediction for searching the most similar region is performed, and a motion vector that is a position difference between the current block and the corresponding region of the reference picture is output.

  The motion compensation unit 504 generates the prediction value of the current block using the corresponding region information of the reference picture indicated by the motion vector. In particular, as described above, the motion compensation unit 504 according to the present invention determines a corresponding region of a plurality of reference pictures by continuously tracking a motion vector included in the current block, and determines the correspondence of the determined reference pictures. A prediction value of the current block is generated through the weighted sum of the regions. A specific configuration and operation of the motion compensation unit 504 according to the present invention will be described later.

  The intra prediction unit 506 performs intra prediction for searching for the prediction value of the current block in the current picture.

  If a prediction block of the current block is generated by inter prediction, intra prediction, or a prediction method using a corresponding region of a plurality of reference pictures according to the present invention, a remainder that is an error value between the current block and the prediction block is generated. Then, the generated remainder is converted into the frequency domain by the conversion unit 508 and quantized by the quantization unit 510. The entropy coding unit 514 encodes the quantized residue and outputs a bit stream.

  The picture quantized to obtain the reference picture is further restored by the inverse quantization unit 516 and the inverse transform unit 518. The current picture restored in this way passes through a filter 520 that performs deblocking filtering, and is then stored in the frame memory 522 and used when predicting the next picture.

  The control unit 525 controls each component of the video encoding device 500 and determines the prediction mode of the current block. Specifically, the control unit 525 determines the cost of a general inter prediction or intra predicted block and the current block, and the block and current block predicted using corresponding regions of a plurality of reference pictures according to the present invention. Are compared, and the prediction mode having the minimum cost is determined. Here, the cost calculation may be performed by various methods. Examples of cost functions used include SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSD (Sum of Squared Difference), MAD (Mean of Absolute Difference), and Lagrange function. The SAD is a value obtained by taking the absolute value of each 4 × 4 block prediction error value and combining the values. SATD is a value obtained by taking the absolute value of a coefficient generated by applying a Hadamard transform to the prediction error value of each 4 × 4 block. The SSD is a value obtained by squaring the prediction error value of each 4 × 4 block prediction sample, and the MAD is an average value obtained by taking an absolute value from the prediction error value of each 4 × 4 block prediction sample. is there. The Lagrangian function is a new function created by including the bit stream length information in the cost function.

  FIG. 6 is a block diagram showing a specific configuration of the motion compensation unit 504 of FIG. Referring to FIG. 6, the motion compensation unit 600 includes a reference picture determination unit 610 and a weighted prediction unit 620.

  The reference picture determination unit 610 determines a corresponding region of the reference picture using the motion vector of the current block generated by the motion prediction unit 502, and tracks the path of the motion vector included in the determined corresponding region of the reference picture. As a result, a corresponding area of a plurality of reference pictures used for prediction of the current block is determined.

  The weighted prediction unit 620 generates a predicted block of the current block by obtaining a weighted sum of the corresponding areas of the determined plurality of reference pictures. Specifically, the weighted prediction unit 620 adds the weighting calculation unit 621 that determines the weighting of the corresponding region of the plurality of reference pictures and the weighting of the corresponding region of the plurality of reference pictures, and then adds the weighting to the current block. A prediction block generation unit 622 that generates a prediction block is provided.

  Hereinafter, a process in which the reference picture determination unit 610 determines corresponding areas of a plurality of reference pictures used for prediction of the current block will be described in detail.

  FIG. FIG. 8 is a diagram illustrating blocks of various sizes used in the H.264 / MPEG-4AVC variable block size motion prediction, and FIG. 8 illustrates an example of a variable block motion predicted video. As shown in FIG. 7, the macroblock may be divided in four ways. That is, the macroblock may be divided into one 16 × 16 macroblock partition, two 16 × 8 partitions, two 8 × 16 partitions, or four 8 × 8 partitions and motion predicted. If the 8 × 8 mode is selected, the four 8 × 8 sub-macroblocks in the macroblock can be divided by four methods. That is, when the 8 × 8 mode is selected, each 8 × 8 block has one 8 × 8 sub-macroblock partition, two 8 × 4 sub-macroblock partitions, two 4 × 8 sub-macroblocks, The partition is divided into one of four 4 × 4 sub-macroblock partitions. Within each macroblock, a large number of combinations of such partitions and sub-macroblocks are possible. The method of dividing a macroblock into sub-blocks of various sizes is called tree structured motion compensation.

  Referring to FIG. 8, in the video, a low energy block is predicted to move to a large size partition, and a high energy block is predicted to move to a small size partition. In describing the present invention, such a tree structure motion compensation is used, and a boundary between each motion block dividing the current picture is defined as a motion block boundary.

  As described above, according to the video coding method of the present invention, tracking is performed using a motion vector included in a corresponding region of a reference picture in order to determine a corresponding region of the reference picture used for prediction of the current block. . However, as shown in FIG. 8, the reference picture is divided into motion blocks of various sizes, so that the corresponding area of the reference picture corresponding to the current block exactly matches one motion block. Instead, it may be formed over a plurality of motion blocks. In such a case, there are a plurality of motion vectors in the corresponding region of the reference picture. In this manner, a process of tracking a motion vector path when a plurality of motion vectors exist in the corresponding region of the reference picture will be described.

  FIG. 9 is a diagram illustrating a process for determining a corresponding area of another reference picture that is referenced by a corresponding area of a reference picture divided based on a motion block boundary by the video encoding method according to the present invention. is there.

Referring to FIG. 9, the corresponding area 91 of the reference picture 1 indicated by the motion vector MV ₁ of the current block 90 exists over a plurality of motion blocks (BLOCK). That is, the corresponding region 91 of the reference picture 1 corresponding to the current block 90 is not matched with any one motion block, but exists over the block A, the block B, the block C, and the block D. In such a case, the reference picture determination unit 610 divides the corresponding area 91 of the reference picture 1 along the motion block boundary of the reference picture 1, and sub-corresponding areas a, b, c, d of the divided reference picture 1 The corresponding region between the reference picture 2 and the reference picture 3 indicated by the motion vector of each motion block of the reference picture 1 having the above is determined. That is, the reference picture determination unit 610 determines the corresponding area a′93 of the reference picture 2 using the motion vector MV _a that the block A to which the divided a area belongs, and the motion that the block B to which the b area belongs has A corresponding area b′94 of the reference picture 2 is determined using the vector MV _b , a corresponding area c′96 of the reference picture 3 is determined using the motion vector MV _c of the block C to which the c area belongs, and d The corresponding area d′ 95 of the reference picture 3 is determined using the motion vector MV _d of the block D to which the area belongs.

  FIG. 9 illustrates a case where blocks A, B, C, and D partially including the corresponding area 91 of the reference picture 1 corresponding to the current block 90 refer to the reference picture 2 and the reference picture 3. Similarly, when the reference pictures referenced by A, B, C, and D are changed, the motion field information that the blocks A, B, C, and D have, that is, the blocks A, B, C, and D Using the motion vector and the reference picture information, it is possible to determine the corresponding area of another reference picture corresponding to the corresponding area of the divided reference picture 1.

  FIG. 10 illustrates another example of a process for determining a corresponding area of another reference picture that is referenced by a corresponding area of a reference picture divided based on a motion block boundary by the video encoding method according to the present invention. FIG. Referring to FIG. 10, when the corresponding area 100 of the reference picture for the current block is partially included in the blocks A, B1, B2, C, and D, as described above, the corresponding area 100 is included in the reference picture. Dividing along the motion block boundary, the motion field information of the block to which each of the divided corresponding areas a, b1, b2, c, d belongs is used to determine the corresponding areas of other reference pictures. That is, the reference picture determining unit 610 uses the motion field information of the block A to which the area a belongs, determines the corresponding area of another reference picture corresponding to the area a, and determines the motion field information of the block B1 to which the area B1 belongs. A corresponding region of another reference picture corresponding to the b1 region is determined, a motion field information of the block B2 to which the b2 region belongs is used, a corresponding region of another reference picture corresponding to the b2 region is determined, and c Using the motion field information of the block C to which the area belongs, determining the corresponding area of another reference picture corresponding to the c area, and using the motion field information of the block D to which the d area belongs, The corresponding area of the reference picture is determined.

  The reference picture determination process is performed not only when determining the corresponding area of another reference picture from the corresponding area of the reference picture indicated by the motion vector of the current block, but also from the determined corresponding area of the other reference picture. The same can be applied when determining other reference pictures. That is, the tracking process using a motion vector can be continuously performed when the corresponding region includes a motion block having motion vector information. However, if the corresponding region belongs to all intra-predicted blocks, or if the size of the corresponding region belonging to the intra-predicted block is greater than or equal to a predetermined critical value, the reference corresponding to the current correspondence is performed without further tracking. Tracking can be done only for pictures. For example, referring again to FIG. 9, if all of the blocks A, B, C, and D to which the corresponding area 91 of the reference picture 1 corresponding to the current block 90 belongs are intra prediction blocks, further tracking is performed. Without performing the above, only the corresponding area 91 of the reference picture 1 is used for the prediction of the current block 90. If the blocks A, B, and C are motion blocks having motion vectors, and only the block D is an intra-prediction block, the reference is given when the size of the corresponding region d belonging to the block D is equal to or greater than a predetermined critical value. A value obtained by multiplying the corresponding area 91 of the picture 1 by a predetermined weight is used for prediction of the current block 90. The process of determining whether or not to perform the tracking process is the same for the corresponding areas of other reference pictures determined from the reference picture.

On the other hand, when a part of the corresponding region is included in the intra prediction block, but the size of the corresponding region included in the intra prediction block is less than a predetermined critical value, the corresponding region of another reference picture is determined subsequently. Perform the tracking process. At this time, a motion vector included in a motion block around the intra prediction block is used, a virtual motion vector is assigned to the intra prediction block, and a corresponding region of another reference picture indicated by the assigned virtual motion vector can be determined. In the above example, the blocks A, B, and C are motion blocks having the motion vectors MV _a , MV _b , and MV _c , and only the block D is an intra prediction block, and the width of the corresponding region d belonging to the block D is predetermined. Assuming that it is below the critical value, the tracking process continues uninterrupted. In such a case, for the corresponding areas a, b, and c belonging to the blocks A, B, and C, as described above, the motion vectors MV _a , MV _b , and MV _c of the blocks A, B, and C are used. The corresponding area of the reference picture is determined, and for the corresponding area d belonging to the block D, an intermediate value (median) or an average of the motion vectors MV _a , MV _b , and MV _{c included} in the peripheral motion blocks A, B, and C of the block D A value is assigned as a virtual motion vector of block D, and a corresponding area of another reference picture indicated by the virtual motion vector can be determined.

  Referring to FIG. 6 again, when the reference picture determination unit 610 tracks the motion vector path of the current block and determines the corresponding areas of the plurality of reference pictures, the weight calculation unit 621 Calculate the weight given to.

  The weighting calculation unit 621 uses the peripheral block pixels of the current block processed previously and the peripheral pixels of the corresponding area of the reference picture corresponding to the peripheral block pixels of the current block. The value that minimizes the difference between the predicted value of the neighboring pixels of the current block predicted through the weighted sum of the pixels and the neighboring pixel values of the current block is determined as the weight.

FIG. 11 is a diagram for explaining a process of calculating a weight given to a corresponding region of a reference picture in the video encoding method according to the present invention. In FIG. 11, the current block is D _t , the corresponding area of the reference picture (t−1) corresponding to the current block D _t is D _t−1 , and the corresponding areas D _t−1 are divided areas a, b, and c. , D corresponding to the reference picture (t-2) corresponding to D _{t-2, a} , D _{t-2, b} , D _{t-2, c} , D _{t-2, d} (collectively “D _t want to) _-2 ', it is assumed that the _{P t} current is the prediction block of the block _{D t.}

The weighting calculation unit 621 assigns weighting in units of reference pictures. That is, the weight calculation unit 621 assigns the same weight to corresponding regions belonging to the same reference picture. In FIG. 11, if the weight assigned to the corresponding area D _t-1 of the reference picture (t-1) is α and the weight assigned to the corresponding area D _t-2 of the reference picture (t-2) is β. , the predicted block _{P t} of the current block _{D t,} as in the following equation (1), and the corresponding region _{D t-1} of the reference picture (t-1), the reference picture (t-2) corresponding region _{D t -2} is calculated as a weighted sum with _-2 .

参照ピクチャの対応領域に付与される重み付けα及びβは、多様なアルゴリズムを介して決定されうる。本発明では、現在ブロックＤ_ｔと予測ブロックＰ_ｔとの誤差を最小にする重み付けを利用する。現在ブロックＤ_ｔと予測ブロックＰ_ｔとの誤差（ＳＳＥ：Sum of Squared Error）は、次の式（２）の通りである。

The weighting α and β given to the corresponding region of the reference picture can be determined through various algorithms. In the present invention, weighting that minimizes the error between the current block D _t and the prediction block P _t is used. An error (SSE: Sum of Squared Error) between the current block D _t and the prediction block P _t is expressed by the following equation (2).

重み付けα及びβは、ＳＳＥをα及びβに対して偏微分したとき、０になる次の式（３）の偏微分方程式を計算することによって決定されうる。

The weights α and β can be determined by calculating the partial differential equation of the following equation (3) that becomes 0 when the SSE is partially differentiated with respect to α and β.

式（３）の偏微分方程式を計算するために、現在ブロックの周辺ブロックの画素と、現在ブロックの周辺ブロックの画素に対応する参照ピクチャの対応領域の周辺画素とを利用する。これは、復号化部で以前に復号化された現在ブロックの周辺画素情報を利用して重み付けを決定することによって、別途に現在ブロックの予測に利用された重み付けを伝送せずとも、以前に処理された周辺画素データを基に重み付けを決定できるようにするためである。従って、本発明では、参照ピクチャの対応領域に割り当てられる重み付けを別途に伝送せずに、符号化部と復号化部とで以前に処理されたデータを利用して重み付けを計算して利用できるように、現在ブロックの周辺ブロックの画素と、現在ブロックの周辺ブロックの画素に対応する参照ピクチャの対応領域の周辺画素とを利用する。

In order to calculate the partial differential equation of Equation (3), the pixels of the peripheral block of the current block and the peripheral pixels of the corresponding region of the reference picture corresponding to the pixels of the peripheral block of the current block are used. This is because the weighting is determined by using the peripheral pixel information of the current block previously decoded by the decoding unit, so that the weighting used for prediction of the current block is not transmitted separately. This is because the weighting can be determined based on the peripheral pixel data. Therefore, according to the present invention, the weight assigned to the corresponding region of the reference picture can be used by calculating the weight using the data previously processed by the encoding unit and the decoding unit without separately transmitting the weight. In addition, the peripheral block pixels of the current block and the peripheral pixels of the corresponding area of the reference picture corresponding to the peripheral block pixels of the current block are used.

To also refer to FIG. 11, a corresponding region _{D t-1} of the reference picture (t-1), the reference picture (t-2) and a corresponding region _{D t-2} using the prediction block of the current block _{D t} to calculate P _t, pixel _{N t} of neighboring blocks of the current block _{D t} is now considering the spatial position of the block _{D t,} the periphery of the corresponding region _{D t-1} of the reference picture (t-1) Pixels N _{t−1, a} , N _{t−1, b} , N _{t−1, c} and peripheral pixels N _{t−2, a} , N _t− of the corresponding region D _t−2 of the reference picture (t−2) _{2, b} , N _{t-2, c} . In this case, the pixel _{N t} of neighboring blocks of the current block _{D t,} the corresponding region of the corresponding region _{D t-1} of the peripheral pixels _{N t-1} and the reference picture of the reference picture (t-1) (t- 2) D t- predicted value N _{t 'SSE} between the peripheral block pixel which is predicted using the _second peripheral pixels N _t-2 is as the following equation (4).

重み付け計算部６２１は、前記式（４）の誤差値ＳＳＥをα及びβに対して偏微分した後、偏微分した値が０になるα及びβを決定する。

The weighting calculation unit 621 partially differentiates the error value SSE of the equation (4) with respect to α and β, and then determines α and β at which the partially differentiated value becomes zero.

  On the other hand, if α + β is normalized by α + β = 1 in equation (1), β = 1−α. If this is substituted into the equation (1), the following equation (5) is obtained. In this case, the SSE is as the equation (6).

式（６）の誤差値（ＳＳＥ）をαに対して偏微分し、

Partially differentiate the error value (SSE) of equation (6) with respect to α,

を満足するαの値は、次の式（７）の通りである。

The value of α satisfying is as in the following formula (7).

前述のように、対応領域それぞれの重み付けを別途に伝送せずとも、重み付けを決定できるようにするために、現在ブロックＤ_ｔの代わりに、以前に処理された周辺ブロックの画素Ｎ_ｔを利用し、Ｄ_ｔ−２の代わりにＮ_ｔ−２，Ｄ_ｔ−１の代わりにＮ_ｔ−１を利用する。

As described above, in order to be able to determine the weight without separately transmitting the weight of each corresponding region, the pixel N _t of the previously processed peripheral block is used instead of the current block D _t. N _t−2 instead of D _t−2 and N _t−1 instead of D _t−1 .

  On the other hand, the weight determination process described above assigns a weight to each reference picture and determines a weight that minimizes an error value between the current block and the prediction block even when more corresponding areas of reference pictures are used. Can be accomplished by

That is, the corresponding areas of n reference pictures (n is an integer) used for prediction of the current block D _t are D ₁ , D ₂ , D ₃ ,..., D _n , respectively, and the weight assigned to each corresponding area is W1, W2, W3, ..., if Wn, the prediction block _{P t} of the current block _{D t is} calculated using the _{P t = W1 * D 1 +} W2 * D 2 + W3 * D 3 + ... + Wn * D n Is done. Each weighting W1, W2, ..., Wn, after partially differentiating the respective weights as parametric the SSE is a square error value of the current block D _t and the prediction block P _t, the value obtained by partially differentiating becomes 0 value To be determined. As described above, as a condition for solving the partial differential equation, the pixels of the peripheral block of the current block and the peripheral pixels of the corresponding region of the reference picture corresponding to this are used.

  Referring to FIG. 6 again, the prediction block generation unit 622 generates the prediction block of the current block by adding the determined weight after multiplying the corresponding region of the reference picture as shown in Equation (1). .

  In the motion compensation unit 504 according to the present invention, a remainder, which is a difference between a prediction block predicted using a corresponding region of a plurality of reference pictures and a current block, is converted into a bitstream through transformation, quantization, and entropy coding processes. Is output as

  On the other hand, in the header of the bitstream encoded by the video encoding method according to the present invention, a 1-bit flag (flag) indicating that prediction is performed using corresponding regions of a plurality of reference pictures in units of blocks. Can be added. For example, “0” may indicate a bitstream encoded according to the prior art, and “1” may indicate a bitstream encoded according to the present invention.

  FIG. 12 is a flowchart illustrating a video encoding method according to the present invention. Referring to FIG. 12, in step 1210, the corresponding region of a plurality of reference pictures used for prediction of the current block is determined by tracking the motion vector path of the corresponding region of the reference picture to which the current block refers. As described above, when the corresponding region of the reference picture is separated using the motion block boundary, the motion vector of the motion block to which each separated corresponding region belongs is used, and the corresponding region of the other reference picture is determined. decide.

  In step 1220, weights to be applied to corresponding regions of a plurality of reference pictures are determined. As described above, the weighting of the corresponding area is performed by using the surrounding pixels of the current block and the surrounding pixels of the corresponding area of the reference picture corresponding to the surrounding pixels of the current block, and predicting the current block from the surrounding pixels of the corresponding area. Is determined to be a value that minimizes the difference in value between the peripheral pixels of the original and the original peripheral pixels.

  In operation 1230, a value obtained by multiplying the corresponding area of the reference picture and the calculated weight is added to generate a prediction block of the current block.

  In operation 1240, the remainder of the difference value between the current block and the prediction block is transformed, quantized, and entropy encoded to generate a bitstream.

  FIG. 13 is a block diagram showing a configuration of a video decoding apparatus according to the present invention. Referring to FIG. 13, a video decoding apparatus 1300 according to the present invention includes an entropy decoder 1310, a rearrangement unit 1320, an inverse quantization unit 1330, an inverse transform unit 1340, a motion compensation unit 1350, an intra prediction unit 1360, and a filter 1370. It has.

  The entropy decoder 1310 and the reordering unit 1320 receive the compressed bit stream, perform entropy decoding, and generate quantized coefficients. The inverse quantization unit 1330 and the inverse transform unit 1340 perform inverse quantization and inverse transform on the quantized coefficients, and extract transform coding coefficients, motion vector information, prediction mode information, and the like. Here, the prediction mode information includes a flag indicating whether or not the current block is encoded through a weighted sum using corresponding regions of a plurality of reference pictures by the video encoding method according to the present invention. Can be. As described above, the corresponding areas of the plurality of reference pictures used for decoding the current block can be determined using the motion vector information of the current block decoded in the same way as the video encoding method. There is no need to separately transmit the corresponding area information of the reference picture used for decoding the current block.

  The intra prediction unit 1360 generates a prediction block using the neighboring blocks of the current block decoded previously for the current block subjected to intra prediction encoding.

  The motion compensation unit 1350 has the same configuration and operation as the motion compensation unit 504 of FIG. That is, when the current block to be decoded is predictively encoded using the weighted sum of the corresponding regions of the plurality of reference pictures, the motion compensation unit 1350 calculates the motion vector of the current block included in the bitstream. The corresponding region of the reference picture that has been previously decoded is used to determine the corresponding region of the reference picture and calculate the weight to be assigned to the corresponding region of each reference picture. After multiplying by weighting, it adds together and generates a prediction block of the current block. As described above, the weighting of the corresponding region of the reference picture is determined using the peripheral pixels of the current block that have been decoded previously and the peripheral pixels of the corresponding region of the reference picture corresponding to the peripheral pixels of the current block. The

In the motion compensation unit 1350 and the intra prediction unit 1360, the generated prediction block is added to the error value D ′ _n between the current block and the prediction block extracted from the bit stream, and the restored video data uF ′ _n is generated. Is done. uF ′ _n passes through the filter 1370 and is finally decoded for the current block.

  FIG. 14 is a flowchart illustrating a video decoding method according to the present invention. Referring to FIG. 14, in step 1410, prediction mode information included in an input bitstream is read to determine a prediction mode of a current block to be decoded.

  If it is determined in step 1420 that the current block to be decoded is predicted using the corresponding regions of the plurality of reference pictures, the corresponding region of the reference picture indicated by the motion vector of the current block included in the bitstream; By tracking the motion vector path of the corresponding area of the reference picture, corresponding areas of a plurality of reference pictures used for prediction of the current block are determined.

  In step 1430, the weights applied to the corresponding regions of the plurality of reference pictures using the peripheral pixels of the current block previously decoded and the peripheral pixels of the corresponding region of the reference picture corresponding to the peripheral pixels of the current block. And a predicted block of the current block is generated by calculating a weighted sum of corresponding regions of a plurality of reference pictures.

  In operation 1440, the current block is decoded by adding the generated prediction block and a difference value between the current block and the prediction block included in the bitstream.

  The present invention can also be embodied as computer readable codes on a computer readable recording medium. Computer-readable recording media include all types of recording devices that can store data that can be read by a computer system. Examples of computer-readable recording media include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM, magnetic tape, floppy (registered trademark) disk, optical data storage device, etc. Also included are those embodied in the form of a carrier wave (for example, transmission via the Internet). Further, the computer-readable recording medium can be distributed in a computer system connected to a network, and computer-readable code can be stored and executed in a distributed manner.

  In the above, this invention was demonstrated centering on the desirable embodiment. Those skilled in the art to which the present invention pertains can understand that the present invention can be embodied in a modified form without departing from the essential characteristics of the present invention. Accordingly, the disclosed embodiments should be considered from an illustrative viewpoint rather than a limiting viewpoint. The scope of the present invention is defined by the terms of the claims, rather than the foregoing description, and all differences that are within the scope equivalent thereto are construed as being included in the present invention.

Claims (25)

In the video encoding method,
Determining a corresponding region of a plurality of reference pictures used for prediction of the current block by tracking a motion vector path of a corresponding region of a reference picture referred to by a current block of the current picture;
Generating a predicted block of the current block by obtaining a weighted sum of corresponding regions of the plurality of reference pictures;
Encoding a difference between the current block and the prediction block. Determining a corresponding region of the plurality of reference pictures,
Performing motion prediction on the current block and determining a corresponding region of a first reference picture corresponding to the current block;
Dividing a corresponding region of the first reference picture along a motion block boundary of the first reference picture;
2. The method of claim 1, further comprising: determining a corresponding area of a second reference picture indicated by a motion vector of a motion block of the first reference picture having the corresponding area of the divided first reference picture. The video encoding method described in 1. Determining a corresponding region of the second reference picture comprises:
Of the corresponding regions of the first reference picture divided along the motion block boundary of the first reference picture, the motion vector of the peripheral motion block of the intra prediction block is used for the corresponding region included in the intra prediction block. 3. The video encoding method according to claim 2, further comprising: determining a virtual motion vector of the intra prediction block, and determining a corresponding region of a second reference picture indicated by the determined virtual motion vector. The virtual motion vector of the intra prediction block is
4. The video encoding method according to claim 3, wherein one of average values or intermediate values of motion vectors of the peripheral motion blocks is used. Tracking the motion vector path of the corresponding region of the reference picture is
Determining the corresponding region of the second reference picture indicated by the motion vector of the current block up to the corresponding region of the nth reference picture indicated by the motion vector of the (n-1) th reference picture; 2. The video encoding method according to claim 1, wherein n is 3 or more, and the nth reference picture is a reference picture of an (n−1) th reference picture. Tracking the motion vector path of the corresponding region of the reference picture is
6. The video according to claim 5, wherein from the corresponding area of the nth reference picture to a reference picture whose corresponding area included in the intra-predicted block is greater than or equal to a predetermined critical value is performed. Encoding method. Generating a prediction block for the current block;
Determining weights of corresponding areas of the plurality of reference pictures;
The video of claim 1, further comprising: generating a prediction block for the current block by multiplying each corresponding region of the plurality of reference pictures by the weight and adding a result of each multiplication. Encoding method. The weight is
The previously processed pixels of the peripheral block of the current block and the peripheral pixels of the corresponding area of the reference picture corresponding to the pixels of the peripheral block of the current block are used to determine the peripheral pixels of the corresponding area of the reference picture. The method of claim 7, wherein a difference between a predicted value of a neighboring pixel of the current block predicted through a weighted sum and a pixel value of a neighboring pixel of the current block is determined to be a minimum. Video encoding method.   The method may further include inserting a predetermined flag indicating a block that has been predictively encoded using the plurality of reference pictures into a predetermined area of the bitstream generated as a result of the encoding. Item 2. The video encoding method according to Item 1. Determining a corresponding region of the plurality of reference pictures,
Of the first reference picture corresponding regions divided along the motion block boundary of the first reference picture, when the size of the region included in the intra prediction block is greater than or equal to a predetermined threshold value, the first reference picture correspondence Determining only the region as the corresponding region of the reference picture used for prediction of the current block;
Generating a prediction block of the current block comprises:
The video encoding method according to claim 1, wherein a value obtained by multiplying a corresponding area of the first reference picture by a predetermined weight is determined as a prediction block of the current block. In a video encoding device,
A reference picture determination unit for determining a corresponding region of a plurality of reference pictures used for prediction of the current block by tracking a motion vector path of a corresponding region of a reference picture to which a current block refers;
A weighted prediction unit that generates a predicted block of the current block by obtaining a weighted sum of corresponding regions of the plurality of reference pictures;
A video encoding apparatus comprising: an encoding unit that encodes a difference between the current block and a prediction block. The reference picture determination unit
The first reference picture corresponding to the first reference picture indicated by the motion vector of the current block is divided along a motion block boundary of the first reference picture, and the first reference picture comprises the divided first reference picture corresponding area. 12. The video encoding apparatus according to claim 11, wherein a corresponding area of the second reference picture indicated by the motion vector of the motion block of the picture is determined. The reference picture determination unit
Of the corresponding regions of the first reference picture divided along the motion block boundary of the first reference picture, the motion vector of the peripheral motion block of the intra prediction block is used for the corresponding region included in the intra prediction block. 13. The video encoding apparatus according to claim 12, wherein a virtual motion vector of the intra prediction block is determined, and a corresponding region of the second reference picture indicated by the determined virtual motion vector is determined. The virtual motion vector of the intra prediction block is
The video encoding apparatus according to claim 13, wherein one of average values or intermediate values of motion vectors of the peripheral motion blocks is used. The reference picture determination unit
The step of determining the corresponding region of the second reference picture indicated by the motion vector of the current block is performed up to the corresponding region of the nth reference picture indicated by the motion vector of the (n−1) th reference picture, where n 12. The video encoding apparatus according to claim 11, wherein is 3 or more, and the n-th reference picture is a reference picture of an (n−1) -th reference picture. The reference picture determination unit
16. The tracking according to claim 15, wherein the tracking is performed up to a reference picture in which the size of the corresponding area included in the intra-predicted block is equal to or greater than a predetermined critical value among the corresponding areas of the nth reference picture. The video encoding device described. The weighted prediction unit includes:
A weight calculation unit for determining weights of corresponding regions of the plurality of reference pictures;
The prediction block generation unit configured to generate a prediction block for the current block by multiplying each corresponding region of the plurality of reference pictures by the weight and adding the result of the multiplication. The video encoding device described. The weight calculation unit
The previously processed pixels of the peripheral block of the current block and the peripheral pixels of the corresponding area of the reference picture corresponding to the pixels of the peripheral block of the current block are used to determine the peripheral pixels of the corresponding area of the reference picture. The weighting is determined as a value that minimizes a difference between a predicted value of a neighboring pixel of the current block predicted through a weighted sum and a pixel value of a neighboring pixel of the current block. The video encoding device described. The encoding unit includes:
The predetermined flag indicating a block that is predictively encoded using the plurality of reference pictures is inserted into a predetermined region of the bitstream generated as a result of the encoding. Video encoding device. The reference picture determination unit
Of the first reference picture corresponding regions divided along the motion block boundary of the first reference picture, when the size of the region included in the intra prediction block is greater than or equal to a predetermined threshold value, the first reference picture correspondence Determining only the region as the corresponding region of the reference picture used for prediction of the current block;
The weighted prediction unit includes:
The video encoding apparatus according to claim 11, wherein a value obtained by multiplying a corresponding area of the first reference picture by a predetermined weight is determined as a prediction block of the current block. In the video decoding method,
Reading prediction mode information provided in the input bitstream and determining a prediction mode of a current block to be decoded;
As a result of the determination, for the current block predicted using the corresponding areas of a plurality of reference pictures, the corresponding area of the reference picture indicated by the motion vector of the current block included in the bitstream, and the reference picture Determining a corresponding region of a plurality of reference pictures used for prediction of the current block by tracking a motion vector path of the corresponding region of
Generating a predicted block of the current block by obtaining a weighted sum of corresponding regions of the plurality of reference pictures;
A video decoding method comprising: adding the generated prediction block and a difference value between the current block and the prediction block included in the bitstream and decoding the current block. Determining a corresponding region of the plurality of reference pictures,
Dividing a corresponding region of a first reference picture indicated by a motion vector of the current block along a motion block boundary of the first reference picture;
The method further comprises: determining a corresponding area of a second reference picture indicated by a motion vector of a motion block of the first reference picture having the corresponding area of the divided first reference picture. The video decoding method according to claim 1. Tracking the motion vector path of the corresponding region of the reference picture is
The step of determining the corresponding region of the second reference picture indicated by the motion vector of the current block is performed up to the corresponding region of the nth reference picture indicated by the motion vector of the (n−1) th reference picture, where n The video decoding method according to claim 21, wherein is 3 or more, and the nth reference picture is a reference picture of an (n-1) th reference picture. Generating a prediction block for the current block;
The previously processed pixels of the peripheral block of the current block and the peripheral pixels of the corresponding area of the reference picture corresponding to the pixels of the peripheral block of the current block are used to determine the peripheral pixels of the corresponding area of the reference picture. A value that minimizes the difference between the predicted value of the neighboring pixels of the current block and the pixel value of the neighboring pixels of the current block predicted through the weighted sum is determined as the weight of the corresponding region of the plurality of reference pictures. Stages,
The video decoding method according to claim 21, further comprising: generating a prediction block for the current block by adding the weighted areas to corresponding areas of the plurality of reference pictures. In the video decoding device,
A prediction mode discriminator that interprets the prediction mode information provided in the input bitstream and discriminates the prediction mode of the current block to be decoded;
As a result of the determination, for the current block predicted using the corresponding areas of a plurality of reference pictures, the corresponding area of the reference picture indicated by the motion vector of the current block included in the bitstream, and the reference picture A reference picture determining unit that determines a corresponding area of a plurality of reference pictures used for prediction of the current block by tracking a motion vector path of the corresponding area of
A weighted prediction unit that generates a predicted block of the current block by obtaining a weighted sum of corresponding regions of the plurality of reference pictures;
A video decoding apparatus comprising: the generated prediction block; and a decoding unit that adds a difference value between the current block and the prediction block provided in the bitstream and decodes the current block.

Images