KR20150000073A - Method and apparatus for video encoding and decoding - Google Patents

Abstract

According to an embodiment of the present invention, provided is a device for encoding a video. The device comprises: a motion estimating unit to calculate a first prediction block of a first reference frame which corresponds to a current block being encoded and a second prediction block of at least one second reference frame through motion estimation; a first calculating unit to calculate first similarity between the first prediction block and the second prediction block based on a pixel difference value between the first prediction block and the second prediction block; a determining unit to determine whether to include each of the second reference frames in a reference frame group including the first reference frame based on the first similarity; a second calculating unit to calculate second similarity between a prediction block of a reference frame included in the reference frame group and the current block based on a pixel difference value between the prediction block of the reference frame included in the reference frame group and the current block; a selecting unit to select one of the reference frames included in the reference frame group based on the second similarity; and an encoding unit to encode the pixel difference value between the prediction block of the selected reference frame and the current block.

Description

METHOD AND APPARATUS FOR VIDEO ENCODING AND DECODING [0002]

The present invention relates to an image encoding / decoding method and apparatus.

The development of the video compression codec greatly contributed to the development of the multimedia service by allowing the video data to be compressed remarkably. Video compression codecs have been developed by adopting a technique that continuously improves the compression ratio from the initial compression codec such as H.261 codec and MPEG-1 codec to the latest codec such as H.264. In particular, the H.264 codec has a compression ratio that is superior to that of the conventional compression codec and is widely used in the market.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a video encoding / decoding method and apparatus capable of further improving a compression ratio of a moving picture compression codec.

According to an embodiment of the present invention, an image encoding method is provided. Calculating a first prediction block of a first reference frame corresponding to a current block to be coded and a second prediction block of at least one second reference frame through motion estimation; Including the first reference frame in a reference frame group; Calculating a first similarity degree between the first prediction block and the second prediction block; Determining whether to include each of the second reference frames in the reference frame group based on the first similarity; Calculating a second degree of similarity between a current block and a prediction block of a reference frame included in the reference frame group; Selecting one of the reference frames included in the reference frame group based on the calculated second similarity; And encoding a pixel difference value between a prediction block of the selected reference frame and the current block.

The calculating of the first and second prediction blocks may include calculating a second prediction block of a third reference frame that is the second reference frame through motion estimation; And calculating a second prediction block of the fourth reference frame, which is the second reference frame, through motion estimation.

The first similarity includes a third similarity between the first prediction block and a second prediction block of the third reference frame and a fourth similarity between the first prediction block and a second prediction block of the fourth reference frame . The calculating of the first similarity may include: calculating a first difference value sum which is a sum of pixel difference values between the first prediction block and a second prediction block of the third reference frame; And sets the third degree of similarity to a first level when the first difference value sum is smaller than the first threshold value and sets the third degree of similarity to the first level when the first difference value sum is larger than the first threshold value, Setting a second level lower than the first level; Calculating a second difference sum which is a sum of pixel difference values between the first prediction block and a second prediction block of the fourth reference frame; And sets the fourth similarity degree to the first level when the second difference value sum is smaller than the first threshold value and sets the fourth similarity degree to the first level when the second difference value sum is larger than the first threshold value, To the second level.

The image encoding method may further include setting a reference frame index indicating a reference frame included in the reference frame group before calculating the second similarity.

Comparing the image complexity of the first predictive block with the image complexity of a second predictive block of each of the second reference frames included in the reference frame group before the step of calculating the second similarity, As shown in FIG.

The image encoding method may further include a step of, between the step of comparing the image complexity and the step of calculating the second degree of similarity, calculating a degree of similarity of each of the second reference frames included in the reference frame group May be included in the reference frame group.

According to another embodiment of the present invention, a video decoding method is provided. The video decoding method includes: extracting a motion vector from a first region of a received bitstream; Calculating a first prediction block of a first reference frame and a second prediction block of at least one second reference frame corresponding to a current block to be decoded using the extracted motion vector; Including the first reference frame in a reference frame group; Calculating a first similarity degree between the first prediction block and the second prediction block based on a pixel difference value between the first prediction block and the second prediction block; Determining whether to include each of the second reference frames in the reference frame group based on the first similarity; And extracting first residual data from a second region of the bitstream located after the first region in a first case where only the first reference frame is included in the reference frame group, And restoring the current block by adding the block and the extracted first residual data.

According to another embodiment of the present invention, an image encoding apparatus is provided. A motion estimator for calculating a first prediction block of a first reference frame corresponding to a current block to be coded and a second prediction block of at least one second reference frame through motion estimation; A first calculation unit for calculating a first similarity degree between the first prediction block and the second prediction block based on a pixel difference value between the first prediction block and the second prediction block; A determination unit for determining whether to include each of the second reference frames in a reference frame group including the first reference frame based on the first similarity; A second calculation for calculating a second similarity degree between a current block and a prediction block of a reference frame included in the reference frame group based on a pixel difference value between a current block and a prediction block of a reference frame included in the reference frame group; part; A selection unit for selecting any one of reference frames included in the reference frame group based on the second similarity; And an encoding unit encoding a pixel difference value between a predictive block of the selected reference frame and the current block.

According to an embodiment of the present invention, the reference frame index bits are used only when there is an effect of using multiple reference frames. Accordingly, the reference frame index bit to be encoded at the time of image encoding can be reduced, and the compression rate of the video codec can be further improved.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows a motion estimation / compensation operation using multiple reference frames.
2 is a diagram showing an example of a prediction block of each of a current block and a multiple reference frame to be coded;
3 is a diagram showing another example of a prediction block of each of a current block and a multiple reference frame to be coded;
FIG. 4 illustrates a conditional statement for determining whether to use multiple reference frames according to an embodiment of the present invention; FIG.
5 illustrates a bitstream structure according to an embodiment of the present invention.
6 is a flowchart illustrating an image encoding process of the image encoding apparatus according to the embodiment of the present invention.
FIG. 7 is a flowchart illustrating a video decoding process of a video decoding apparatus according to an embodiment of the present invention; FIG.
8 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

1 is a diagram illustrating a motion estimation / compensation operation using multiple reference frames.

Motion estimation / compensation in a moving image compression codec is a key technique for increasing the compression ratio. Through motion estimation / compensation, temporal redundancy data between consecutive pictures can be removed. In particular, in the H.264 codec, a plurality of reference frames are used as shown in FIG. 1 in order to increase the efficiency of motion estimation / compensation. In FIG. 1, for convenience of explanation, three reference frames F110, F120, and F130 are used in motion estimation / compensation. The current frame F210 is the Nth frame among the N frames. The reference frame F110 is the (N-1) th frame immediately before the current frame F210. The reference frame F120 is the (N-2) th frame, and the reference frame F130 is the (N-3) th frame. 1, each frame F210, F110, F120, and F130 may include objects F212, F112, F122, and F132. Of the three reference frames F110, F120, and F130, the reference frame having the highest degree of similarity to the current frame F210 is selected. Specifically, motion estimation / compensation in a codec such as H.264 is performed on a block-by-block basis. When coding the current block F213 of the current frame F210, a block (prediction block) similar to the current block F213 is searched in the reference frames F110, F120, and F130 to eliminate the temporal redundancy of the video data. This is called a motion estimation process. That is, the motion estimation process is a process of searching from which position of the current frame F213 the reference frames F110, F120, and F130 have moved. Whether the current block F213 has moved from the reference frames F110, F120 or F130 is indicated by a motion vector. For example, it is assumed that the current block F213 has shifted three pixels in the horizontal direction and two pixels in the vertical direction from the same position of the previous frame F110 (i.e., the same position as the current block F213) In this case, the motion vector is (3, 2). In the normal case, in order to speed up the calculation speed rather than searching the entire area of the reference frames F110, F120 and F130 in the motion estimation, the partial areas (the reference frames F110, F120 and F130) F121 and F131 in the motion search areas F111, F121 and F131 and searches for the predicted block in the motion search areas F111, F121 and F131. Also, in order to obtain good performance even in the case where sudden temporal noise or the like occurs, a motion estimation process is performed in multiple reference frames (F110, F120, F130) rather than a single reference frame. On the other hand, when the decoding apparatus decodes the current block F213, the reference frame index and the motion vector information are read from the compressed bitstream to obtain a prediction block of the current block F213, and then motion compensation is performed.

As a result, the probability of finding a reference frame (e.g., F120) that is most similar to the current frame F210 is higher when using multiple reference frames at the time of motion estimation / compensation than when using a single reference frame. Accordingly, the motion estimation / compensation is performed from the reference frame F120 most similar to the current frame F210, whereby the temporal redundant data of the current frame F210 can be removed more efficiently. This improves the compression ratio of the video codec.

2 is a diagram showing an example of a prediction block of each of a current block and a multiple reference frame to be encoded.

The prediction block F113 of the reference frame F110, the prediction block F123 of the reference frame F120 and the prediction block F133 of the reference frame F 130 correspond to the current block F213 of the current frame F210 Blocks.

The motion estimation process using the multiple reference frame is a process of finding an area having the pixel value most similar to the pixel value of the current block F213 of the current frame F210 in the reference frames F110, F120 and F130. FIG. 2 illustrates the prediction blocks F113, F123, and F133 of the reference frames F110, F120, and F130 found by the motion estimation process.

On the other hand, similarity measurement methods mainly used in the motion estimation process include a method using Sum of Absolute Differences (SAD), a method using Sum of Squared Differences (SSE), and a method using Sum of Absolute Transformed Differences . Here, SAD is calculated by the following equation (1).

In Equation (1), y and x are position coordinates in the block of the pixel, and N and M are the vertical and horizontal sizes of the block, respectively. cur (y, x) represents pixel values of y and x coordinates in the current block F213 and ref (y + i, x + j) represents pixel values of the prediction blocks F113, F123 and F133. In the mathematical expression, i, j denotes a motion vector. If the pixel values of the prediction blocks F113, F123 and F133 of the reference frames F110, F120 and F130 and the pixel values of the current block F213 of the current frame F210 are the same as illustrated in FIG. 2, The SAD value of the prediction block F113 is 10 * 16 = 160, the SAD value of the prediction block F123 is 0 * 16 = 0, and the SAD value of the prediction block F133 is 20 * 16 = 320. Since the SAD value between the predicted block F123 of the reference frame F120 and the current block F213 of the current frame F210 is zero and has the minimum value among the three prediction blocks F113, F123, and F133, Motion estimation / compensation is performed. In the example illustrated in FIG. 2, the SAD values of the three prediction blocks F113, F123, and F133 are different from each other, so that the reference frame F120 having the smallest SAD value among the three can be selected. That is, the motion estimation / compensation can be performed from the reference frame F120 having the prediction block F123 most similar to the current block F213, and the compression ratio is improved. As a result, the example illustrated in FIG. 2 is the case where there is an effect of using multiple reference frames. In this case, the reference frame index REF_IDX indicates the reference frame F120. Here, the reference frame index REF_IDX represents a reference frame used in motion estimation / compensation.

3 is a diagram showing another example of a prediction block of each of a current block and a multiple reference frame to be coded.

Each of the prediction blocks F113, F123, and F133 illustrated in FIG. 3 has no pixel value difference from the current block F213. In this case, even if any of the three reference frames F110, F120, and F130 is selected to perform motion estimation / compensation, the compression rate is the same. As a result, the example illustrated in FIG. 3 is the case where there is no effect using multiple reference frames.

In the H.264 codec, a reference frame index (REF_IDX) is encoded to indicate which reference frame should be used for motion estimation / compensation even in the case of FIG. 3 do.

In the present invention, when the pixel values of the prediction blocks F113, F123, and F133 of the multiple reference frames are the same / similar, it is determined that there is no effect of using multiple reference frames. In this case, the multiple reference frames F110, , I.e., the immediately preceding frame (i.e., the (N-1) th frame) of the current frame F210 without using the frame F130. Hereinafter, for convenience of explanation, it is assumed that the frame that has been previously promised is the immediately preceding frame F110 of the current frame F210. This reduces the number of bits necessary for coding the reference frame index (REF_IDX) or the number of reference frame index bits, thereby improving the compression ratio.

As described above, in order to have a motion estimation / compensation effect (hereinafter referred to as a multiple reference frame effect) by using multiple reference frames, prediction of each of the reference frames F113, F123, and F133 There must be a pixel value difference between blocks. The pixel value of a y, x coordinate current block (F213) of the current frame (F210) cur (y, x ), refer to the prediction block of a frame (F110) (F113) in y, the pixel value of the x coordinate ref _{(N 1)} th frame (i.e., the _(N-1) th frame, F110) and the residual signal (residual signal) The residual signal residual _(N-1) of the current frame F210 is defined by the following equation (2).

Similar to the reference frame F110, the residual signal residual _(N-2) of the reference frame F120 and the residual signal The residual _(N-3) of the reference frame F130 is defined by the following equations (3) and (4).

As can be seen from Equations (2) to (4), the residual signals actually encoded in the video compression are ref _(N-1) (y, x), ref _(N-2) , x) and ref _(N-3) (y, x). If a reference frame _{(F110, F120), ref (} N-1) (y, x), ref (N-2) (y, x), ref (N-3) (y, x) of the F130), the same _(N-1) , residual _(N-2) , and residual _(N-3) also have the same value. Incidentally, the reference frame (F110, F120, F130) in _{ref (N-1) (y} , x), ref (N-2) (y, x), ref (N-3) (y, x) is similar to _(N-1) , residual _(N-2) , and residual _(N-3) have similar values. Therefore, by checking the pixel values of the prediction blocks F113, F123, and F133 of the multiple reference frames F110, F120, and F130, it is possible to determine whether there is an effect of motion estimation / compensation by using multiple reference frames. The present invention uses multiple reference frames only if there is a multiple reference frame effect (in this case, a reference frame index (REF_IDX) is used in encoding), otherwise a single promised single frame (e.g., current frame F210 ) (In this case, the reference frame index REF_IDX is not used in coding). This reduces the number of bits required to encode the reference frame index (REF_IDX).

In the present invention, ref _(N-1) (y, x) of a predicted block F113 of an already-promised frame (for example, the N-1th frame F110) (SAD) and difference (SAD) of difference values between ref _(N-2) (y, x) and ref _(N-3) (y, x) of the prediction blocks F123 and 133 of the outer reference frames F120 and F130 The variance is compared with the threshold value TH1 and the threshold value TH2, respectively. That is, the sum of the pixel difference values SAD _(N-2 ) between the prediction block F113 and the prediction block F123 is compared with the threshold value TH1, and the pixel variance value Var (ref _(N-1) (y, x)) and the pixel variance value Var (ref _(N-2) (y, x)) of the prediction block F123 is smaller than the threshold value TH2 do. Then, the sum of the pixel difference values SAD _(N-3 ) between the prediction block F113 and the prediction block F133 is compared with the threshold value TH1, and the pixel variance value Var (ref _(N-1) (y, x)) and the pixel variance value Var (ref _(N-3) (y, x)) of the prediction block F133 is smaller than the threshold value TH2 do. Here, the threshold value TH1 may be determined according to the quantization parameter value of the H.264 codec in order to adaptively determine the compression rate and the picture quality of the moving picture. The H.264 codec supports 52 levels of quantization coefficients ranging from 0 to 51, and defines a quantization step size (Q _step ) according to each quantization coefficient. Specifically, the threshold value TH1 can be determined by the following equation (5).

In Equation (5),? Is an adjustment value, which can be adaptively specified at the time of coding considering the complexity of the image and the image quality. For example, alpha may be 2.

The threshold value TH2 is determined by the pixel variance value Var (ref _(N-1) (y, x)) of the predicted block F113 of the previously promised frame (for example, do. Specifically, the threshold value TH2 can be determined by the following equation (6).

In Equation (6),? Is an adjustment value and can be adaptively determined in consideration of image complexity and image quality. For example, beta may be 0.1.

On the other hand, in the image encoding, the pixel difference value and the variance value between the respective screens may vary greatly according to the screen change. In order to adaptively calculate the TH1 and TH2 values according to the change between the screens in the image, the present invention includes the adjustment values α and β in the encoding, and includes the slice header information in the decoding, Calculate TH1 and TH2 values.

The present invention relates to a method for producing SAD _(N-2) , SAD SAD _(N-3) , _{Var (ref (N-1)} (y, x)), Var (ref (N-2) (y, x)), Var (ref (N-3) (y, x)), and the threshold value (TH1 , TH2) are calculated. For example, if the SAD _(N-2) in the example of Figure 2 is _{10 * 16 = 160, SAD (} N-3) is 10 *, and _{16 = 160, Var (ref (} N-1) (y, x) ), Var (ref _(N-2) (y, x)) and Var (ref _(N-3) Thereafter, it is determined whether to use multiple reference frames according to the conditional statements shown in Fig.

4 is a diagram illustrating conditional statements for determining whether to use multiple reference frames according to an embodiment of the present invention.

The reference frame F110 is included in the reference frame group. It is determined whether each of the reference frames F120 and F130 is included in the reference frame group through the conditional statement shown in FIG.

First, the reference frame F120 is compared with the reference frame F110. Specifically, it is determined whether SAD _(N-2) is smaller than the threshold value TH1. If SAD _(N-2) is larger than the threshold value TH1, the reference frame F120 is included in the reference frame group. If the SAD _(N-2) is less than the threshold value (TH1), the difference between the _{Var (ref (N-2)} (y, x)) and _{Var (ref (N-1)} (y, x)) threshold Is smaller than the value TH2. If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is greater than the threshold value TH2, Group. If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is less than the threshold value TH2, Do not include in the group.

Next, the reference frame F130 is compared with the reference frame F110. Specifically, it is determined whether SAD _(N-3) is smaller than the threshold value TH1. If SAD _(N-3) is larger than the threshold value TH1, the reference frame F130 is included in the reference frame group. If the SAD _(N-3) is less than the threshold value (TH1), the difference between the _{Var (ref (N-3)} (y, x)) and _{Var (ref (N-1)} (y, x)) threshold Is smaller than the value TH2. If the difference between Var (ref _(N-3) (y, x)) and Var (ref _(N-1) (y, x)) is greater than the threshold value TH2, Group. If the difference between Var (ref _(N-3) (y, x)) and Var (ref _(N-1) (y, x)) is less than the threshold value TH2, Do not include in the group.

A single reference frame is used without using multiple reference frames when the number of reference frames included in the reference frame group is one (i.e., only the reference frame F110 is included in the reference frame group). A multi-reference frame is used when the number of reference frames included in the reference frame group is two or more. Specifically, when the number of reference frames included in the reference frame group is two (e.g., when the reference frame F110 and the reference frame F130 are included in the reference frame group), the reference frame indexes are reordered . This reduces the number of reference frame index bits. In this case, the rearranged reference frame indices are shown in Table 1 below.

A table showing the reordered reference frame indexes Existing index Whether it belongs to the reference frame group Reordered index In the reference frame F110, 0 Belong 0 In the reference frame F120, One Does not belong - In the reference frame F130, 2 Belong One

According to Table 1, not only in the case where the number of reference frames included in the reference frame group is 1 but also in the case where the number of reference frames included in the reference frame group is 2, the number of reference frame index bits can be reduced. For example, if a reference frame F130 of a plurality of reference frames F110, F120, and F130 is selected and used in motion estimation / compensation, two of the three values (0, 1, 2) 2 bits are required because it must be indicated in the reference frame index. However, according to the present invention, 1 bit is required since 1 of the two values (0, 1) can be indicated in the reference frame index.

In the present invention, the reference frame index bitstream is located after the motion vector bitstream as shown in FIG. 5 in order to omit the reference frame index bit according to whether the multiple reference frame is used or not.

5 is a diagram illustrating a bitstream structure according to an embodiment of the present invention.

In the existing bitstream (BS1) structure, a motion vector bitstream is located after the reference frame index bitstream. In the bit stream (BS2) structure according to the present invention, the reference frame index bit stream is located after the motion vector bit stream. When decoding a compressed moving picture, it is necessary to know a motion vector and a reference frame index in order to obtain a prediction block for motion compensation. In the existing H.264 codec structure, since the reference frame index is decoded before the motion vector is known, a reference frame index is necessarily required to obtain a prediction block for motion compensation. Therefore, it is impossible to omit the reference frame index. However, since the motion vector is decoded first in the bit stream (BS2) structure according to the present invention, the prediction blocks F113, F123 and F133 of the multiple reference frames can be obtained before decoding the reference frame indexes. As described above with reference to FIG. 4, the reference frame index is decoded only when multiple reference frames are used by determining whether to use multiple reference frames. That is, if a multiple reference frame is not used (when a single reference frame is used), a reference frame index is not needed because it uses a predetermined frame (for example, F110 being the (N-1) th frame). Therefore, when multiple reference frames are not used, the reference frame index can be omitted. That is, when using multiple reference frames, the reference frame index bitstream is located after the motion vector bitstream, and the residual bitstream is located after the reference frame index bitstream. On the other hand, when the multiple reference frame is not used, the reference frame index is omitted and the residual bitstream is located after the motion vector bitstream.

6 is a flowchart illustrating an image encoding process of the image encoding apparatus according to the embodiment of the present invention.

First, a motion vector is encoded (S110).

The prediction blocks F113, F123, and F133 of the reference frames F110, F120, and F130 are calculated (S120).

Threshold values TH1 and TH2 are calculated (S130).

The reference frame F110 is included in the reference frame group. The number (USED_REFERENCE_NUMBER) of reference frames included in the current reference frame group is 1 (S140).

The reference frame F120 is compared with the reference frame F110. Specifically, SAD _(N-2) , Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) are calculated (S150). It is determined whether SAD _(N-2) is smaller than the threshold value TH1 (S160). If the SAD _(N-2) is larger than the threshold TH1, the reference frame F120 is included in the reference frame group, and the number of reference frames (USED_REFERENCE_NUMBER) included in the reference frame group is 2 (S190) . If the SAD _(N-2) is less than the threshold value (TH1), the difference between the _{Var (ref (N-2)} (y, x)) and _{Var (ref (N-1)} (y, x)) threshold TH2 (S170). If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is greater than the threshold value TH2, Group, and the number of reference frames (USED_REFERENCE_NUMBER) included in the reference frame group is 2 (S190). If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is less than the threshold value TH2, Group (S180).

The reference frame F130 is compared with the reference frame F110. That is, the above-described steps S150 to S190 are performed on the reference frame F130.

The above-described comparison process is performed on all of the reference frames F120 and F130. That is, the above-described comparison process is performed twice (number of reference frames (NUMBER_OF_REFERENCE_FRAMES) - 1) (S200).

When the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is 1, the next syntax (synax) is encoded (S210, S240). Since the reference frame F110 is previously predicted to be used for motion estimation / compensation when the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is one (i.e., when multiple reference frames are not used) The reference frame index is omitted.

If the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is not one (i.e., when multiple reference frames are used), the reference frame indexes for the reference frames included in the reference frame group are rearranged (S210, S220). The SAD value between the prediction block of each of the reference frames included in the reference frame group and the current block F213 of the current frame F210 is obtained. Then, the reference frame having the smallest SAD value is selected from the reference frames included in the reference frame group. The reference frame index indicating the selected reference frame is encoded (S230). Then, the following syntax (synax) is encoded (S240).

FIG. 7 is a flowchart illustrating an image decoding process of an image decoding apparatus according to an embodiment of the present invention. The image decoding process is similar to the image encoding process described above with reference to FIG.

First, the motion vector is decoded (S310).

The prediction blocks F113, F123, and F133 of the reference frames F110, F120, and F130 are calculated (S320).

Threshold values TH1 and TH2 are calculated (S330).

The reference frame F110 is included in the reference frame group. The number (USED_REFERENCE_NUMBER) of reference frames included in the current reference frame group is 1 (S340).

The reference frame F120 is compared with the reference frame F110. Specifically, SAD _(N-2) , Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) are calculated (S350). It is determined whether SAD _(N-2) is smaller than the threshold value TH1 (S360). If the SAD _(N-2) is larger than the threshold value TH1, the reference frame F120 is included in the reference frame group, and the number of reference frames (USED_REFERENCE_NUMBER) included in the reference frame group is 2 (S390) . If the SAD _(N-2) is less than the threshold value (TH1), the difference between the _{Var (ref (N-2)} (y, x)) and _{Var (ref (N-1)} (y, x)) threshold (TH2) (S370). If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is greater than the threshold value TH2, Group, and the number of reference frames (USED_REFERENCE_NUMBER) included in the reference frame group is 2 (S390). If the difference between Var (ref _(N-2) (y, x)) and Var (ref _(N-1) (y, x)) is less than the threshold value TH2, Group (S380).

The reference frame F130 is compared with the reference frame F110. That is, the above-described steps S350 to S390 are performed on the reference frame F130.

The above-described comparison process is performed on all of the reference frames F120 and F130. That is, the above-described comparison process is performed twice (number of reference frames (NUMBER_OF_REFERENCE_FRAMES) - 1) (S400).

When the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is 1, the next syntax (synax) is encoded (S410, S440). Since the reference frame F110 is previously predicted to be used for motion estimation / compensation when the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is one (i.e., when multiple reference frames are not used) And performs motion estimation / compensation using the reference frame F110. That is, the prediction block F113 of the reference frame F110 and the residual signal extracted from the received residual bitstream are added to reconstruct the current block F213 of the current frame F210.

If the number of reference frames included in the reference frame group (USED_REFERENCE_NUMBER) is not one (i.e., when multiple reference frames are used), the reference frame indexes for the reference frames included in the reference frame group are rearranged (S410, S420). The reference frame index is extracted from the received reference frame index bit stream, and the extracted reference frame index is decoded (S430). The prediction block of the reference frame corresponding to the decoded reference frame index and the residual signal extracted from the received residual bitstream are added to reconstruct the current block F213 of the current frame F210. Then, the next syntax (synax) is decoded (S440).

8 is a diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus shown in FIG. 8 performs the image encoding operation described above. More specifically, the image encoding apparatus includes a motion estimation unit 100, a first calculation unit 200, a determination unit 300, a second calculation unit 400, a selection unit 500, and a coding unit 600 do.

The motion estimation unit 100 calculates the motion vectors of the reference frames F113, F123 and F133 of the reference frames F110, F120 and F130 corresponding to the current block F213 of the current frame F210 to be encoded do.

The first calculation unit 200 calculates the degree of similarity between the prediction block F113 and the prediction blocks F123 and F133 based on the pixel difference value between the prediction block F113 and the prediction blocks F123 and F133. Specifically, the first calculation unit 200 calculates SAD _(N-2) and SAD _(N-3) .

The determination unit 300 determines whether or not to include the reference frames F120 and F130 in the reference frame group including the reference frame F110 based on the similarity calculated by the first calculation unit 200 do.

The second calculation unit 400 calculates the difference between the prediction block of the reference frame included in the reference frame group and the current block F213 based on the pixel difference value between the prediction block of the reference frame included in the reference frame group and the current block F213, F213). Specifically, the second calculation unit 400 obtains the SAD value between the prediction block of each reference frame included in the reference frame group and the current block F213 of the current frame F210.

The selection unit 500 selects any one of the reference frames included in the reference frame group based on the similarity calculated by the second calculation unit 400. [ Specifically, the selector 500 selects a reference frame having the smallest SAD value among the SAD values calculated by the second calculator 400 among the reference frames included in the reference frame group.

The encoding unit 600 encodes the pixel difference value between the prediction block of the reference frame selected by the selection unit 500 and the current block F213.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

100: motion estimation unit 200: first calculation unit
300: determination unit 400: second calculation unit
500: selection unit 600: encoding unit

Claims (25)

Calculating a first prediction block of a first reference frame corresponding to a current block to be coded and a second prediction block of at least one second reference frame through motion estimation;
Including the first reference frame in a reference frame group;
Calculating a first similarity degree between the first prediction block and the second prediction block;
Determining whether to include each of the second reference frames in the reference frame group based on the first similarity;
Calculating a second degree of similarity between a current block and a prediction block of a reference frame included in the reference frame group;
Selecting one of the reference frames included in the reference frame group based on the calculated second similarity; And
Encoding a pixel difference value between a prediction block of the selected reference frame and the current block
/ RTI > The method according to claim 1,
Wherein the step of calculating the first prediction block and the second prediction block comprises:
Calculating a second prediction block of a third reference frame that is the second reference frame through motion estimation; And
And calculating a second prediction block of a fourth reference frame that is the second reference frame through motion estimation
Image encoding method. 3. The method of claim 2,
Wherein the first similarity comprises a third similarity between the first prediction block and a second prediction block of the third reference frame and a fourth similarity between the first prediction block and a second prediction block of the fourth reference frame ,
Wherein the step of calculating the first degree of similarity comprises:
Calculating a first difference value sum which is a sum of pixel difference values between the first prediction block and a second prediction block of the third reference frame;
And sets the third degree of similarity to a first level when the first difference value sum is smaller than the first threshold value and sets the third degree of similarity to the first level when the first difference value sum is larger than the first threshold value, Setting a second level lower than the first level;
Calculating a second difference sum which is a sum of pixel difference values between the first prediction block and a second prediction block of the fourth reference frame; And
And sets the fourth degree of similarity to the first level when the second difference value sum is smaller than the first threshold value and sets the fourth degree of similarity to the first level when the second difference value sum is larger than the first threshold value And setting the second level to the second level
Image encoding method. The method of claim 3,
Wherein the first threshold value is a first threshold value,
Is determined according to the quantization step size corresponding to the quantization parameter
Image encoding method. 5. The method of claim 4,
The first threshold value is determined according to the following equation (1)
Image encoding method.
[Equation 1]
Th1 = Q _step * N * M *?
(Th1: the first threshold value, Q _step : the quantization step size, N: the length of the first prediction block, M: the width of the first prediction block, Variable) 6. The method of claim 5,
Further comprising the step of including alpha in the slice header information in Equation (1)
Image encoding method. The method of claim 3,
Wherein the step of determining whether to include in the reference frame group comprises:
Including the third reference frame in the reference frame group when the third degree of similarity is the first level; And
And including the fourth reference frame in the reference frame group when the fourth degree of similarity is the first level
Image encoding method. The method according to claim 1,
Before the step of calculating the second degree of similarity,
And setting a reference frame index indicating a reference frame included in the reference frame group
Image encoding method. 9. The method of claim 8,
Wherein the step of setting the reference frame index comprises:
And adjusting the number of bits for the reference frame index corresponding to the number of reference frames included in the reference frame group
Image encoding method. The method according to claim 1,
Before the step of calculating the second degree of similarity,
And comparing the image complexity of the first predictive block with the image complexity of a second predictive block of each second reference frame included in the reference frame group
Image encoding method. 11. The method of claim 10,
Between the step of comparing the image complexity and the step of calculating the second degree of similarity,
Determining whether to exclude each of the second reference frames included in the reference frame group from the reference frame group according to the comparison result in the step of comparing the image complexity
Image encoding method. 12. The method of claim 11,
Wherein the step of comparing the image complexity comprises:
Calculating a first pixel variance value representing an image complexity of the first prediction block;
Calculating a second pixel variance value representing an image complexity of a second prediction block of each second reference frame included in the reference frame group; And
And determining whether a difference between the first pixel variance value and the second pixel variance value is less than a second threshold value
Image encoding method. 13. The method of claim 12,
Wherein the second threshold value is determined according to the first pixel variance value
Image encoding method. 14. The method of claim 13,
The second threshold value is determined according to the following equation (2)
Image encoding method.
&Quot; (2) "
Th2 = Var1 *?
(Th2: the second threshold value, Var1: the first pixel variance value, and?: A variable whose value varies depending on the image complexity and image quality) 15. The method of claim 14,
Further comprising the step of including beta in the slice header information in Equation (2)
Image encoding method. 14. The method of claim 13,
Wherein determining whether to exclude from the reference frame group comprises:
The step of excluding a second reference frame having the second pixel variance value from the reference frame group when it is determined that the difference between the first pixel variance value and the second pixel variance value is smaller than the second threshold value, Included
Image encoding method. 10. The method of claim 9,
Wherein adjusting the number of bits for the reference frame index comprises:
And adjusting the number of bits for the reference frame index to 0 when the number of reference frames included in the reference frame group is 1
Image encoding method. 18. The method of claim 17,
Encoding a motion vector of the current block indicating a prediction block of the selected reference frame;
Encoding a reference frame index indicating the selected reference frame;
Inserting the encoded motion vector into a first region of a bitstream; And
Further comprising inserting the encoded reference frame index into a second region of the bitstream located after a first region of the bitstream,
Wherein the second region of the bitstream has a size corresponding to the number of bits for the reference frame index
Image encoding method. Extracting a motion vector from a first region of the received bitstream;
Calculating a first prediction block of a first reference frame and a second prediction block of at least one second reference frame corresponding to a current block to be decoded using the extracted motion vector;
Including the first reference frame in a reference frame group;
Calculating a first similarity degree between the first prediction block and the second prediction block based on a pixel difference value between the first prediction block and the second prediction block;
Determining whether to include each of the second reference frames in the reference frame group based on the first similarity; And
Extracting first residual data from a second region of the bitstream located after the first region in a first case where only the first reference frame is included in the reference frame group, And restoring the current block by adding the extracted first residual data
And decodes the decoded image. 20. The method of claim 19,
Wherein the step of calculating the first degree of similarity comprises:
Calculating a first threshold value using a first adjustment value included in the received slice header information; And
And calculating the first similarity by comparing the sum of the first threshold value and the pixel difference value
Video decoding method. 20. The method of claim 19,
Performing index setting on a reference frame included in the reference frame group in a second case where the number of reference frames included in the reference frame group is two or more;
Further comprising: 22. The method of claim 21,
Extracting a reference frame index from the second region in the second case and extracting second residual data from a third region of the bitstream located after the second region;
Selecting a reference frame corresponding to the extracted reference frame index from the reference frames included in the reference frame group in the second case; And
And restoring the current block by adding the predicted block of the selected reference frame and the extracted second residual data in the second case
Further comprising: 20. The method of claim 19,
Before the step of restoring the current block,
Calculating a first pixel variance value of the first prediction block and a second pixel variance value of a second prediction block of each second reference frame included in the reference frame group; And
If the difference between the first pixel variance value and the second pixel variance value is smaller than a first threshold value, excluding a second reference frame having the second pixel variance value from the reference frame group
Further comprising: 24. The method of claim 23,
Wherein the step of excluding from the reference frame group comprises:
And calculating the first threshold value using a first adjustment value included in the received slice header information
Video decoding method. A motion estimator for calculating a first prediction block of a first reference frame corresponding to a current block to be coded and a second prediction block of at least one second reference frame through motion estimation;
A first calculation unit for calculating a first similarity degree between the first prediction block and the second prediction block based on a pixel difference value between the first prediction block and the second prediction block;
A determination unit for determining whether to include each of the second reference frames in a reference frame group including the first reference frame based on the first similarity;
A second calculation for calculating a second similarity degree between a current block and a prediction block of a reference frame included in the reference frame group based on a pixel difference value between a current block and a prediction block of a reference frame included in the reference frame group; part;
A selection unit for selecting any one of reference frames included in the reference frame group based on the second similarity; And
An encoding unit for encoding a pixel difference value between a prediction block of the selected reference frame and the current block;
And an image encoding device.

Images