JP4169767B2 - Encoding method - Google Patents

Description

  The present invention relates to an encoding method for encoding a moving image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. In addition, there is a next-generation image compression technique such as SVC (Scalable Video Coding) in which one stream includes a high-quality stream and a low-quality stream.

  When streaming a high-resolution moving image or storing it in a recording medium, it is necessary to increase the compression rate of the moving image stream so as not to compress the communication band or increase the storage capacity. In order to enhance the compression effect of moving images, motion compensation interframe predictive coding is performed. In motion-compensated interframe predictive coding, the encoding target frame is divided into blocks, the motion from a reference frame that has already been encoded is predicted for each block, a motion vector is detected, and motion vector information is encoded along with the difference image. Turn into.

In Patent Document 1, when a predicted motion vector predicted from a residual motion vector and the number of remaining frames exists in the vicinity of a motion vector between frames, a predicted motion vector in the vicinity of a motion vector between frames is used as a motion vector. A motion-compensated predictive coding method is described in which when a motion vector predictor does not exist in the vicinity of a motion vector between frames, the motion vector between frames is used as a motion vector.
JP-A-2-219391

  H. In the H.264 / AVC standard, in order to perform more detailed prediction in motion compensation, the block size of motion compensation can be made variable, and the pixel accuracy of motion compensation can be reduced to ¼ pixel accuracy. Therefore, the amount of code related to the motion vector increases. In SVC, which is a next-generation image compression technology, MCTF (Motion Compensated Temporal Filtering) technology is being studied in order to improve temporal scalability. This is a combination of subband division in the time axis direction and motion compensation. Since hierarchical motion compensation is performed, information on motion vectors becomes very large. As described above, the recent video compression coding technology tends to increase the data amount of the entire video stream due to an increase in the amount of information related to motion vectors, and there is a further demand for a technology for reducing the amount of codes resulting from motion vector information. ing.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a moving picture coding technique with high coding efficiency.

  One embodiment of the present invention is a method for encoding a moving picture. This method is characterized in that the motion state of each block between a plurality of encoding target pictures is estimated, and motion mode information representing the estimated motion state is included in the encoded data of the moving image.

  “Picture” is a unit of encoding, and its concept includes a frame, a field, a VOP (Video Object Plane), and the like. In addition, a “block” in a picture to be encoded is a block that includes a plurality of pixels included in a certain range, such as a macroblock and an object, and can be a matching target in motion compensation prediction.

  According to this aspect, since the motion mode in which the motion state of the block in the encoding target picture is estimated is included in the encoded data, encoding or decoding using the motion mode can be realized.

  When a reference motion vector, which is a motion vector of a block in the second reference picture based on the first reference picture, is obtained, the first reference picture is assigned to the block in the current picture. Determine the encoding target motion vector that is the reference motion vector, determine the ratio of the encoding target motion vector component and the reference motion vector component for multiple encoding target pictures, and refer to this ratio for each block. May be estimated. According to this, using the motion vector of the second reference picture, the motion vector of the picture to be encoded is expressed and the motion state of the block is estimated. Therefore, since it is not necessary to encode the motion vector itself to be encoded, the code amount of the entire motion vector data can be reduced, and the encoding efficiency of moving images can be improved.

  The “first reference picture” is, for example, the “forward reference frame” in the embodiment, and the “second reference picture” is the “back reference frame” in the embodiment. One reference picture may be a backward reference frame and the second reference picture may be a forward reference frame. In addition, the “encoding target picture” is, for example, a B frame in the embodiment.

  A block that is a target of motion compensation prediction in each of a plurality of encoding target pictures is identified by matching with a block in the second reference picture, and an encoding target motion vector is obtained for the identified block. Good.

  The motion state of a block may be estimated by taking a difference in ratios obtained between adjacent encoding target pictures. Thereby, the motion state of the block can be estimated by a simple method.

  A coefficient for obtaining an encoding target motion vector from a reference motion vector according to the motion mode may be obtained, and information on the coefficient may be included in the encoded data of the moving image.

  The motion mode includes a constant velocity motion mode in which the block is estimated to move at a constant velocity between the encoding target pictures, and the coefficient is determined based on a time interval between the first reference picture and the encoding target picture. May be. This eliminates the need to encode the coefficients, thereby reducing the code amount of the entire motion vector data and improving the encoding efficiency of moving images.

  The motion mode includes a uniform acceleration motion mode in which the block is estimated to perform a uniform acceleration motion between the encoding target pictures, and the coefficient is determined based on a time interval between the first reference picture and the encoding target picture. May be. Also in this case, it is not necessary to encode the coefficients, the code amount of the entire motion vector data can be reduced, and the encoding efficiency of the moving image can be improved.

  When coding a coefficient, a constant closest to the coefficient is selected from a plurality of constants to which a variable length code is assigned in advance, and the code assigned to the selected constant is included in the encoded data of the moving image Also good. Accordingly, it is not necessary to encode the coefficient itself, and only a pre-assigned code needs to be included in the encoded data of the moving image, so that the code amount of the encoded data can be suppressed.

  An adjustment vector representing an error between the encoding target motion vector and a vector obtained by multiplying the reference motion vector by a coefficient may be obtained, and information on the adjustment vector may be included in the encoded data of the moving image. As a result, even when the coefficients are approximated with the above-described constants, if the adjustment vector is calculated for each macroblock, the error is absorbed by the adjustment vector, so that the accuracy of motion compensation prediction does not decrease. When encoding the adjustment vector, a variable length code corresponding to the appearance frequency may be given.

  Also, motion mode information may be encoded for a plurality of pictures, and coefficient and adjustment vector information may be encoded for each motion vector to be encoded.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, the amount of code of motion vector data can be reduced.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in terms of software, it is realized by a program having an image encoding function loaded in the memory. So, functional blocks that are realized by their cooperation are drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The coding apparatus 100 according to the present embodiment is a moving picture experts group (MPEG-1) series standard (MPEG-1, MPEG) standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission). -2 and MPEG-4), standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector), which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264 respectively).

  In the MPEG series standard, an image frame for intra-frame encoding is an I (Intra) frame, an image frame for forward inter-frame predictive encoding with a past frame as a reference image, a P (Predictive) frame, and past and future An image frame that performs bidirectional inter-frame predictive coding using this frame as a reference image is referred to as a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference image may be a past two frames as a reference image or a future two frames as a reference image regardless of the time. Further, three or more frames can be used as the reference image regardless of the number of frames that can be used as the reference image. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame because the time of the reference image does not matter before and after.

In the specification of the present application, a frame and a picture are used in the same meaning, and an I frame, a P frame, and a B frame are also called an I picture, a P picture, and a B picture, respectively.
In this specification, a frame is used as an example of the encoding unit. However, the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, encodes the moving image, and outputs an encoded stream.

  The block generation unit 10 divides the input image frame into macro blocks. Macroblocks are formed in order from the upper left to the lower right of the image frame. The block generation unit 10 supplies the generated macroblock to the differentiator 12 and the motion compensation unit 60.

  If the image frame supplied from the block generation unit 10 is an I frame, the differentiator 12 outputs it to the DCT unit 20 as it is, but if it is a P frame or a B frame, the difference image 12 is supplied from the motion compensation unit 60. Is calculated and supplied to the DCT unit 20.

  The motion compensation unit 60 uses a past or future image frame stored in the frame memory 80 as a reference image, and has the smallest error for each macroblock of the P frame or the B frame input from the block generation unit 10. A prediction area is searched from the reference image, and a motion vector indicating a deviation from the macroblock to the prediction area is obtained. The motion compensation unit 60 performs motion compensation for each macroblock using the motion vector, and generates a predicted image. The motion compensation unit 60 supplies the generated motion vector to the variable length encoding unit 90 and supplies the predicted image to the differentiator 12 and the adder 14.

  The motion compensation unit 60 can apply both bidirectional prediction and unidirectional prediction. In the unidirectional prediction, the motion compensation unit 60 generates a forward motion vector indicating the motion with respect to the forward reference frame. In the bi-directional prediction, in addition to the forward motion vector, two motion vectors of a backward motion vector indicating motion with respect to the backward reference frame are generated.

  The differentiator 12 obtains a difference between the current image output from the block generation unit 10 (that is, the image to be encoded) and the predicted image output from the motion compensation unit 60 and outputs the difference to the DCT unit 20. The DCT unit 20 performs a discrete cosine transform (DCT) on the difference image given from the differentiator 12 and gives a DCT coefficient to the quantization unit 30.

  The quantization unit 30 quantizes the DCT coefficient and provides it to the variable length coding unit 90. The variable length coding unit 90 performs variable length coding on the quantized DCT coefficient of the difference image together with the motion vector supplied from the motion compensation unit 60, and generates an encoded stream. The variable length encoding unit 90 performs processing of rearranging the encoded frames in time order when generating the encoded stream.

  The quantization unit 30 supplies the quantized DCT coefficient of the image frame to the inverse quantization unit 40. The inverse quantization unit 40 inversely quantizes the supplied quantized data and supplies the quantized data to the inverse DCT unit 50. The inverse DCT unit 50 performs inverse discrete cosine transform on the supplied inverse quantized data. Thereby, the encoded image frame is restored. The restored image frame is input to the adder 14.

  If the image frame supplied from the inverse DCT unit 50 is an I frame, the adder 14 stores it in the frame memory 80 as it is. If the image frame supplied from the inverse DCT unit 50 is a P frame or a B frame, the adder 14 is a difference image, and thus is supplied from the difference image supplied from the inverse DCT unit 50 and the motion compensation unit 60. By adding the predicted image, the original image frame is reconstructed and stored in the frame memory 80.

  In the case of P frame or B frame encoding processing, the motion compensation unit 60 operates as described above. However, in the case of I frame encoding processing, the motion compensation unit 60 does not operate and is not shown here. The I frame is supplied to the DCT unit 20 after intra prediction.

  Next, a conventional motion vector calculation method will be described, and then a motion vector calculation method according to an embodiment of the present invention will be described.

FIG. 2 is a diagram for explaining calculation of a conventional motion vector. The figure shows five frames in order of display time, with the flow from left to right as the time flow. The display is in the order of I frame 201, B ₁ frame 202, B ₂ frame 203, B ₃ frame 204, and P frame 205. Made. The encoding order is different from the display order. First, the I frame 201 in the figure is encoded, and then the fifth P frame 205 is encoded with motion compensation using the I frame 201 as a reference image. . Thereafter, the B ₂ frame 203 is encoded, and the motion compensation is performed in the order of the B ₁ frame 202 and the B ₃ frame 204 and encoded.

  As a reference frame for encoding a P frame, a temporally previous I frame or P frame is used. In addition, as the reference frame for encoding the B frame, the previous I frame or the temporally preceding and following P frames are used. In the case of P frame motion compensation prediction, the prediction unit is, for example, a 16 × 16 macroblock, and one motion vector is used. For the B frame, motion compensation is performed by selecting an optimal prediction from forward, backward and bidirectional predictions. The I frame 201 may be a P frame. The P frame 205 may be an I frame.

At this time, it is assumed that the encoding of the I frame 201 and the P frame 205 is completed and the B _{1 to} B ₃ frames 202 to 204 are encoded. The B _{1 to} B ₃ frames 202 to 204 are called “encoding target frames”, and the I frame 201 displayed before the encoding target frame is displayed as “forward reference frame” and after the encoding target frame. The P frame 205 is referred to as a “backward reference frame”. Also, the motion vector of the P frame 205 is expressed as “MV _P ”, and the motion vector of the B _{1 to} B ₃ frames is expressed as “MV _B1 to MV _B3 ”.
In FIG. 2, the two-dimensional image is shown one-dimensionally, but it should be noted that the actual motion vector has two-dimensional components in the horizontal and vertical directions of the image.

As shown in FIG. 2, a motion vector MV _P 225 indicating the macro block 211 of the forward reference frame 201 is obtained for the macro block 215 in the P frame 205. Next, for the macroblock 213 in the encoding target frame 203, a motion vector MV _B2 222 indicating either the forward reference frame 201 or the backward reference frame 205 is obtained. FIG. 2 shows a case where the forward reference frame 201 is pointed. Subsequently, a motion vector MV _B1 221 indicating the macro block of either the forward reference frame 201 or the backward reference frame 205 is obtained for the macro block 212 in the encoding target frame 202.

  In the present embodiment, instead of encoding the motion vector of each macroblock in the encoding target frame obtained as described above (hereinafter referred to as “encoding target motion vector”) as it is, a backward reference frame or The motion vector of the forward reference frame (hereinafter referred to as “reference motion vector”) is multiplied by a coefficient to represent the encoding target motion vector, and the reference motion vector and the coefficient are encoded. Thereby, the code amount of motion vector data can be reduced.

In the following, already a case is explained using the motion vector MV _P of the backward reference frame 205 the motion vector is detected, may be used motion vectors and other motion vectors of the forward reference frame 201.

FIG. 3 is a diagram illustrating the configuration of the motion compensation unit 60 according to the present embodiment.
Motion compensation unit 60, when performing motion compensation of the backward reference frame 205, and detects a motion vector MV _P of each macro block of the backward reference frame 205, already detected motion vector information of the backward reference frame 205 It is held in the motion vector holding unit 62.

  The block matching unit 61 identifies a macroblock that is a target of motion compensation prediction in the encoding target frames 202 to 204 by block matching with the macroblock in the backward reference frame 205.

The following procedure is used to identify a macroblock that is subject to motion compensation prediction within the encoding target frame. First, a motion vector of a macroblock of an encoding target frame (hereinafter referred to as “normal motion vector” in this section) is obtained by a method such as normal block matching. Subsequently, a certain range including the position pointed to by the normal motion vector on the encoding target frame is determined. Then, the motion vector of the backward reference frame that passes through the determined range is extracted. When a plurality of motion vectors are extracted, a motion vector that passes through a position closest to the position pointed to by the previously obtained normal motion vector is selected. Thus the motion vector extracted or selected and can be considered as reference motion vector MV _P macroblock coding target frame is to be referred.

The motion vector calculation unit 63 obtains motion vectors MV _{B1 to} MV _B3 indicating the macro blocks of the forward reference frame 201 for each macro block in the encoding target frames 202 to 204.

Ratio calculation unit 64 refers to the information of the reference motion vector from the motion vector holding unit 62, the coding target frame 202 to 204, the reference motion vector MV components _P and coded motion vector _MV B1 _{to MV B3} The ratio with each component of is obtained. When the ratios for the B ₁ frame 202, the B ₂ frame 203, and the B ₃ frame 204 are expressed as c ₁ , c ₂ , and c ₃ , the following equation is obtained.
c ₁ = [MV _B1 ] / [MV _P ]
c ₂ = [MV _B2 ] / [MV _P ]
c ₃ = [MV _B3 ] / [MV _P ]
Here, [MV _B1 ] to [MV _B3 ], [MV _P ] represent horizontal components or vertical components of the respective motion vectors. Here, for the sake of simplicity, a component in one direction is described, but in practice, all components are calculated.

The motion analysis unit 65 refers to the ratios c _{1 to} c ₃ calculated by the ratio calculation unit 64 for each macro block, and estimates the motion state of the macro block. Specifically, first, a difference in ratio is taken for each macroblock between adjacent encoding target frames. If the encoding target frame is 3 frames, (c ₂ -c ₁ ) and (c ₃ -c ₂ ) are calculated. Then, the relationship between these difference values is examined.

  When the difference value is 0, it indicates that the motion vector has not changed in the encoding target frame, and therefore this macroblock is estimated to be stopped. If the difference value is not 0 but is approximately equal, it is estimated that the macroblock is moving at a constant velocity. If the difference values are uniformly increased or decreased, it is determined that the macroblock is moving at a constant acceleration. If the difference value is not in any of the above relations, it is estimated that the macroblock is moving irregularly.

The exercise mode selection unit 66 selects an exercise mode according to the estimation. Preferably, the motion modes include a constant velocity motion mode in which it is estimated that the macroblock moves at a constant velocity between the encoding target frames, and a constant acceleration motion mode in which it is estimated that the macroblock moves at a constant acceleration between the encoding target frames. . Furthermore, motion mode selection unit 66, according to the selected exercise mode, to calculate the coefficients alpha ₁ to? ₃ for obtaining the coding target motion vector _MV B1 _{to MV B3} from the reference motion vector MV _P. The coefficients α _{1 to} α ₃ are determined based on the time interval between the forward reference frame and the encoding target frame. This will be described later with reference to FIGS.

Error calculating unit 67, a reference motion vector MV motion vector obtained by multiplying the calculated coefficients in the motion mode selection unit 66 to _{P α 1 · MV P ~α 3} · MV P, coded motion vector _{MV B1} ~ The adjustment vectors β _{1 to} β ₃ representing the error from the MV _B3 are calculated. A method for calculating the adjustment vector β will be described later.

The motion compensation prediction unit 68 performs motion compensation using the motion vector (α · MV _P + β) of each macroblock represented by the coefficient α and the adjustment vector β, generates a prediction image, and the differencer 12 adds To the device 14.

  The variable length encoding unit 90 encodes information representing the motion mode selected by the motion mode selection unit 66, the coefficient α of each macroblock, and the adjustment vector β, and includes them in the encoded stream.

  Next, a method for obtaining the coefficient α by the motion mode selection unit 66 will be described with reference to FIGS. 4 to 6 use the same reference numerals as those in FIG. 2 for the conventional motion vector, and a description common to FIG. 2 is omitted.

4A and 4B show a case where the macroblock is moving at a constant speed. When estimated as moving at constant velocity by the kinetic mode selection unit 66, the reference motion vector MV _P of the backward reference frame 205, by allocating in proportion to the time interval between frames, the coefficient of the coding target frame 202 to 204 α _{1 to} α ₃ can be obtained. Motion vector MV _P of the reference macro block 215 of the backward reference frame 205, between the time difference t between the backward reference frame 205 and the forward reference frame 201, because the reference macro block 215 is indicative of the amount and direction of movement, macro When it is estimated that the block is moving at a constant velocity, in the encoding target frames 202 to 204, the movement of MV _P × (tr / t) between the time difference tr between each encoding target frame and the forward reference frame 201 It is predicted that Therefore, as shown in FIG. 4A, when there are three frames to be encoded, α ₁ = 0.25 in the B ₁ frame 202, α ₂ = 0.5 in the B ₂ frame 203, B ₃ In the frame 204, α ₃ = 0.75 is calculated.

Thus, since the coefficients α _{1 to} α ₃ can be obtained by calculation in the uniform velocity motion mode, instead of encoding the motion vectors MV _{B1 to} MV _B3 , information about the uniform velocity motion mode and the reference motion vector and MV _P may be encoded.

FIGS. 5A and 5B show a case where the macro block is moving at a constant acceleration. When estimated as uniformly accelerated motion by the motion mode selection unit 66, the reference motion vector MV _P of the backward reference frame 205, by allocating based on a time interval between frames, the coefficient of the coding target frame 202 to 204 alpha it can be obtained ₁ to? _3. When it is estimated that the macroblock is moving at the same acceleration, in the encoding target frames 202 to 204, MV _P × (tr ² / t ²⁾ between the time differences tr of the encoding target frames and the forward reference frame 201. ) Is predicted to show movement. In other words, the coefficient α is a value proportional to the square of the time interval between frames. Therefore, as shown in FIG. 5A, when there are three frames to be encoded, α ₁ = 0.0625 in the B ₁ frame 202, α ₂ = 0.25 in the B ₂ frame 203, and B ₃ In the frame 204, α ₃ = 0.5625 is calculated.

Thus, since the coefficients α _{1 to} α ₃ can be obtained by calculation even in the uniform acceleration motion mode, instead of encoding the motion vectors MV _{B1 to} MV _B3 , information about the uniform acceleration motion mode and the reference motion vector and MV _P may be encoded.

FIGS. 6A and 6B show a case where the macroblock is moving irregularly. In this case, since the coefficients α _{1 to} α ₃ cannot be obtained from the calculation, the ratios c _{1 to} c ₃ described above are regarded as the coefficients for the encoding target frames 202 to 204. That is, α ₁ = c ₁ , α ₂ = c ₂ , and α ₃ = c ₃ . Irregular when the motion mode, instead of coding the motion vector _MV B1 _{to MV B3,} encodes the reference motion vector MV _P and coefficient alpha ₁ to? _3.

FIG. 7 is a diagram illustrating a method for calculating the adjustment vector β. Taking the B ₂ frame 203 as an example, the adjustment vector β ₂ includes a motion vector MV _B2 222 obtained by a normal method and α ₂ · MV _P 223 obtained by multiplying the reference motion vector MV _P by a coefficient α ₂ . It corresponds to the error. That is, since the movement speed of the macroblock is not always constant among a plurality of frames, the adjustment vector β is set to represent the difference between the predicted movement position of the macroblock and the actual movement position. Ask.

FIG. 8 is a flowchart showing a motion vector encoding method according to the present embodiment. First, the motion compensation unit 60 for each macroblock in the backward reference frame 205, calculates a reference motion vector MV _P relative to the forward reference frame 201, these movements are stored in the vector holding unit 62 (S10). The block matching unit 61 specifies each block to be subjected to motion compensation prediction in the encoding target frames 202 to 204 by block matching with the macroblock in the backward reference frame 205 (S11). The motion vector calculation unit 63 obtains encoding target motion vectors MV _{B1 to} MV _{B3 based} on the forward reference frame 201 for each identified block (S12). Ratio calculation unit 64, for each of a plurality of encoding target frame, determining the ratio _c 1 to c ₃ of the component and the reference motion vector MV _P component of coded motion vectors _MV B1 _{~MV B3} (S14). The motion analysis unit 65 analyzes the ratios c _{1 to} c ₃ to estimate the motion state of each block between a plurality of encoding target frames, and the motion mode selection unit 66 represents a motion mode representing the estimated motion state. Is selected (S16). Motion mode selection unit 66, according to the selected motion mode, obtains the coefficients alpha ₁ to? ₃ for obtaining the coding target motion vector _MV B1 _{to MV B3} from the reference motion vector MV _P (S18). Further, the error calculating unit 67, the coding target motion vector _MV B1 _{to MV B3,} the reference motion vector MV _P to represent the error between the vector obtained by multiplying the coefficient alpha ₁ to? ₃ adjustment vector β ₁ ~β ₃ Is obtained (S20). The motion mode, the coefficient α, and the adjustment vector β are output to the variable length encoding unit 90 and encoded according to the above-described procedure, and the encoded information is included in the encoded stream (S22).

  Note that, as described above, the comparison between the encoding target motion vector component and the reference motion vector component is performed both in the horizontal direction and in the vertical direction. When the motion of the object is linear, the ratio between the horizontal direction and the vertical direction is almost equal. Therefore, the code amount can be further reduced by making the motion mode, the coefficient α, and the adjustment vector β the same. . If the motion of the object is not linear, the motion mode, coefficient α, and adjustment vector β are encoded for each of the horizontal and vertical components.

FIG. 9 shows an example of the correspondence between motion modes and codes. In this example, the variable length coding unit 90 gives a 2-bit code to each motion mode. Here, the motion mode represented as “normal” in the figure is a mode in which the motion vector MV _B detected by the conventional method is used as it is.

  The variable length encoding unit 90 does not need to encode the coefficient α when the constant velocity motion mode and the constant acceleration motion mode are selected. In the case of the irregular motion mode, it is necessary to encode all the coefficients α for each encoding target frame.

The coefficient α may be encoded as it is. However, since the value of the coefficient α is usually a decimal number, there is a possibility that the amount of code may be increased if it is encoded. Therefore, when encoding the coefficient α, the variable-length encoding unit 90 selects a constant closest to the coefficient α among a plurality of constants to which a variable-length code is assigned in advance, and is assigned to the selected constant. The encoded code may be included in the encoded data of the moving image. FIG. 10 shows an example. As shown in the figure, constants such as “1”, “1/2”, “1/3”, “1/4”, and “1/5” are prepared in advance as logarithms for approximating the coefficient α. A variable length code is assigned. Then, the variable length encoding unit 90 selects a constant closest to the coefficient α received from the motion compensation unit 60. Then, the variable length code assigned to the constant is included in the encoded data. As a result, the code amount of the coefficient α can be reduced.
In this case, the accuracy of the motion vector is reduced, but if the adjustment vector β is calculated for each macroblock, the error is absorbed by the adjustment vector β, so that the accuracy of motion compensation prediction is reduced. Absent.

  When the adjustment vector β is encoded, the variable length encoding unit 90 may add a variable length code corresponding to the appearance frequency, or may use fixed length encoding.

  The variable length encoding unit 90 may encode only one motion mode information for a plurality of frames. That is, since a motion vector is obtained for a frame in one GOP based on a certain reference frame, it is sufficient to determine one motion mode for each GOP. The coefficient α and the adjustment vector β need to be encoded for each macroblock. However, when the motion mode of the macroblock is the constant velocity motion mode or the constant acceleration motion mode, it is not necessary to encode the coefficient α.

  Collecting exercise modes is not limited to GOP units. For example, when it is determined as the irregular motion mode, a plurality of frames in which the motion state of the object is constant may be selected in the GOP, and one motion mode may be determined in the plurality of frames.

  FIG. 11 is a configuration diagram of the decoding device 300 according to the embodiment. These functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding apparatus 300 receives an input of the encoded stream, decodes the encoded stream, and generates an output image.

  The variable length decoding unit 310 performs variable length decoding on the input encoded stream, supplies the decoded image data to the inverse quantization unit 320, and supplies motion vector information to the motion compensation unit 360.

  The inverse quantization unit 320 inversely quantizes the image data decoded by the variable length decoding unit 310 and supplies the image data to the inverse DCT unit 330. The image data inversely quantized by the inverse quantization unit 320 is a DCT coefficient. The inverse DCT unit 330 restores the original image data by performing inverse discrete cosine transform (IDCT) on the DCT coefficients inversely quantized by the inverse quantization unit 320. The image data restored by the inverse DCT unit 330 is supplied to the adder 312.

  When the image data supplied from the inverse DCT unit 330 is an I frame, the adder 312 outputs the I frame image data as it is and also serves as a reference image for generating a predicted image of a P frame or a B frame. And stored in the frame memory 380.

  When the image data supplied from the inverse DCT unit 330 is a P frame, the adder 312 is supplied from the differential image supplied from the inverse DCT unit 330 and the motion compensation unit 360 because the image data is a difference image. By adding the predicted images, the original image data is restored and output.

  The motion compensation unit 360 generates a predicted image of P frame or B frame using the motion vector information supplied from the variable length decoding unit 310 and the reference image stored in the frame memory 380, and supplies the predicted image to the adder 312. To do.

  FIG. 12 is a configuration diagram of the motion compensation unit 360. Hereinafter, the operation of the motion compensation unit 360 for decoding the B frame encoded in the present embodiment will be described. The motion compensation unit 360 detects the motion vector of each macroblock of the backward reference frame when performing motion compensation of the backward reference frame, and the motion vector information and macroblock information of the backward reference frame already detected are detected. It is held in the motion vector holding unit 364.

  The motion vector acquisition unit 361 acquires motion vector information from the variable length decoding unit 310. This motion vector information includes the motion mode, the proportional coefficient α, and the adjustment vector β described above. The motion vector acquisition unit 361 provides motion vector information to the motion vector calculation unit 362. By including the motion mode in the encoded stream, the motion compensation unit 360 restores the original motion vector from the proportional coefficient α and the adjustment vector β even if a plurality of motion modes are included in one encoding target frame. can do.

  The motion vector calculation unit 362 acquires the motion vector of the macroblock of the backward reference P frame from the motion vector holding unit 364, and calculates the motion vector of the encoding target frame. The calculated motion vector is given to the motion compensation prediction unit 366 and held in the motion vector holding unit 364 for use in calculation of motion vectors of other frames.

  The motion compensation prediction unit 366 generates a prediction image of the encoding target frame using the received motion vector and outputs the prediction image to the adder 312.

  As described above, according to the present embodiment, the motion vector of the encoding target frame (B frame) is expressed using the motion vector of the backward reference frame (P frame). Therefore, for the B frame, it is not necessary to encode the motion vector itself, and only the coefficient α, the adjustment vector β, and the motion mode need be encoded. In addition, in the case of the constant velocity motion mode or the constant acceleration motion mode, the value of the coefficient α is obtained by the ratio of the frame interval, so there is no need to encode the coefficient α, and only the adjustment vector β and the motion mode are encoded. It is enough to make it.

  In recent high-quality compression coding, a search for a motion vector with 1/4 pixel accuracy is often performed, and the amount of code of motion vector information further increases. According to the present embodiment, although the amount of calculation processing required for encoding increases, the code amount of motion vector data is reduced, so that the data amount of the encoded stream is reduced and the encoding efficiency of moving images is improved.

  In the conventional direct mode, only constant velocity motion is supported. However, according to the present embodiment, it is possible to cope with acceleration motion or complex motion, and the code amount of motion vector data can be reduced. Can do.

  In the above description, the case of forward prediction of the B frame has been described. However, the present embodiment can be applied to backward prediction in the same procedure. In addition, in the case of bidirectional prediction as well as unidirectional motion prediction, the present embodiment is applied to the encoding of two independent motion vectors indicating the motion for each of the forward reference frame and backward reference frame. Can do. That is, a plurality of motion vectors may be prepared for each of forward prediction and backward prediction in the same manner as in the embodiment.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above description, the encoding device 100 and the decoding device 300 are MPEG series standards (MPEG-1, MPEG-2, and MPEG-4), H.264, and H.264. 26x series standards (H.261, H.262 and H.263), or H.264 Although video encoding and decoding are performed in accordance with H.264 / AVC, the present invention can also be applied to the case of hierarchical video encoding and decoding with temporal scalability. In particular, the present invention is effective in reducing the amount of motion vector codes in the encoding of motion vectors when the MCTF technique is used.

  In the above-described embodiment, it has been described that the motion mode is estimated by analyzing the ratio c. However, the motion compensation of the encoding target frame is performed using a plurality of motion vectors according to a plurality of motion modes to obtain a prediction image, and the code amount of the difference image, which is the difference between the prediction image and the original image, is minimized. You may comprise so that the motion mode from which the motion vector which was obtained may be selected. Hereinafter, this method will be described.

A code amount estimation unit (not shown) estimates a code amount when a differential image that is a difference between a predicted image and an original image is encoded in the constant velocity motion mode or the constant acceleration motion mode. The estimated code amount is held in a code amount holding unit (not shown) in association with the motion mode.
A motion vector selection unit (not shown) compares the code amounts of the difference images held in the code amount holding unit, and selects the motion mode with the smallest code amount. As a result, the code amount of the encoded data of the moving image can be reduced and the encoding efficiency can be improved.

As another method, the motion vector encoding according to the motion mode described above may be executed only when the code amount of the difference image is smaller than the motion vector MV _B obtained by the normal procedure.

Specifically, first, the motion compensation prediction unit 68 calculates a normal motion vector MV _B , and a code amount estimation unit (not shown) uses the motion vector MV _B to generate a prediction image. The code amount is calculated. Subsequently, the motion compensation prediction unit 68 calculates the motion vector α · MV _P according to the constant velocity motion mode or the constant acceleration motion mode, and the code amount estimation unit uses the motion vector α · MV _P to calculate the predicted image. The code amount of the difference image when it is generated is calculated. Then, the code amounts of the two difference images are compared, and only when the code amount of the motion vector α · MV _{P in} the constant velocity motion mode or the constant acceleration motion mode is smaller, the constant velocity motion mode or the constant acceleration motion mode Select one of the following.

  In the above-described embodiment, the case of detecting a motion vector in units of macroblocks has been described. However, in the case of detecting a motion vector in units of blocks (8 × 8 pixels or 4 × 4 pixels), and in the case of detecting motion vectors in units of objects. The present invention can also be applied to such cases.

  In the above description, it has been described that the motion vector is expressed for each macroblock in the encoding target frame using the motion vector of the backward reference frame. However, the same applies not only to the macroblock unit but also to other regions in the frame, for example, a slice as an encoding unit and a region of interest (ROI) set on a moving image by a ROI region setting unit (not shown). The motion vector may be obtained by this method.

Specifically, the motion compensation unit 60 stores for each slice or region of interest in the backward reference frame 205, calculates a reference motion vector MV _P relative to the forward reference frame 201, these motion vector holding unit 62 To do. The block matching unit 61 specifies a slice or attention area that is a target of motion compensation prediction in the encoding target frames 202 to 204 by matching with a slice or attention area in the backward reference frame 205. The motion vector calculation unit 63 obtains encoding target motion vectors MV _{B1 to} MV _{B3 based} on the forward reference frame 201 for each identified slice or region of interest. Ratio calculation unit 64, for each of a plurality of encoding target frame, determining the ratio _c 1 to c ₃ of the component and the reference motion vector MV _P component of coded motion vectors _MV B1 _{to MV B3.} The motion analysis unit 65 analyzes the ratios c _{1 to} c ₃ to estimate the motion state of each slice or region of interest between the plurality of encoding target frames, and the motion mode selection unit 66 determines the estimated motion state. Select the motion mode to represent. Motion mode selection unit 66, according to the selected motion mode, obtains the coefficients alpha ₁ to? ₃ for obtaining the coding target motion vector _MV B1 _{to MV B3} from the reference motion vector MV _P. Further, the error calculating unit 67, the coding target motion vector _MV B1 _{to MV B3,} the reference motion vector MV _P to represent the error between the vector obtained by multiplying the coefficient alpha ₁ to? ₃ adjustment vector β ₁ ~β ₃ Ask for. The motion mode, the coefficient α, and the adjustment vector β are output to the variable length encoding unit 90 and encoded according to the above-described procedure, and the encoded information is included in the encoded stream for each slice or region of interest. .

  Furthermore, instead of determining the motion mode for each macroblock in the encoding target frame, the motion mode may be determined in units of frames or GOPs. In this case, there are the following two procedures.

Procedure 1. The motion compensation unit 60 performs encoding in frame units or GOP units for each candidate motion mode. That is, encoding is executed by applying one motion mode to all macroblocks or intra-frame regions. At this stage, encoded data is not output, and only the code amount of the encoded data is held in a code amount holding unit (not shown). After calculating the code amount of the encoded data for all the motion modes, the motion mode selection unit 66 selects a motion mode that minimizes the code amount. Then, the motion compensation prediction unit 68 performs encoding again according to the selected motion mode, and encoded data is output at this stage.

Procedure 2. The motion compensation unit 60 performs encoding in frame units or GOP units for each candidate motion mode. That is, encoding is executed by applying one motion mode to all macroblocks or intra-frame regions. At this stage, encoded data is not output, but the encoded data itself and the code amount of the encoded data are held in the code amount holding unit. After calculating the code amount of the encoded data for all the motion modes, the motion mode selection unit 66 selects a motion mode that minimizes the code amount. Then, encoded data corresponding to the selected motion mode is output from the code amount holding unit.

  Of the procedures 1 and 2, the amount of calculation required for the encoding of the procedure 1 is larger than that of the procedure 2 by the amount of re-encoding after selecting the motion mode. However, since the procedure 2 needs to hold the encoded data itself and the code amount for each motion mode, a larger storage area than the procedure 1 is required. Thus, since procedure 1 and procedure 2 are in a trade-off relationship, an appropriate one may be selected according to the situation.

  Furthermore, in the encoding according to the MCTF technique described above, the method of the present invention can be applied to motion vectors between a plurality of frames included in an encoding layer generated by MCTF.

  This will be described with reference to FIG. FIG. 13 shows a state where four frames 101 to 104 are encoded according to the MCTF technique, and shows images and motion vectors output in each layer.

An MCTF processing unit (not shown) sequentially acquires two consecutive frames 101 and 102 and generates a high-frequency frame 111 and a low-frequency frame 112. Also, the two frames 103 and 104 are sequentially acquired to generate a high-frequency frame 113 and a low-frequency frame 114. These are referred to as level 1. The MCTF processing unit detects the motion vector MV _1a from the two frames 101 and 102, and detects the motion vector MV _1b from the frames 103 and 104.

The MCTF processing unit further generates a high-frequency frame 121 and a low-frequency frame 122 from the low-frequency frames 112 and 114 in layer 1. These are referred to as level 2. The MCTF processing unit detects a motion vector MV ₀ from the two low frequency frames 112 and 114.

  In FIG. 13, the motion vector is detected in units of frames for the sake of simplicity. However, the motion vector may be detected in units of macroblocks, or blocks (8 × 8 pixels or 4 × 4 pixels). ) A motion vector may be detected in units.

When there is an MCTF layer as shown in FIG. 13, the above method is applied to encoding of the motion vectors MV _1a and MV _1b of the layer 1. Since the motion vectors MV _1a and MV _{1b in the} hierarchy 1 are half the distance in the motion vector MV _{0 in} the hierarchy 0, the motion is also estimated to be half. Therefore, the motion vectors MV _1a and MV _1b are calculated by the following calculation formula.
MV _1a = (1/2) · MV ₀ + β _a
MV _1b = (1/2) · MV ₀ + β _b
Here, β _a and β _b are adjustment vectors representing deviations from the predicted values. Therefore, instead of encoding the motion vectors MV _1a and MV _1b of the layer 1, the motion vector MV ₀ and the adjustment vectors β _a and β _b of the layer 0 may be encoded.

As can be seen from the above equation, layer 1 cannot be encoded until motion vector MV ₀ of layer ₀ is obtained. Therefore, it is necessary to hold the motion vector information and difference information of layer 1 until the motion vector MV ₀ of layer 0 is obtained.

  Even when there are three or more MCTF layers, the method of the present invention can be applied to motion vectors in layers other than layer 0.

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure explaining the calculation order of the conventional motion vector. It is a figure explaining the structure of the motion compensation part of FIG. (A), (b) is a figure explaining the coefficient (alpha) of constant velocity motion mode. (A), (b) is a figure explaining the coefficient (alpha) of a uniform acceleration motion mode. (A), (b) is a figure explaining the coefficient (alpha) of irregular motion mode. It is a figure explaining the calculation method of adjustment vector (beta). 5 is a flowchart of a motion vector encoding method according to the embodiment. It is a table which shows an example of the motion mode of a motion vector, and its code | symbol. It is a table which shows an example of the code | symbol allocated to the constant for approximating coefficient (alpha), and a constant. It is a figure which shows the structure of the decoding apparatus which concerns on embodiment. It is a block diagram of the motion compensation part of FIG. It is a figure which shows a mode that four frames are encoded according to MCTF technique.

Explanation of symbols

  10 block generation units, 12 differentiators, 14 adders, 20 DCT units, 30 quantization units, 40 inverse quantization units, 50 inverse DCT units, 60 motion compensation units, 61 block matching units, 62 motion vector holding units, 63 Motion vector calculation unit, 64 ratio calculation unit, 65 motion analysis unit, 66 motion mode selection unit, 67 error calculation unit, 68 motion compensation prediction unit, 80 frame memory, 90 variable length coding unit, 100 coding device, 201 I Frame (forward reference frame), 203 B frame (encoding target frame), 205 P frame (backward reference frame), 300 decoding device, 310 variable length decoding unit, 312 adder, 320 inverse quantization unit, 330 inverse DCT unit 360 motion compensation unit, 361 motion vector acquisition unit, 62 motion vector calculating unit, 364 a motion vector holding unit, 366 a motion compensation prediction unit, 380 a frame memory.

Claims (11)

A method of encoding a moving picture, comprising:
When a reference motion vector that is a motion vector of a block in the second reference picture based on the first reference picture is obtained,
Obtaining an encoding target motion vector that is a motion vector based on the first reference picture for a block in the encoding target picture;
For each of a plurality of encoding target pictures, obtain a ratio between the encoding target motion vector component and the reference motion vector component;
Referring to the ratio, each block stop between a plurality of coded pictures, from among the uniform motion, a predetermined motion mode indicating whether is in one state of the uniformly accelerated motion or irregular movements, Select one exercise mode,
It marks Goka how to characterized the inclusion of code for the selected motion mode predetermined for the encoded data of a moving image. Each of the blocks to be subjected to motion compensation prediction in a plurality of encoding target pictures by matching with a block in the second reference picture,
For a particular block, the encoding method according to claim 1, characterized in that obtaining the encoding target motion vector. It is characterized by estimating whether each block is in a stopped state, a constant velocity motion, a constant acceleration motion, or an irregular motion by taking a difference of ratios obtained in adjacent pictures to be encoded. The encoding method according to claim 1 or 2 . Obtains the coefficients for obtaining the encoding target motion vector from the reference motion vector according to the motion mode, any one of claims 1, characterized in the inclusion of information the coefficient in the encoded data of a moving picture 3 The encoding method described in 1. If the as motion mode, the uniform velocity motion mode indicating that the block is constant velocity motion among a plurality of coded pictures is selected,
The encoding method according to claim 4 , wherein the coefficient is determined based on a time interval between the first reference picture and the encoding target picture. If the as motion mode, the constant acceleration motion mode indicating that the block is uniformly accelerated motion across multiple encoding target picture is selected,
The encoding method according to claim 4 , wherein the coefficient is determined based on a time interval between the first reference picture and the encoding target picture. When encoding the coefficient, a constant closest to the coefficient is selected from a plurality of constants to which a variable-length code is assigned in advance, and the code assigned to the selected constant is used as the encoded data of the moving image. The encoding method according to claim 4 , wherein the encoding method is included. An adjustment vector representing an error between the encoding target motion vector and a vector obtained by multiplying the reference motion vector by the coefficient is obtained, and information on the adjustment vector is included in encoded data of a moving image encoding method according to any one of claims 4 to 7. The encoding according to claim 8 , wherein one code of the motion mode is selected for a plurality of pictures, and the information on the coefficient and the adjustment vector is encoded for each encoding target motion vector. Method. A method of encoding a moving picture, comprising:
When a reference motion vector that is a motion vector of a second reference picture based on the first reference picture is obtained,
For the encoding target picture, obtain an encoding target motion vector that is a motion vector based on the first reference picture;
For each of a plurality of encoding target pictures, obtain a ratio between the encoding target motion vector component and the reference motion vector component;
Referring to the ratio, indicating whether each picture relevant Yama other among a plurality of encoding target picture is in one of states of a plurality of pictures stopped, constant velocity motion, uniformly accelerated motion or irregular motion Select one exercise mode from the predetermined exercise modes,
An encoding method comprising : a predetermined code for a selected motion mode is included in encoded data of a moving image. In an encoding method for recursively performing motion compensation time filtering on a moving picture picture to obtain a plurality of layers having different frame rates,
When a reference motion vector that is a motion vector of the second reference picture based on the first reference picture included in the hierarchy is obtained,
For an encoding target picture included in the hierarchy, obtain an encoding target motion vector that is a motion vector based on the first reference picture;
For each of a plurality of encoding target pictures, obtain a ratio between the encoding target motion vector component and the reference motion vector component;
Referring to the ratio, from among a predetermined motion mode indicating whether each picture is in a stopped state, a constant velocity motion, a constant acceleration motion, or an irregular motion among a plurality of encoding target pictures Select one exercise mode,
An encoding method comprising : a predetermined code for a selected motion mode is included in encoded data of a moving image.

Images