JP2008011455A - Coding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a coding technique for a moving image which is high in coding efficiency by decreasing a code amount resulting from motion vector information, and decoding technique. <P>SOLUTION: A local motion vector detection unit 66 obtains a local motion vector LMV for each macro block in a coding target image to be coded. A region setting unit 64 sets multiple global regions in a frame image. A global motion vector calculation unit 68 calculates a global motion vector GMV within each global region. An update decision unit 50 generates a reference table by imparting indexes to global motion vectors. A motion compensating prediction unit 70 determines whether motion compensation is performed by using one of a global motion vector determined in the reference table and a local motion vector for each macro block with respect to the macro block. A local motion vector coding unit 72 adds a GMV use flag indicating whether the motion compensation is performed by using the global motion vector to coded data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding a moving image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. There is a next-generation image compression technology such as SVC that has both a high-quality stream and a low-quality stream in one stream.

  When streaming a high-resolution moving image or storing it in a recording medium, it is necessary to increase the compression rate of the moving image stream so as not to compress the communication band or increase the storage capacity. In order to enhance the compression effect of moving images, motion compensation interframe predictive coding is performed. In motion-compensated interframe predictive coding, the encoding target frame is divided into blocks, the motion from a reference frame that has already been encoded is predicted for each block, a motion vector is detected, and motion vector information is encoded along with the difference image. Turn into.

Patent Document 1 discloses an encoding technique that reduces the amount of vector data information by treating a block in a frame that is related to the size and direction of a motion vector as one object.
JP 2004-48306 A

  H. In the H.264 / AVC standard, in order to perform more detailed prediction in motion compensation, the block size of motion compensation can be made variable, and the pixel accuracy of motion compensation can be reduced to ¼ pixel accuracy. Therefore, the amount of code related to the motion vector increases. In addition, in SVC (Scalable Video Coding), which is a next-generation image compression technology, MCTF (Motion Compensated Temporal Filtering) technology is being studied in order to improve temporal scalability. This is a combination of subband division in the time axis direction and motion compensation. Since hierarchical motion compensation is performed, information on motion vectors becomes very large. As described above, the recent video compression coding technology tends to increase the data amount of the entire video stream due to an increase in the amount of information related to motion vectors, and there is a further demand for a technology for reducing the amount of codes resulting from motion vector information. ing.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a moving picture coding technique and a decoding technique with high coding efficiency by reducing the amount of codes resulting from motion vector information.

  According to an encoding method of an aspect of the present invention, there is provided a global motion vector indicating global motion in a picture constituting a moving image in a picture that is inter-picture predictively encoded or in a region defined in the picture, and the global motion vector. Information of a table that stores an index for specifying a global motion vector in association with each other and information of an index designated for each region in a picture are included in the encoded data of the moving image.

  The “global motion vector” is a vector that represents the motion of the entire region.

  “Picture” is a unit of encoding, and its concept includes a frame, a field, a VOP (Video Object Plane), and the like.

  According to this aspect, a table in which a global motion vector and an index are made to correspond in units of pictures or for each region defined in the picture, and the global motion vector used for predictive coding in each of the regions in the picture is created. Can be specified by index. By using a table, it is not necessary to include global motion vector information redundantly in the encoded data, so the amount of data resulting from the global motion vector can be reduced, and the compression efficiency of the encoded data of the moving image can be increased. it can.

  Another aspect of the present invention is also an encoding method. In this method, information representing a global motion vector indicating a global motion within a picture that is inter-picture predictively encoded or an area defined within a picture, and a global motion vector, And index information for specifying the index information indicating the global motion vector used for predictive coding in the picture or the area included in the picture in which the global motion vector is calculated. It is included in the encoded data of the moving image.

  According to this aspect, since it is not necessary to include the same global motion vector information in the encoded data in the same picture or in the picture area, the amount of data resulting from the global motion vector can be reduced.

  When the first global motion vector calculated in the first picture or in the first picture area is different from the second global motion vector calculated in the second picture or in the second picture area, Difference information between the first global motion vector and the second global motion vector may be included in the encoded data in association with the index. In this way, the data amount of the global motion vector information itself can be reduced, and the compression efficiency of moving image encoded data can be increased.

  The global motion vectors in the second picture or the second picture area are rearranged in the order of the appearance frequency of the global motion vectors in the first picture or the first picture area, and an index is assigned according to this order. May be.

  The picture or picture from which the global motion vector is calculated, including information indicating the global motion vector and index information for identifying the global motion vector in the header of the encoded data, the header of the picture, or the header of the area within the picture The header of the area included in the inner area may include index information indicating a global motion vector used for predictive coding in the area. The global motion vector and index information is included in the header of a smaller coding unit such as each stream, each picture, or a slice or macroblock, and the global motion vector is specified using an index in a lower coding unit. be able to. Therefore, the combination of the coding unit for calculating the global motion vector and the coding unit using the global motion vector for predictive coding can be arbitrarily selected according to the nature of the moving image.

  For example, information representing a global motion vector and index information for identifying the global motion vector are included in the header of the picture, and the global motion vector used for predictive coding is indicated in the header of the slice or block in the picture. Index information may be included. Alternatively, a header of a slice in a picture includes information indicating a global motion vector and index information for specifying the global motion vector, and a global motion vector used for predictive encoding is included in a header of a block in the slice. Information on the index to be pointed may be included.

  For each picture or a region defined in a picture, a flag indicating whether or not to perform predictive coding using a global motion vector may be included in the coded data. Further, for each block in the picture, a flag indicating which of the global motion vector indicated by the index and the local motion vector calculated for the block is used for predictive encoding may be included in the encoded data. . In this way, motion prediction can be performed using either a local motion vector or a global motion vector in units of blocks in a picture, and the data amount of encoded data may be further reduced.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the encoding efficiency of moving images and perform motion compensation with high accuracy.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The coding apparatus 100 according to the present embodiment is a moving picture experts group (MPEG-1) series standard (MPEG-1, MPEG) standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission). -2 and MPEG-4), standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector), which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264 respectively).

  In the MPEG series standard, an image frame for intra-frame encoding is an I (Intra) frame, an image frame for forward inter-frame predictive encoding with a past frame as a reference image, a P (Predictive) frame, and past and future An image frame that performs bidirectional inter-frame predictive coding using this frame as a reference image is referred to as a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference image may be a past two frames as a reference image or a future two frames as a reference image regardless of the time. Further, three or more frames can be used as the reference image regardless of the number of frames that can be used as the reference image. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame because the time of the reference image does not matter before and after.

  In the embodiment, a frame is used as an example of the encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, encodes the moving image, and outputs an encoded stream. The input moving image frame is stored in the frame memory 80.

  The motion compensation unit 60 uses a past or future image frame stored in the frame memory 80 as a reference image, performs motion compensation for each macroblock of the P frame or the B frame, and generates a motion vector and a predicted image. . The motion compensation unit 60 takes the difference between the P frame or B frame image to be encoded and the predicted image, and supplies the difference image to the DCT unit 20. In addition, the motion compensation unit 60 supplies the encoded motion vector information to the multiplexing unit 92.

  The DCT unit 20 performs discrete cosine transform (DCT) on the image supplied from the motion compensation unit 60, and gives the obtained DCT coefficient to the quantization unit 30.

  The quantization unit 30 quantizes the DCT coefficient and provides it to the variable length coding unit 90. The variable length coding unit 90 performs variable length coding on the quantized DCT coefficient of the difference image and supplies the result to the multiplexing unit 92. The multiplexing unit 92 multiplexes the encoded DCT coefficient given from the variable length coding unit 90 and the coded motion vector information given from the motion compensation unit 60 to generate an encoded stream. . When generating the encoded stream, the multiplexing unit 92 performs a process of rearranging the encoded frames in time order.

  In the case of P frame or B frame encoding processing, the motion compensation unit 60 operates as described above. However, in the case of I frame encoding processing, the motion compensation unit 60 does not operate and is not shown here. The I frame is supplied to the DCT unit 20 after intra prediction.

Example 1.
FIG. 2 is a diagram illustrating the configuration of the motion compensation unit 60. The motion compensation unit 60 detects a motion vector (hereinafter referred to as a “local motion vector”) of the macroblock unit of the encoding target frame, and for each predetermined region provided on the image, A motion vector indicating a smooth motion (hereinafter referred to as “global motion vector”) is obtained. The global motion vector is a vector that represents the motion of the entire region. For example, the global motion vector of each region may represent a local motion vector in units of individual macroblocks within that region.

  The motion compensation unit 60 performs motion compensation based on the local motion vector or the global motion vector, outputs a difference image, encodes the local motion vector and the global motion vector, and outputs them as motion vector information.

  In this embodiment, after obtaining one or a plurality of global motion vectors within a frame, a reference table is created in which the global motion vector and an index for identifying the global motion vector are associated with each other in the entire frame. FIG. 3 shows an example of a reference table. In FIG. 3, the index “0” corresponds to the global motion vector (33.50, −5.75), and the index “1” corresponds to the global motion vector (5.25, 0).

  When a global motion vector is used for predictive encoding of each macroblock, an index is stored in the encoded stream. As a result, when macroblocks included in different areas use the same global motion vector in duplicate, the global motion vector is not encoded a plurality of times, and the amount of codes can be reduced. Further, in this embodiment, a flag indicating which one or a plurality of global motion vectors or local motion vectors defined in the reference table is used for predictive encoding is set in an encoded stream for each macroblock. Store. This makes it possible to select one or a plurality of global motion vectors and local motion vectors that minimize the code amount of the macro block in units of macro blocks.

  Returning to FIG. 2, the local motion vector detection unit 66 refers to the reference image held in the frame memory 80 and detects a predicted macroblock having the smallest error with respect to the target macroblock of the encoding target image from the reference image. Then, a local motion vector LMV indicating the motion from the target macroblock to the predicted macroblock is obtained. Motion detection is performed by searching for a reference macroblock in a reference image matching the target macroblock in units of integer pixels or decimal pixels. The search is normally repeated a plurality of times within the pixel region, and the reference macroblock that best matches the target macroblock is selected as the predicted macroblock in the plurality of searches.

  The local motion vector detection unit 66 gives the obtained local motion vector LMV to the global motion vector calculation unit 68, the motion compensation prediction unit 70, and the local motion vector encoding unit 72.

  The region setting unit 64 sets a region (hereinafter referred to as “global region”) for obtaining the global motion vector GMV on the frame image. A plurality of global areas are provided in the image. For example, the area setting unit 64 may set a predetermined area as the global area, such as setting one area around the center of the frame image as one global area and setting the periphery as another global area. The global area may be set by the user.

  In addition, when an object such as a person is captured in the image, the region setting unit 64 may automatically extract a region where the object is captured and set it as a global region.

  The region setting unit 64 refers to the local motion vector LMV in the image detected by the local motion vector detection unit 66, automatically extracts a region occupied by a macroblock having a certain amount of motion, and sets the region as a global region. May be.

  The region setting unit 64 provides information on the set global region to the global motion vector calculation unit 68 and the global motion vector encoding unit 74.

  The global motion vector calculation unit 68 calculates a global motion vector GMV indicating a global motion in each global region set by the region setting unit 64. For example, the global motion vector calculation unit 68 obtains an average of local motion vectors LMV in the region and sets it as the global motion vector GMV.

  Further, the global motion vector calculation unit 68 may acquire information on global motion in each global region and calculate a global motion vector GMV for each global region based on the information. For example, when the camera is zoomed or panned, the screen is scrolled, etc., the global motion in each global region can be determined from the information regarding the motion of the entire screen, and the global motion vector GMV can be calculated. . The global motion vector calculation unit 68 may automatically extract the motion of an object such as a person on the screen, determine a global motion in each global region from the motion of the object, and calculate a global motion vector GMV. it can.

  The global motion vector calculation unit 68 gives the obtained global motion vector GMV to the update determination unit 50.

  The update determination unit 50 adds an index to the global motion vector received from the global motion vector calculation unit 68 and creates the above-described reference table. The update determination unit 50 provides the indexed global motion vector GMV to the motion compensation prediction unit 70 and the global motion vector encoding unit 74.

  The update determination unit 50 also compares the global motion vector received from the global motion vector calculation unit 68 with the global motion vector calculated in the frame processed before the processing target frame, and determines the global motion vector in the reference table. It is determined whether or not to update the motion vector.

  A global motion vector (hereinafter referred to as “old global motion vector”) calculated in the previously processed frame is held in the global motion vector temporary storage unit 52. If the old global motion vector is the same value as the current global motion vector, the update determination unit 50 does not change the correspondence between the old global motion vector and the index in the reference table, and the global motion vector is included in the encoded stream. Stores a flag indicating that the value of does not change. If the current global motion vector is different from the old global motion vector, a new index is assigned to the current global motion vector.

  The motion compensated prediction unit 70 generates a predicted image by performing motion prediction on the target macroblock using the local motion vector LMV or the global motion vector GMV, and outputs a difference image between the encoding target image and the predicted image to the DCT unit 20. .

  The motion compensation prediction unit 70 includes a motion vector selection unit 54, a predicted image generation unit 56, and a difference image output unit 58.

  The motion vector selection unit 54 determines whether to perform motion prediction for the target macroblock using one or a plurality of global motion vectors GMV defined in the reference table and a local motion vector LMV for each macroblock. . For this purpose, the predicted image generation unit 56 generates a predicted image that has been motion-predicted using the local motion vector LMV and a predicted image that has been motion-predicted using each of the global motion vectors GMV. Create a difference image with the image. The motion vector selection unit 54 compares the code amounts of the macro blocks obtained by combining the code amount of the generated difference image and the code amount of the motion vector, and is used in generating a prediction image with the minimum code amount. Select a motion vector. The difference image output unit 58 outputs the difference image with the smallest code amount to the DCT unit 20.

  The motion vector selection unit 54 provides the local motion vector encoding unit 72 with information on which of the local motion vector LMV or the global motion vector GMV has been selected.

  The local motion vector encoding unit 72 receives the local motion vector LMV for each macroblock from the local motion vector detection unit 66. In addition, the local motion vector encoding unit 72 generates a local motion vector LMV for each macroblock for a macroblock in which the local motion vector is used to generate a predicted image with the minimum code amount in the motion compensation prediction unit 70. Variable length coding. The local motion vector encoding unit 72 supplies the encoded local motion vector LMV to the multiplexing unit 92 as motion vector information.

  In addition, the local motion vector encoding unit 72 indicates, for each macroblock, a flag indicating whether or not a global motion vector has been used to generate a predicted image with the minimum code amount (hereinafter referred to as “GMV use flag”). ) And a global motion vector, index information corresponding to the global motion vector in the reference table is given to the multiplexing unit 92.

  The global motion vector encoding unit 74 receives the indexed global motion vector GMV from the update determination unit 50 and performs variable length encoding.

  When the update determination unit 50 determines to update the global motion vector in the reference table, the global motion vector encoding unit 74 changes the global motion vector GMV determined to be updated and the corresponding index. Encode long. In this case, a flag indicating whether or not the reference table has been updated (hereinafter referred to as “table update flag”) and the number of updated global motion vectors (denoted as “Num”) are provided to the multiplexing unit 92. When the update determination unit 50 determines not to update the global motion vector, the global motion vector encoding unit 74 does not execute variable length encoding of the global motion vector.

  The global motion vector encoding unit 74 may variable-length encode the difference (GMV-GMV ') from the global motion vector GMV' before update instead of variable-length encoding the entire global motion vector GMV.

  The global motion vector encoding unit 74 gives the global motion vector GMV and the index of each global region after encoding to the multiplexing unit 92 as motion vector information. At this time, the global motion vector encoding unit 74 adds the region information about the global region set by the region setting unit 64 to a part of the motion vector information.

  The multiplexing unit 92 is given a global motion vector GMV, an index, a local motion vector LMV, and various flags.

  FIG. 4 is a flowchart illustrating the procedure of differential encoding of the motion vector by the motion compensation unit 60 according to the first embodiment. The encoding procedure will be described with reference to the examples of FIGS. 5 and 6 as appropriate.

  An encoding target image is input to the frame memory 80 of the encoding apparatus 100 (S10). The local motion vector detection unit 66 of the motion compensation unit 60 detects the local motion vector LMV for each macroblock in the encoding target image (S12).

  Next, the region setting unit 64 sets a global region on the image (S14), and the global motion vector calculation unit 68 calculates a global motion vector GMV for each global region (S16).

  The update determination unit 50 assigns an index to each global motion vector and determines whether to update the global motion vector reference table. The global motion vector encoding unit 74 performs variable length encoding by associating the global motion vector GMV with the index (S18). As will be described later, this data is stored in the header of the frame.

  If the motion compensation prediction unit 70 performs motion compensation for each target macroblock using either the global motion vector GMV indexed in S18 or the local motion vector LMV calculated for each target macroblock, the difference image It is determined whether or not the code amount of the macroblock that is the sum of the code amount and the motion vector code amount is minimized (S20). The motion compensation prediction unit 70 transmits a motion vector that minimizes the code amount of the macroblock to the local motion vector encoding unit 72.

  The local motion vector encoding unit 72 encodes the local motion vector when it is determined that the code amount of the macroblock is minimized when the local motion vector is used in S20 (S22). Furthermore, a GMV use flag indicating whether or not predictive coding has been executed using the global motion vector is given to the multiplexing unit.

  When the processing of S20 and S22 is completed for all the macroblocks in the frame (Y of S24), this flow is finished.

  FIG. 5A illustrates an example of the global area. In the example of FIG. 5A, the area setting unit 64 sets the first global area 112 and the second global area 114 on the frame 110. Further, a third global area 116 included in the second global area 114 is set. The global motion vector calculation unit 68 obtains the first global motion vector GMV1 in the first global area 112, obtains the second global motion vector GMV2 in the second global area 114, and obtains the third global motion vector GMV3 in the third global area 116. Ask for. In FIG. 5A, a global motion vector is not set in the background portion represented by a white square.

  The update determination unit 50 creates a reference table by assigning indexes to the global motion vectors GMV1 to GMV3 for each frame. FIG. 5B is an example of a reference table, and indexes 1 to 3 are assigned to the global motion vectors GMV1 to GMV3, respectively. The global motion vector information and the index are stored in the frame header of the encoded stream.

  In the case of FIG. 5A, the motion compensation prediction unit 70 performs any of the local motion vector LMV calculated for each macroblock and the global motion vectors GMV1 to GMV3 when performing predictive coding of each macroblock. Decide whether to use.

  In the figure, for the macroblock 120, since the code amount of the macroblock is minimized when the motion compensation is performed using the third global motion vector GMV3 rather than the local motion vector LMV calculated for the macroblock, the third global It is assumed that the motion vector GMV3 is selected. For the macroblock 118, the code amount of the macroblock is minimized when the motion compensation is performed using the second global motion vector GMV2 rather than the local motion vector LMV calculated for the macroblock. Assume that the motion vector GMV2 is selected.

  For macroblocks other than the macroblocks 118 and 120 in the background area, the amount of code is smaller when motion compensation is performed using the local motion vector LMV of each macroblock, so the local motion vector is selected without using the global motion vector. Is done.

  FIG. 6 is a diagram for explaining the data structure of the encoded stream 200 generated by the multiplexing unit 92. The encoded stream 200 includes a stream parameter 202, a frame header 204, and data of each frame.

  In this embodiment, reference frame information, that is, global motion vector and index information is stored in the frame header 204, and a GMV use flag 270 is stored for each macroblock in the slice data 208.

  “Num” 210 represents the number of global motion vectors stored in the frame header 204. The number of indices 212 and global motion vector (GMV) 214 represented by Num 210 follows.

  At this time, the decoding apparatus that processes the encoded stream 200 may update only the global motion vector to which the same index as the index of the reference table set in the preceding frame is processed. For example, it is assumed that the reference table in FIG. 5B is set in a frame preceding processing. When only the index 1 and the index 3 are included in the frame header 204 of the subsequent frame, these global motion vectors GMV1 and GMV3 are updated, but the global motion vector GMV2 to which the index 2 is assigned remains valid. It becomes. Alternatively, the reference table may be reset for each frame, and only the frame header 204 having an index and global motion vector pair may be used as the reference table for the frame.

  The slice data 208 stores data for each macroblock. The GMV use flag 270 indicates whether or not the corresponding macroblock has been subjected to predictive encoding using the global motion vector GMV. When the GMV use flag 270 is “1”, it indicates that the global motion vector is used. Therefore, following this flag, an index 272 indicating which of the plurality of global motion vectors in the reference table is used is displayed. Stored. When the GMV use flag 270 is “0”, it indicates that the local motion vector is used without using the global motion vector, so the index is not stored. The slice data 208 includes information on difference images and local motion vectors LMV.

  FIG. 7 shows an example of a method for assigning an index to a global motion vector. Assume that indexes 1 to 3 are assigned to the global motion vectors GMV1 to GMV3, respectively, in a certain frame. The update determination unit 50 obtains information on the number of blocks in which the global motion vectors GMV1 to GMV3 are used when performing motion compensation for each macroblock (“frequency” in FIG. 7A) from the motion compensation prediction unit 70. The global motion vector temporary storage unit 52 is stored together with the reference table (see FIG. 7A). When receiving the global motion vector of the subsequent frame, the update determination unit 50 compares the global motion vectors GMV1 to GMV3 held in the global motion vector temporary storage unit 52 with each other. When these are the same value, the update determination unit 50 refers to the frequency with which the global motion vectors GMV1 to GMV3 are used, and reassigns the indexes 1 to 3 in ascending order (see FIG. 7B). As a result, since an index that can be expressed with a small number of pits is assigned to a global motion vector that is used more frequently, the code amount of the index can be reduced.

Example 2
In the first embodiment, a reference table in which a global motion vector and an index are associated with each other in frame units is created. In the second embodiment, a reference table in which a global motion vector and an index are associated with each slice is created. Hereinafter, Example 2 will be described with reference to FIGS.

  FIG. 8 is a flowchart for explaining a procedure of differential encoding of a motion vector by the motion compensation unit 60 according to the second embodiment. S30 to S36 are the same as S10 to S16 in FIG.

  The update determination unit 50 assigns an index to each global motion vector for each slice, and determines whether to update the global motion vector reference table. The global motion vector encoding unit 74 performs variable length encoding by associating the global motion vector GMV with the index (S38). As will be described later, this data is stored in the slice header.

  The motion compensation prediction unit 70 performs motion compensation for each macroblock included in the slice using either the global motion vector GMV to which the index is assigned in S38 or the local motion vector LMV calculated for each macroblock. Then, it is determined whether or not the code amount of the macroblock combining the code amount of the difference image and the code amount of the motion vector is minimized (S40). The motion compensation prediction unit 70 transmits a motion vector that minimizes the code amount of the macroblock to the local motion vector encoding unit 72.

  The local motion vector encoding unit 72 encodes the local motion vector when it is determined that the code amount of the macroblock is minimized when the local motion vector is used in S40 (S42). Furthermore, a GMV use flag indicating whether or not predictive coding has been executed using the global motion vector is given to the multiplexing unit.

  When the processing of S38 to S42 is completed for all the slices in the frame (Y of S44), this flow is finished.

  FIG. 9A illustrates an example of the global area in the second embodiment. The first to third global areas 112 to 116 are set, and the global motion vectors GMV1 to GMV3 are calculated for each area, as in FIG. In FIG. 9A, it is assumed that one horizontal row of slices is set in the frame 110.

  In the second embodiment, a global motion vector reference table is created for each slice. For example, since slice 3 in the figure includes the first global region 112 and the second global region 114, the update determination unit 50 assigns indexes 1, 2 to the first global motion vector GMV1 and the second global motion vector GMV2. To create a reference table (see FIG. 9B). Information of the reference table, that is, the global motion vector GMV and the index are stored in the slice header of the encoded stream.

  When the motion compensation prediction unit 70 performs predictive coding of each macroblock included in the slice 3, the motion compensation prediction unit 70 calculates the local motion vector LMV calculated for each macroblock, and the reference table of the slice 3 (FIG. 9B). It is determined which one of the global motion vectors GMV1 and GMV2 set to is used.

  Since slice 6 in the figure includes first global region 112, second global region 114, and third global region 116, update determination unit 50 assigns indexes 1 to 3 to global motion vectors GMV1 to GMV3, respectively. Thus, a reference table is created (see FIG. 9C). When the motion compensation prediction unit 70 performs predictive encoding of each macroblock included in the slice 6, the motion compensation prediction unit 70 calculates the local motion vector LMV calculated for each macroblock and the reference table of the slice 6 (FIG. 9 (c)). It is determined which of the global motion vectors GMV1 to GMV3 set to is used.

  FIG. 10 is a diagram for explaining the data structure of the encoded stream 220 generated by the multiplexing unit 92.

  In this embodiment, reference table information, that is, global motion vector and index information is stored in the slice header 206, and a GMV use flag 270 is stored for each macroblock in the slice data 208.

  The table update flag 258 is a flag indicating whether or not to update the global motion vector reference table in the slice. If the table update flag 258 is “0”, the reference table is not updated, and the reference table set in the slice before the slice continues to be valid. If the table update flag 258 is “1”, the reference table is updated. In this case, “Num” 260 represents the number of global motion vectors stored in the slice header 206. A pair of index 262 and global motion vector (GMV) 264 in the lower figure represented by Num 260 follows.

  The slice data 208 stores the difference image and local motion vector data corresponding to the GMV use flag 270 for each macroblock. The data structure of the slice data 208 is the same as that shown in FIG.

Example 3
In the third embodiment, as in the first embodiment, a reference table in which global motion vectors and indexes are associated with each other is created in units of frames. However, in the third embodiment, any one of the motion vectors in the reference table is designated for each slice, and it is determined whether or not to use the motion vector designated by the slice in each macroblock.

  This will be described with reference to FIGS. The update determination unit 50 creates a reference table as shown in FIG. The motion compensation prediction unit 70 designates any global motion vector from the reference table in FIG. 5B using an index for each slice. For example, referring to FIG. 9 (a), in slice 3, the number of blocks in which the second global region 114 overlaps the slice of the first global region 112 and the second global region 114 is larger, so that motion compensation is performed. The prediction unit 70 designates the second global motion vector GMV2. When predictive coding of each macroblock included in slice 3 is performed, either the local motion vector LMV calculated for each macroblock or the second global motion vector GMV2 previously specified in units of slices is used. To determine.

  FIG. 11 is a flowchart illustrating the procedure of differential encoding of a motion vector by the motion compensation unit 60 according to the third embodiment. S50 to S56 are the same as S10 to S16 in FIG.

  The update determination unit 50 assigns an index to each global motion vector GMV and determines whether to update the global motion vector reference table. The global motion vector encoding unit 74 performs variable length encoding by associating the global motion vector GMV with the index (S58). This data is stored in the header of the frame.

  The motion compensation prediction unit 70 designates one of the global motion vectors GMV in the reference table for each slice (S60).

  If the motion compensation prediction unit 70 performs motion compensation for each macroblock included in the slice using either the global motion vector GMV specified in S60 or the local motion vector LMV calculated for each macroblock, the difference is obtained. It is determined whether or not the code amount of the macroblock that is the sum of the code amount of the image and the code amount of the motion vector is minimized (S62). The motion compensation prediction unit 70 transmits a motion vector that minimizes the code amount of the macroblock to the local motion vector encoding unit 72.

  The local motion vector encoding unit 72 encodes the local motion vector when it is determined that the code amount of the macroblock is minimized when the local motion vector is used in S62 (S64). Furthermore, a GMV use flag indicating whether or not predictive coding has been executed using the global motion vector is given to the multiplexing unit.

  When the processing of S60 to S64 is completed for all the slices in the frame (Y of S66), this flow is finished.

  FIG. 12 is a diagram for explaining the data structure of the encoded stream 240 generated by the multiplexing unit 92.

  In this embodiment, reference table information, that is, global motion vector GMV and index information are stored in the frame header 204. The slice header 206 stores index information that specifies any of the global motion vectors GMV in the frame header. Further, a GMV use flag 284 is stored for each macro block in the slice data 208.

  The data structure of the frame header 204 is the same as that described with reference to FIG. The slice header 206 stores a GMV use flag 280 indicating whether or not the global motion vector is used in the slice. If the GMV usage flag 280 is “1”, an index 282 that specifies the global motion vector to be used follows. If the GMV use flag 280 is “0”, the global motion vector is not used in each macroblock included in the slice, and only the local motion vector is used. Therefore, the GMV use flag 284 described later does not exist.

  The GMV use flag 284 indicates whether or not to use the global motion vector specified by the index 282 in each macro block. When the GMV use flag 284 is “1”, the macro motion is compensated using the global motion vector, and when the GMV use flag 284 is “0”, the macro is used using the local motion vector. Motion compensation for the block.

  FIG. 13 is a configuration diagram of the decoding device 300 according to the embodiment. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding apparatus 300 receives an input of the encoded stream, decodes the encoded stream, and generates an output image. The input encoded stream is stored in the frame memory 380.

  The variable length decoding unit 310 performs variable length decoding on the encoded stream stored in the frame memory 380, supplies the decoded image data to the inverse quantization unit 320, and supplies the decoded motion vector information to the motion compensation unit 360. Supply.

  The inverse quantization unit 320 inversely quantizes the image data decoded by the variable length decoding unit 310 and supplies the image data to the inverse DCT unit 330. The image data inversely quantized by the inverse quantization unit 320 is a DCT coefficient. The inverse DCT unit 330 restores the original image data by performing inverse discrete cosine transform (IDCT) on the DCT coefficients inversely quantized by the inverse quantization unit 320. The image data restored by the inverse DCT unit 330 is supplied to the motion compensation unit 360.

  The motion compensation unit 360 uses a past or future image frame as a reference image, generates a prediction image using the motion vector information supplied from the variable length decoding unit 310, and a difference image supplied from the inverse DCT unit 330. Is added to restore and output the original image data.

  FIG. 14 is a diagram illustrating the configuration of the motion compensation unit 360. The encoded stream input to the decoding device 300 is encoded by the encoding device 100 of FIG. 1, and the global motion vector GMV, the index, various flags, and the local motion vector LMV are stored in the motion compensation unit 360. Supplied. The motion compensation unit 360 refers to these pieces of information to obtain the local motion vector LMV of the decoding target frame and performs motion compensation.

  The table creation unit 364 receives the input of the global motion vector and the index from the variable length decoding unit 310, and creates a global motion vector reference table.

  For example, when the global motion vector reference table information is encoded for each frame, as shown in FIG. 6 or FIG. 12, the table creation unit 364 recognizes the number of global motion vectors from the information of the Num 210. Then, a table having the number of entries is created, and the index 212 and the corresponding global motion vector 214 are stored therein, thereby creating a reference table as shown in FIG.

When global motion vector reference table information is encoded for each slice as shown in FIG. 10, first, the table update flag 258 is referenced to update the global motion vector reference table in the slice to be processed. It is determined whether or not to do. When updating the reference table, as described above, the number of global motion vectors is grasped from the information of the Num 260, a table having entries corresponding to the number is created, and the index 262 and the corresponding global motion vector 264 are created there. Are stored, a reference table as shown in FIGS. 9B and 9C is created.
The table creation unit 364 gives the created reference table to the image restoration unit 366.

  The image restoration unit 366 generates a prediction image using the reference image and the local motion vector LMV for each macroblock in each global region or the global motion vector GMV in the reference table, and the difference image given from the inverse DCT unit 330 And the predicted image are added to restore the original image and output it.

  When the encoded stream as shown in FIG. 6 or FIG. 10 is input, the image restoration unit 366 refers to the GMV use flag 270 set corresponding to each macroblock, and determines the macroblock to be processed. It is determined which of the global motion vector and the local motion vector is used to generate the predicted image. When a global motion vector is used, a global motion vector associated with the reference table received from the table creation unit 364 is acquired using the index 272 following the GMV use flag 270, and the acquired global motion vector is used. To generate a predicted image. When the flag 270 is “0”, the image restoration unit 366 generates a predicted image using the local motion vector for each macroblock to be processed. Using the encoded stream of FIG. 6 as an example, the image restoration unit 366 generates a prediction image using the global motion vector specified by the index 272 for the first macroblock, and generates the second macroblock. For, a predicted image is generated using a local motion vector.

  When an encoded stream as shown in FIG. 12 is input, it is determined whether a global motion vector is designated with reference to the GMV use flag 280 for each slice. When the GMV use flag 280 is “1” and a global motion vector is designated, the global motion vector associated with the reference table received from the table creation unit 364 is acquired using the index 282. Then, referring to the GMV use flag 284 for each macroblock, a predicted image is generated using either the global motion vector acquired earlier or the local motion vector for each macroblock. Using the encoded stream of FIG. 12 as an example, a predicted image is generated using the global motion vector specified by the index 282 for the first macroblock, and a local motion vector is used for the second macroblock. To generate a predicted image.

  As described above, according to coding apparatus 100 of the present embodiment, a table in which global motion vectors and indexes are associated with each other in units of frames or for each slice is prepared, and predictive coding is performed in each macroblock. The global motion vector to be used for can be specified by index. This eliminates the need for redundant global motion vectors to be included in the encoded stream when performing predictive encoding using the same global motion vector in macroblocks included in different global regions within the same frame or slice. Therefore, the data amount of the encoded data resulting from the global motion vector can be reduced, and the compression efficiency of the moving image stream can be increased.

  In addition, by storing a GMV use flag indicating whether or not to use a global motion vector for each macroblock in an encoded stream, motion compensation can be performed using either a local motion vector or a global motion vector for each macroblock. And the amount of encoded data of the macroblock may be further reduced.

  In addition, when multiple global motion vectors are set in the reference table in frame units or slice units, and any global motion vector can be used by using an index, macroblocks are not included in the same global area However, motion compensation using the same global motion vector can be performed.

  The global motion vector encoded with the index may be difference data from the global motion vector in the reference table in the frame or slice preceding the process. In this way, the data amount of the motion vector information itself can be reduced, the code amount of the entire moving image stream can be reduced, and the compression efficiency can be increased.

  The index assigned to the global motion vector may be assigned a number in the order in which the global motion vector is calculated, but is smaller than the global motion vector that is predicted to occur frequently in a frame or slice. If the index is assigned, the data amount of the index can be further reduced.

  Also, according to the decoding apparatus 300 of the present embodiment, a reference table in which a global motion vector and an index are associated with each other from a video stream with high compression efficiency encoded by the encoding apparatus 100 for each frame or for each slice. Build a motion vector from the lookup table using an index for each slice or macroblock. In this way, motion compensation using either a local motion vector or a global motion vector can be executed.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above description, if the motion vector selection unit 54 performs motion prediction using either one or a plurality of global motion vectors GMV and the local motion vector LMV calculated for each macroblock, It has been described that it is determined whether or not the code amount of the macroblock combined with the code amount of the motion vector is minimized. However, the motion vector selection unit 54 may select an optimal motion vector as follows. That is, the motion vector selection unit 54 calculates the code amount of the macroblock and decodes an image encoded using one or a plurality of global motion vectors GMV and local motion vectors LMV of each macroblock. The difference value between the decoded image and the original image at the time of conversion is calculated. Then, the motion vector selection unit 54 obtains an evaluation value calculated by substituting the code amount of the macroblock and the difference value into a predetermined evaluation formula for each motion vector, and a motion vector that minimizes the evaluation value Select. By doing so, it is possible to select an optimal motion vector in consideration of both the code amount of the encoded data and the image quality of the decoded image.

  In the above description, a reference table for specifying a motion vector using an index is provided for each frame, for each region, and for each macroblock. However, the reference table is provided for a larger unit, for example, the entire stream or GOP. You can also In this case, the information of the reference table is stored in the stream parameter or GOP header. Then, for each frame, slice, global region, or macroblock, index information that specifies a motion vector set in the reference table is stored in a header or the like. In this way, for example, when the motion vector of the moving image does not change over the entire stream or GOP, the amount of codes resulting from the motion vector information can be greatly reduced.

  In the above description, a table for specifying a motion vector with an index for each global region or slice in a frame is created. However, a region of interest (Region of Interest) set on a moving image by a ROI region setting unit (not shown). ; For each ROI), a table for specifying a motion vector by an index may be created. The attention area may be selected by the user specifying a specific area on the image, or a predetermined area such as a center area of the image may be selected. In addition, an important area such as an area in which a person or character is shown may be automatically extracted as the attention area. Further, the attention area may be automatically selected in units of frames by tracking the movement of a specific object or the like in the moving image.

  In the above description, the encoding device 100 and the decoding device 300 are MPEG series standards (MPEG-1, MPEG-2, and MPEG-4), H.264, and H.264. 26x series standards (H.261, H.262 and H.263), or H.264 Although video encoding and decoding are performed in accordance with H.264 / AVC, the present invention can also be applied to the case of hierarchical video encoding and decoding with temporal scalability. In particular, the present invention is effective in reducing the amount of motion vector codes in the encoding of motion vectors when the MCTF technique is used.

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure explaining the structure of the motion compensation part of FIG. It is a figure which shows an example of the reference table of a global motion vector. 6 is a flowchart illustrating a procedure for differential encoding of a motion vector according to the first embodiment. (A) is a figure explaining the example of a global area | region, (b) is a figure which shows an example of a reference table. It is a figure explaining the data structure of the encoding stream which concerns on Example 1. FIG. It is a figure which shows an example of the method of providing an index to a global motion vector. 10 is a flowchart illustrating a procedure for differential encoding of a motion vector according to the second embodiment. (A) is a figure explaining the example of a global area | region, (b), (c) is a figure which shows an example of a reference table. It is a figure explaining the data structure of the encoding stream which concerns on Example 2. FIG. 10 is a flowchart illustrating a procedure for differential encoding of a motion vector according to a third embodiment. It is a figure explaining the data structure of the encoding stream which concerns on Example 3. FIG. It is a block diagram of the decoding apparatus concerning embodiment. It is a figure explaining the structure of the motion compensation part of FIG.

Explanation of symbols

  20 DCT section, 30 quantization section, 50 update determination section, 52 global motion vector temporary storage section, 60 motion compensation section, 64 region setting section, 66 local motion vector detection section, 68 global motion vector calculation section, 70 motion compensation prediction 72, local motion vector encoding unit, 74 global motion vector encoding unit, 80 frame memory, 90 variable length encoding unit, 92 multiplexing unit, 100 encoding device, 300 decoding device, 310 variable length decoding unit, 320 Inverse quantization unit, 330 inverse DCT unit, 360 motion compensation unit, 364 table creation unit, 366 image restoration unit, 380 frame memory.

Claims (6)

  In a picture constituting a moving picture, information indicating a global motion vector indicating global motion in a picture that is inter-picture predictively encoded or in an area defined in the picture, and for specifying the global motion vector And the index information indicating the global motion vector used for predictive encoding in the picture in which the global motion vector is calculated or in the area included in the picture area. An encoding method comprising the encoded data of the moving image.   When the first global motion vector calculated in the first picture or in the first intra-picture area is different from the second global motion vector calculated in the second picture or in the second intra-picture area, The encoding method according to claim 1, wherein difference information between the first global motion vector and the second global motion vector is included in the encoded data in association with the index.   The global motion vectors in the second picture or the second picture area are rearranged in the order of the appearance frequency of the global motion vector in the first picture or the first picture area, and this order is made to correspond to this order. The encoding method according to claim 1, wherein the index is assigned.   A picture in which a global motion vector is calculated, including information indicating the global motion vector and information on the index for specifying the global motion vector in a header of encoded data, a header of a picture, or a header of a region in a picture The code according to any one of claims 1 to 3, wherein the header of an area included in an intra-picture area includes information on the index indicating the global motion vector used for predictive encoding in the area. Method.   5. The encoded data includes a flag indicating whether or not predictive encoding is performed using the global motion vector for each picture or a region defined in the picture. An encoding method according to claim 1.   For each block in the picture, a flag indicating which of the global motion vector indicated by the index and the local motion vector calculated for the block is used for predictive encoding is included in the encoded data. The encoding method according to claim 1, wherein the encoding method is any one of claims 1 to 4.

Images