JP2007036889A - Coding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a coding technique of motion images which has high coding efficiency and can perform high-accuracy motion prediction. <P>SOLUTION: A region setting section 64 sets a plurality of global regions on a frame image, and a bit number adjuster 62 adjusts the number of bits of local motion vectors LMV to be obtained in a global region. A local motion vector detector 66 obtains a local motion vector LMV by macroblock units at the number of bits adjusted by the bit number adjuster 62 in each global region. A large region motion vector calculator 68 calculates a global motion vector GMV indicating large region motion in each global region. A local motion vector difference coder 72 obtains and codes a difference ΔLMV between the local motion vector LMV and the global motion vector GMV in the global region for each global region. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding a moving image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. There is a next-generation image compression technology such as SVC that has both a high-quality stream and a low-quality stream in one stream.

  When streaming a high-resolution moving image or storing it in a recording medium, it is necessary to increase the compression rate of the moving image stream so as not to compress the communication band or increase the storage capacity. In order to enhance the compression effect of moving images, motion compensation interframe predictive coding is performed. In motion-compensated interframe predictive coding, the encoding target frame is divided into blocks, the motion from a reference frame that has already been encoded is predicted for each block, a motion vector is detected, and motion vector information is encoded along with the difference image. Turn into.

Patent Document 1 discloses a moving image coding technique that selects a method with the highest coding efficiency from among various types of motion compensation using intra-frame coding, normal motion compensation, and global motion vectors. .
JP 2003-299101 A

  H. In the H.264 / AVC standard, in order to perform more detailed prediction in motion compensation, the block size of motion compensation can be made variable, and the pixel accuracy of motion compensation can be reduced to ¼ pixel accuracy. Therefore, the amount of code related to the motion vector increases. In addition, in SVC (Scalable Video Coding), which is a next-generation image compression technology, MCTF (Motion Compensated Temporal Filtering) technology is being studied in order to improve temporal scalability. This is a combination of subband division in the time axis direction and motion compensation. Since hierarchical motion compensation is performed, information on motion vectors becomes very large. As described above, the recent video compression coding technology tends to increase the data amount of the entire video stream due to an increase in the amount of information related to motion vectors, and there is a further demand for a technology for reducing the amount of codes resulting from motion vector information. ing.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a moving picture coding technique that has high coding efficiency and can perform highly accurate motion prediction.

  In order to solve the above-described problem, an encoding method according to an aspect of the present invention is a picture constituting a moving image, wherein a plurality of areas are defined on a picture to be subjected to inter-picture prediction encoding, and motion is performed for each area. Sets the conditions for vector encoding.

  A “picture” is an encoding unit including a frame, a field, a VOP (Video Object Plane), and the like.

  According to this aspect, it is possible to encode a moving image with different motion vector encoding conditions for each region.

  The motion vector encoding condition may be a condition relating to pixel accuracy of motion compensation, a condition relating to a maximum value that the motion vector can take, or a combination of these conditions. Also good. According to this, it is possible to encode a moving image by changing at least one of the pixel accuracy of motion compensation and the maximum value that can be taken by the motion vector for each region. Also, by making these encoding conditions variable for each region, it is possible to generate optimized encoded data of a moving image.

  The motion vector encoding condition may be included in the encoded data of the moving image in association with each region in which the condition is set. Thus, when decoding the encoded moving image, the decoding process can be performed with reference to various conditions when each region is encoded.

  A motion vector is determined in each of the plurality of regions after adjusting at least one of the pixel accuracy of motion compensation and the maximum value that the motion vector can take for each region, and the obtained motion vector is encoded. It may be included in the encoded data.

  By changing the pixel accuracy of motion compensation for each region, the number of bits allocated to the motion vector to be obtained in the region may be adjusted. According to this, when the pixel accuracy of motion compensation required for each region is different, the number of bits of the motion vector can be adjusted according to the required pixel accuracy, and the code amount of the motion vector can be reduced. Can do.

  By changing the maximum value that the motion vector can take for each region, the number of bits allocated to the motion vector to be obtained in the region may be adjusted. The maximum value that the motion vector can take may be changed according to the width of the motion search range for each region. According to this, when there is a difference in the magnitude of motion for each region, the number of bits allocated to the motion vector can be adjusted according to the magnitude of the motion, and the amount of motion vector code can be reduced. .

  Another aspect of the present invention is an encoding device. The apparatus includes a region setting unit that sets a plurality of regions on a picture that is inter-picture predictively encoded in a moving image, and at least one of the maximum value that can be taken by the pixel accuracy of motion compensation and a motion vector for each region. An adjustment unit that adjusts a motion vector, a motion vector detection unit that obtains a motion vector in each of the plurality of regions based on an adjustment result by the adjustment unit, and a motion vector encoding unit that encodes the obtained motion vector Including.

  Yet another aspect of the present invention is a data structure of a moving image stream. The data structure of the moving picture stream is a data structure of a moving picture stream in which a picture of a moving picture is encoded, and a motion compensation pixel for each of a plurality of areas defined on a picture to be subjected to inter-picture prediction encoding. The motion vector obtained by the region unit after the accuracy and at least one of the maximum values that the motion vector can take is adjusted, and is encoded together with the picture to be inter-predictively encoded.

  According to this aspect, since the motion vector is obtained and encoded for each region after adjusting the pixel accuracy of motion compensation and / or the maximum value that the motion vector can take for each region, the motion vector Can provide an optimized video stream.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the encoding efficiency of moving images.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 100 according to the present embodiment includes an MPEG (Moving Picture Experts Group) series standard (MPEG-1, MPEG, standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission)). -2 and MPEG-4), standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector), which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264 respectively).

  In the MPEG series standard, an image frame for intra-frame encoding is an I (Intra) frame, an image frame for forward inter-frame predictive encoding with a past frame as a reference image, a P (Predictive) frame, and past and future An image frame that performs bidirectional inter-frame predictive coding using this frame as a reference image is referred to as a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference image may be a past two frames as a reference image or a future two frames as a reference image regardless of the time. Further, three or more frames can be used as the reference image regardless of the number of frames that can be used as the reference image. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame because the time of the reference image does not matter before and after.

  In the embodiment, a frame is used as an example of the encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, encodes the moving image, and outputs an encoded stream. The input moving image frame is stored in the frame memory 80.

  The motion compensation unit 60 uses a past or future image frame stored in the frame memory 80 as a reference image, performs motion compensation for each macroblock of the P frame or the B frame, and generates a motion vector and a predicted image. . The motion compensation unit 60 takes the difference between the P frame or B frame image to be encoded and the predicted image, and supplies the difference image to the DCT unit 20. In addition, the motion compensation unit 60 supplies the encoded motion vector information to the multiplexing unit 92.

  The DCT unit 20 performs discrete cosine transform (DCT) on the image supplied from the motion compensation unit 60, and gives the obtained DCT coefficient to the quantization unit 30.

  The quantization unit 30 quantizes the DCT coefficient and provides it to the variable length coding unit 90. The variable length coding unit 90 performs variable length coding on the quantized DCT coefficient of the difference image and supplies the result to the multiplexing unit 92. The multiplexing unit 92 multiplexes the encoded DCT coefficient given from the variable length coding unit 90 and the coded motion vector information given from the motion compensation unit 60 to generate an encoded stream. . When generating the encoded stream, the multiplexing unit 92 performs a process of rearranging the encoded frames in time order.

  In the case of P frame or B frame encoding processing, the motion compensation unit 60 operates as described above. However, in the case of I frame encoding processing, the motion compensation unit 60 does not operate and is not shown here. The I frame is supplied to the DCT unit 20 after intra prediction.

  FIG. 2 is a diagram illustrating the configuration of the motion compensation unit 60. The motion compensation unit 60 detects a motion vector (hereinafter referred to as a “local motion vector”) of a macroblock unit of an encoding target frame, and for each predetermined region provided on the image, A motion vector indicating a smooth motion (hereinafter referred to as “global motion vector”) is obtained. The motion compensation unit 60 performs motion prediction based on the local motion vector, outputs a difference image, encodes a difference value between the local motion vector and the global motion vector, and outputs it as motion vector information.

  The region setting unit 64 sets a region (hereinafter referred to as “global region”) for obtaining the global motion vector GMV on the frame image. A plurality of global areas are provided in the image. For example, the region setting unit 64 may set a predetermined region as the global region, such as setting the vicinity of the center of the frame image as one global region and setting the periphery thereof as another global region. The global area may be set by the user.

  In addition, when an object such as a person appears in the image, the area setting unit 64 may automatically extract an area of an arbitrary shape occupied by the object and set it as a global area.

  The region setting unit 64 refers to the local motion vector LMV in the image detected by the local motion vector detection unit 66, automatically extracts a region occupied by a macroblock having a certain amount of motion, and sets the region as a global region. May be.

  The region setting unit 64 gives information on the set global region to the bit number adjustment unit 62, the global motion vector calculation unit 68, and the global motion vector difference encoding unit 74.

  The bit number adjustment unit 62 determines the number of bits of the local motion vector LMV to be obtained in each global region by determining the size of the motion search range and the pixel accuracy of motion compensation for each global region set by the region setting unit 64. adjust.

  For example, the bit number adjusting unit 62 sets the pixel accuracy of motion compensation for each global region to an integer pixel unit, or a decimal pixel unit such as a 1/2 pixel unit or a 1/4 pixel unit, so that the local motion vector LMV is set. Adjust the number of bits. When the pixel accuracy of motion compensation is an integer pixel unit, the local motion vector LMV is represented by only the bits of the integer part. However, when the pixel accuracy is 1/2 pixel unit or 1/4 pixel unit, the local motion vector LMV is A fractional part is required in addition to the integer part. One bit is required for a 1/2 pixel unit, and an extra bit is required for the fractional part by 2 bits for a 1/4 pixel unit.

  Further, the bit number adjusting unit 62 can also adjust the bit number of the local motion vector LMV by changing the maximum value that the local motion vector LMV can take for each global region. The bit number adjusting unit 62 adjusts the maximum value that can be taken by changing the number of digits of the integer part of the local motion vector LMV in accordance with the width of the motion search range of each global region and the magnitude of the motion in each global region.

  The local motion vector detection unit 66 refers to the reference image held in the frame memory 80, detects a prediction macroblock having the smallest error with respect to the target macroblock of the encoding target image from the reference image, and determines from the target macroblock. A local motion vector LMV indicating the motion to the predicted macroblock is obtained. Motion detection is performed by searching for a reference macroblock in a reference image that matches the target macroblock with a motion search range and pixel accuracy set by the bit number adjustment unit 62. The search is normally repeated a plurality of times within the pixel region, and the reference macroblock that best matches the target macroblock is selected as the predicted macroblock in the plurality of searches.

  The local motion vector detection unit 66 converts the local motion vector LMV obtained by the bit number adjusted by the bit number adjustment unit 62 into a global motion vector calculation unit 68, a motion compensation prediction unit 70, and a local motion vector difference encoding unit 72. To give.

  The motion compensation prediction unit 70 performs motion compensation on the target macroblock using the local motion vector LMV, generates a prediction image, and outputs a difference image between the encoding target image and the prediction image to the DCT unit 20.

  The global motion vector calculation unit 68 calculates a global motion vector GMV indicating a global motion in each global region set by the region setting unit 64. For example, the global motion vector calculation unit 68 obtains the average of local motion vectors LMV in the region and sets it as the global motion vector GMV. Here, the number of bits of the global motion vector GMV in each global region is the same as the number of bits of the local motion vector LMV obtained in each global region, and is the number of bits adjusted by the bit number adjustment unit 62.

  Further, the global motion vector calculation unit 68 may acquire information on global motion in each global region and calculate a global motion vector GMV for each global region based on the information. For example, when the camera is zoomed or panned, the screen is scrolled, etc., the global motion in each global region can be determined from the information regarding the motion of the entire screen, and the global motion vector GMV can be calculated. . The global motion vector calculation unit 68 may automatically extract the motion of an object such as a person on the screen, determine a global motion in each global region from the motion of the object, and calculate a global motion vector GMV. it can.

  The global motion vector calculation unit 68 gives the global motion vector GMV obtained by the bit number adjusted by the bit number adjustment unit 62 to the local motion vector difference encoding unit 72 and the global motion vector difference encoding unit 74.

  The local motion vector differential encoding unit 72 receives the input of the local motion vector LMV from the local motion vector detection unit 66 and the input of the global motion vector GMV from the global motion vector calculation unit 68, respectively, and local motion in the global region for each global region. A difference ΔLMV = LMV−GMV between the vector LMV and the global motion vector GMV is calculated, and the local motion vector difference ΔLMV is variable-length encoded. The local motion vector difference encoding unit 72 supplies the encoded local motion vector difference ΔLMV to the multiplexing unit 92 as motion vector information.

The global motion vector differential encoding unit 74 receives an input of the global motion vector GMV of each region from the global motion vector calculation unit 68 and selects it based on the global motion vector GMV of at least one region. The reference global motion vector GMV is referred to as a reference global motion vector GMV _B. Global motion vector difference coding unit 74 calculates a difference ΔGMV = GMV-GMV _B of the global motion vector GMV and the reference global motion vector GMV _B reference global motion each global region other than the vector GMV _B, the reference global motion vector GMV Variable length coding is performed on _B and the global motion vector difference ΔGMV.

The global motion vector difference encoding unit 74 gives the encoded reference global motion vector GMV _B and the global motion vector difference ΔGMV of each global region to the multiplexing unit 92 as motion vector information. At this time, the global motion vector difference encoding unit 74 adds the region information about the global region set by the region setting unit 64 to a part of the motion vector information. Furthermore, the global motion vector difference encoding unit 74 determines the size of the motion search range for each global region determined by the bit number adjustment unit 62, the pixel accuracy for motion compensation, and the maximum value that can be taken by the local motion vector LMV for each global region. The motion compensation parameter information such as is added to a part of the motion vector information. These various parameters for motion compensation are referred to when performing motion compensation in the decoding apparatus 300.

The multiplexing unit 92 is provided with the reference global motion vector GMV _B , the global motion vector difference ΔGMV, and the local motion vector difference ΔLMV as motion vector information.

  FIG. 3 is a flowchart for explaining the procedure of motion vector differential encoding by the motion compensation unit 60. The encoding procedure will be described with reference to the examples of FIGS.

  An encoding target image is input to the frame memory 80 of the encoding apparatus 100 (S10). The area setting unit 64 sets a global area on the image (S12). The bit number adjustment unit 62 adjusts the number of bits of the local motion vector LMV for each global region (S13).

  The local motion vector detection unit 66 of the motion compensation unit 60 detects the local motion vector LMV in units of macroblocks with the adjusted number of bits for each global region in the encoding target image (S14).

  Next, the global motion vector calculation unit 68 calculates a global motion vector GMV for each global region (S16).

  The local motion vector difference encoding unit 72 calculates and encodes the local motion vector difference ΔLMV for each global region (S18). The global motion vector difference encoding unit 74 calculates and encodes the global motion vector difference ΔGMV for each global region (S20).

  4A to 4C are diagrams illustrating an example of the global area. In the example of FIG. 4A, the region setting unit 64 sets the first global region 211 and the second global region 212 on the encoding target image 200. The global motion vector calculation unit 68 obtains the first global motion vector GMV1 in the first global area 211 and obtains the second global motion vector GMV2 in the second global area 212. In this example, no area for obtaining a global motion vector is set in the background portion other than the first global area 211 and the second global area 212.

  In the case of FIG. 4A, when the local motion vector difference encoding unit 72 encodes the local motion vector LMV for each macroblock in the first global region 211, the difference ΔLMV from the first global motion vector GMV1 = LMV-GMV1 is obtained and encoded. Similarly, the local motion vector difference encoding unit 72 obtains and encodes the difference ΔLMV = LMV−GMV2 between the local motion vector LMV and the second global motion vector GMV2 for each macroblock in the second global region 212.

  In the example of FIG. 4A, since the global motion vector GMV is not obtained for the background regions other than the first global region 211 and the second global region 212, the local motion vector difference encoding unit 72 The local motion vector LMV is encoded as it is without obtaining the difference.

  In the example of FIG. 4B, unlike the example of FIG. 4A, the region setting unit 64 sets the background portion other than the first global region 211 and the second global region 212 as the third global region 210. Then, the global motion vector calculation unit 68 obtains the third global motion vector GMV0 in the third global region 210. The local motion vector difference encoding unit 72 obtains and encodes the difference ΔLMV = LMV−GMV0 between the local motion vector LMV and the third global motion vector GMV0 for each macroblock in the third global region 210.

  FIG. 4C shows an example in which a plurality of global regions have an inclusion relationship with each other on the encoding target image 200. In this example, the second global area 212 is included in the first global area 211, and the first global area 211 and the second global area 212 are entirely included in the third global area 210.

  The local motion vector difference encoding unit 72 encodes the difference between the local motion vector LMV and the second global motion vector GMV2 for each macro block inside the second global region 212, and the second outside the second global region 212. Inside the one global area 211, the difference between the local motion vector LMV and the first global motion vector GMV1 is encoded for each macroblock. Further, in the third global area 210 further outside the first global area 211, the difference between the local motion vector LMV and the third global motion vector GMV0 is encoded for each macroblock.

  FIGS. 5A to 5C are diagrams illustrating an example in which a global motion vector difference is calculated by the global motion vector difference encoding unit 74. FIG. Here, as shown in FIG. 4 (b) or FIG. 4 (c), when three global areas are set and global motion vectors GMV0, GMV1, and GMV2 of the respective global areas are obtained, these three global movements are obtained. An example in which the vectors GMV0, GMV1, and GMV2 are encoded will be described.

  FIG. 5A shows a case where no hierarchical structure is provided between the three global motion vectors GMV0, GMV1, and GMV2. In this case, the global motion vector difference encoding unit 74 treats all three global motion vectors GMV0, GMV1, and GMV2 as reference global motion vectors, and obtains 9-bit global motion vectors GMV0, GMV1 and GMV2 are encoded as they are and output.

  FIG. 5B shows a case where a hierarchical structure is provided between the three global motion vectors GMV0, GMV1, and GMV2, where GMV0 is located in the upper hierarchy, and GMV1 and GMV2 are located in the hierarchy immediately below GMV0. . At this time, the global motion vector differential encoding unit 74 uses the global motion vector GMV0 in the upper layer as the reference global motion vector, and the difference ΔGMV1 = GMV1 between the global motion vectors GMV1 and GMV2 in the lower layer and the reference global motion vector GMV0. -Encode GMV0, [Delta] GMV2 = GMV2-GMV0. Thereby, the code amount of each of the global motion vectors GMV1 and GMV2 in the lower layer, which has 9 bits, is reduced to 3 bits and 4 bits by taking the difference from the global motion vector GMV0 in the upper layer.

  FIG. 5C shows a case where another hierarchical relationship is provided between the three global motion vectors GMV0, GMV1, and GMV2, where GMV0 is located in the highest hierarchy, GMV1 is located in the next hierarchy of GMV0, GMV2 is located in a layer below GMV1. At this time, the global motion vector differential encoding unit 74 uses the first-layer global motion vector GMV0 as a reference global motion vector, and the difference ΔGMV1 = GMV1 between the second-layer global motion vector GMV1 and the first-layer global motion vector GMV0. -Encode GMV0. The code amount of the global motion vector GMV1 in the second layer, which was 9 bits, is reduced to 3 bits by taking the difference from the global motion vector GMV0 in the first layer.

  Next, the global motion vector difference encoding unit 74 encodes the difference ΔGMV2 = GMV2−GMV1 between the global motion vector GMV2 in the third hierarchy and the global motion vector GMV1 in the second hierarchy. The code amount of the third-layer global motion vector GMV2 that is 9 bits is reduced to 2 bits by taking the difference from the global motion vector GMV1 of the second layer.

  5B and 5C, the global motion vector difference encoding unit 74 outputs the reference global motion vector GMV0 and the two global motion vector differences ΔGMV1 and ΔGMV2 as motion vector information. At this time, information indicating a hierarchical structure between the three global motion vectors GMV0, GMV1, and GMV2 is added as a part of the motion vector information.

  As shown in the examples of FIGS. 5B and 5C, a hierarchical structure is appropriately provided between global motion vectors, and the difference between adjacent layers is taken to reduce the code amount of the global motion vector. be able to. In the above example, the difference between the global motion vector at the lower level and the reference global motion vector at the higher level is encoded using the global motion vector at the higher level of the hierarchical structure as a reference. A difference between an upper global motion vector and a lower reference global motion vector may be encoded with a certain global motion vector as a reference.

  The global motion vector hierarchical structure may be determined regardless of the global region inclusion relationship, or the order of the global motion vector hierarchical structure may be determined according to the order of the global region inclusion relationship.

  For example, as shown in FIG. 4B, when the first global region 211 and the second global region 212 are included in the third global region 210, the global motion vector differential encoding unit 74 includes the global region. Using the relationship, as shown in FIG. 5B, the global motion vector GMV0 of the third global region 210 is set as the upper layer, and the global motion vectors GMV1 and GMV2 of the first global region 211 and the second global region 212 are used as the upper layer. A differential structure of the global motion vector is performed by providing a hierarchical structure immediately below.

  Further, as shown in FIG. 4C, the second global region 212 is included in the first global region 211, and the first global region 211 and the second global region 212 are included in the third global region 210 as a whole. When there is a relationship, the global motion vector differential encoding unit 74 sets the global motion vector GMV0 of the third global region 210 as the highest layer, the global motion vector GMV1 of the first global region 211 as the second layer, and the second global A hierarchical structure having the global motion vector GMV2 in the region 212 as the third hierarchy is provided, and differential encoding of the global motion vector is performed.

  In this way, if the inclusion relation of the global area set by the area setting unit 64 is used as it is in the global motion vector hierarchical structure, if information on the inclusion relation of the global area is included in a part of the motion vector information, the global area It is not necessary to separately include information on the hierarchical structure of the motion vector in the motion vector information, and the data amount of the header information can be reduced.

  In addition, if the inclusion relationship of the global area reflects the relative difference in the amount of motion of the image, such as near the center of the image and the background area, or the area of the specific object and its background, etc. It is generally expected that the number of bits when the difference is taken is reduced by reflecting the inclusion relationship as it is in the hierarchical structure of the global motion vector and obtaining the global motion vector difference according to the hierarchical structure.

  FIG. 6 is a diagram for explaining the number of bits of the local motion vector LMV adjusted by the bit number adjustment unit 62.

  As an example, the x coordinate value and the y coordinate value of the local motion vector LMV are composed of an integer part of 8 bits and a decimal part of 2 bits, and is represented by a maximum of 10 bits as a whole. The number of digits in the integer part is determined according to the maximum value that the local motion vector LMV can take. The number of digits of the decimal part is determined according to the pixel accuracy of motion compensation. In order to represent a motion vector with 1/2 pixel accuracy, 1-bit information is required after the decimal point, and in the case of 1/4 pixel accuracy, 2-bit information is required after the decimal point.

  Here, as shown in FIG. 4B or FIG. 4C, when a global area corresponding to each of the three global motion vectors GMV0, GMV1, and GMV2 is set, for each macroblock in each global area. An example of adjusting the number of bits of the local motion vector LMV required is shown in FIG.

  The local motion vectors in the first, second, and third global regions in which the first global motion vector GMV1, the second global motion vector GMV2, and the third global motion vector GMV0 are determined are the first local motion vector LMV1 and the second local motion vector, respectively. They are called a motion vector LMV2 and a third local motion vector LMV0.

As indicated by reference numeral 240, the third local motion vector LMV0 is provided with 2 bits for the decimal part and 6 bits for the integer part, and is represented by 8 bits as a whole. In this case, the pixel accuracy is ¼ pixel, and the maximum value of a positive integer that can be expressed by 6 bits is 2 ⁶ = 64. Therefore, the maximum value that each coordinate value of the motion vector can take is ± 32 pixels. It is. Therefore, it is desirable that the region where the motion search range is within ± 32 pixels and the motion should be obtained with ¼ pixel accuracy is selected as the third global region. For example, the motion is fine and high-precision motion compensation is required. An area occupied by an object such as a person is selected.

  As indicated by reference numeral 241, the first local motion vector LMV1 is provided with 1 bit for the decimal part and 6 bits for the integer part, and is represented by 7 bits as a whole. In this case, the pixel accuracy is ½ pixel, and the maximum value that each coordinate value of the motion vector can take is ± 32 pixels. Therefore, it is desirable that the region where the motion search range is within ± 32 pixels and the motion should be obtained with 1/2 pixel accuracy is selected as the first global region, for example, motion compensation with relatively little motion and very high accuracy. A background area that is not required is selected.

As indicated by reference numeral 242, the second local motion vector LMV2 is provided with 1 bit for the decimal part and 8 bits for the integer part, and is represented by 9 bits as a whole. In this case, the pixel accuracy is ½ pixel, and the maximum value of a positive integer that can be expressed in 8 bits is 2 ⁸ = 256. Therefore, the maximum value that each coordinate value of the motion vector can take is ± 128 pixels. It is. Therefore, it is desirable to select a region in which the motion search range is within ± 128 pixels and the motion should be obtained with 1/2 pixel accuracy as the second global region. A region is selected.

  When a global region is set by the region setting unit 64, the bit number adjustment unit 62 may set the size of the motion search range and the pixel accuracy for motion compensation in each global region in advance. In that case, after the number of bits of the local motion vector is determined, the local motion vector detection unit 66 detects the local motion vector in each global region.

  As another encoding procedure, after the local motion vector detection unit 66 detects the local motion vector in each global region, the bit number adjustment unit 62 determines the size of the local motion vector detected in each global region. Evaluation may determine the number of bits required to represent a local motion vector in the global region. In that case, even in the same global region, the number of bits of the local motion vector can be made variable according to the temporal change of the motion.

  FIG. 7 is a configuration diagram of the decoding device 300 according to the embodiment. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding apparatus 300 receives an input of the encoded stream, decodes the encoded stream, and generates an output image. The input encoded stream is stored in the frame memory 380.

  The variable length decoding unit 310 performs variable length decoding on the encoded stream stored in the frame memory 380, supplies the decoded image data to the inverse quantization unit 320, and supplies the decoded motion vector information to the motion compensation unit 360. Supply.

  The inverse quantization unit 320 inversely quantizes the image data decoded by the variable length decoding unit 310 and supplies the image data to the inverse DCT unit 330. The image data inversely quantized by the inverse quantization unit 320 is a DCT coefficient. The inverse DCT unit 330 restores the original image data by performing inverse discrete cosine transform (IDCT) on the DCT coefficients inversely quantized by the inverse quantization unit 320. The image data restored by the inverse DCT unit 330 is supplied to the motion compensation unit 360.

  The motion compensation unit 360 uses a past or future image frame as a reference image, generates a prediction image using the motion vector information supplied from the variable length decoding unit 310, and a difference image supplied from the inverse DCT unit 330. Is added to restore and output the original image data.

FIG. 8 is a diagram illustrating the configuration of the motion compensation unit 360. The encoded stream input to the decoding apparatus 300 is encoded by the encoding apparatus 100 of FIG. 1, and the reference global motion vector GMV _B , global motion is the motion vector information supplied to the motion compensation unit 360. There are a vector difference ΔGMV and a local motion vector difference ΔLMV. The motion compensation unit 360 obtains a local motion vector LMV of the decoding target frame with reference to the motion vector information, and performs motion compensation. The motion compensation unit 360 performs motion compensation such as the size of the motion search range for each global region provided as part of the motion vector information, the pixel accuracy for motion compensation, and the maximum value that can be taken by the local motion vector LMV for each global region. The following motion compensation processes are performed with reference to the parameters.

The global motion vector calculation unit 362 receives the input of the reference global motion vector GMV _B and the global motion vector difference ΔGMV of each global region from the variable length decoding unit 310, obtains the global motion vector GMV = ΔGMV + GMV _B for each global region, This is given to the motion vector calculation unit 364.

  The local motion vector calculation unit 364 receives the input of the local motion vector difference ΔLMV from the variable length decoding unit 310 and the input of the global motion vector GMV of each global region from the global motion vector calculation unit 362, and the local motion vector for each global region. LMV = ΔLMV + GMV is obtained. The local motion vector calculation unit 364 gives the local motion vector LMV in each global region to the image restoration unit 366.

  The image restoration unit 366 generates a prediction image using the reference image and the local motion vector LMV for each macroblock in each global region, adds the difference image given from the inverse DCT unit 330 and the prediction image, and adds the original image Restore and output the image.

  As described above, according to the encoding device 100 of the present embodiment, when encoding a motion vector, the motion vector accuracy and absolute amount can be changed by changing the number of bits of the motion vector for each region. Therefore, in the area where there is not much requirement, the number of bits of the motion vector can be reduced, and the encoding efficiency of the motion vector can be improved.

  Since the number of bits of the motion vector can be made different for each region, for example, the pixel accuracy can be increased in a region where the motion is fine, or the maximum value that can be taken by the motion vector can be increased in a region where the motion is intense. Conversely, the pixel accuracy can be lowered in a region where the motion is rough, or the maximum value that the motion vector can take can be reduced in a region where the motion is small. This makes it possible to optimally adjust the number of bits allocated to the motion vector according to the fineness of motion and the magnitude of motion for each region, or according to the accuracy of motion compensation required for each region. The reproduction quality of the image can be improved, and the compression efficiency of the moving image stream can be improved.

  In addition, when encoding a motion vector, the motion vector information in the spatial region is represented by a difference value from the global motion vector of the region, thereby reducing the data amount of the motion vector information itself and the entire moving image stream. It is possible to increase the compression efficiency by reducing the code amount. In addition, by providing a hierarchical structure between the global motion vectors of each spatial region and obtaining and encoding the difference of the global motion vectors between different layers, the amount of code of the motion vector information can be further reduced.

  Further, according to decoding apparatus 300 of the present embodiment, a motion vector is acquired for each area from a video stream with high compression efficiency in which a motion vector having a different number of bits for each area is encoded by encoding apparatus 100. By performing motion compensation, a moving image with high image quality can be restored. Since a motion vector having an optimum number of bits is encoded for each region, motion compensation for each region is performed efficiently and with high accuracy.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above description, the encoding device 100 and the decoding device 300 are MPEG series standards (MPEG-1, MPEG-2, and MPEG-4), H.264, and H.264. 26x series standards (H.261, H.262 and H.263), or H.264 Although video encoding and decoding are performed in accordance with H.264 / AVC, the present invention can also be applied to the case of hierarchical video encoding and decoding with temporal scalability. In particular, the present invention is effective in reducing the amount of motion vector codes in the encoding of motion vectors when the MCTF technique is used.

  In the above embodiment, the bit number adjustment unit 62 has adjusted the number of bits of the local motion vector for each global region for which the global motion vector is obtained. The present invention is not limited to the global area described in the embodiment. The motion compensation unit 60 may not include a configuration for obtaining and encoding a global motion vector, but may include only a configuration for detecting and encoding a local motion vector.

  Further, the encoding apparatus 100 is provided with an ROI region setting unit, a region of interest (ROI) is set on the moving image, and the bit number adjusting unit 62 performs local motion vector for each set region of interest. The number of bits may be adjusted.

  The attention area may be selected by the user specifying a specific area on the image, or a predetermined area such as a center area of the image may be selected. In addition, an important area such as an area in which a person or a character is shown may be automatically extracted as the attention area. Further, the attention area may be automatically selected in units of frames by tracking the movement of a specific object or the like in the moving image.

  Further, when priority is set between a plurality of attention areas, the bit number adjustment unit 62 may adjust the number of bits of the local motion vector in the attention area according to the priority of the attention area. The region of interest is encoded so as to be reproducible with an image quality corresponding to the priority, but the local motion vector can be increased by increasing the motion search range or increasing the pixel accuracy for motion compensation for the region of interest with higher priority. By increasing the number of bits, it is possible to further improve the image quality of the attention area reproduced by motion compensation.

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure explaining the structure of the motion compensation part of FIG. 3 is a flowchart illustrating a procedure for differential encoding of a motion vector by a motion compensation unit in FIG. 2. It is a figure explaining the example of the area | region set on an image by the area | region setting part of FIG. It is a figure explaining the example which calculates a global motion vector difference by the global motion vector difference encoding part of FIG. It is a figure explaining the bit number of the local motion vector adjusted by the bit number adjustment part of FIG. It is a block diagram of the decoding apparatus which concerns on embodiment. It is a figure explaining the structure of the motion compensation part of FIG.

Explanation of symbols

  20 DCT section, 30 quantization section, 60 motion compensation section, 62 bit number adjustment section, 64 area setting section, 66 local motion vector detection section, 68 global motion vector calculation section, 70 motion compensation prediction section, 72 local motion vector difference Encoding unit, 74 Global motion vector differential encoding unit, 80 frame memory, 90 variable length encoding unit, 92 multiplexing unit, 100 encoding device, 300 decoding device, 310 variable length decoding unit, 320 dequantization unit, 330 Inverse DCT unit, 360 motion compensation unit, 362 global motion vector calculation unit, 364 local motion vector calculation unit, 366 image restoration unit, 380 frame memory.

Claims (6)

  An encoding method comprising: defining a plurality of regions on a picture constituting a moving image and being encoded with inter-picture prediction encoding, and setting a condition for encoding a motion vector for each region.   The encoding method according to claim 1, wherein the motion vector encoding condition is a condition relating to pixel accuracy of motion compensation.   The encoding method according to claim 1, wherein the motion vector encoding condition is a condition related to a maximum value that the motion vector can take.   The encoding method according to any one of claims 1 to 3, wherein the motion vector encoding condition is included in the encoded data of the moving image in association with each region in which the condition is set.   5. The encoding method according to claim 1, wherein an area occupied by an object extracted from the moving image is set as one of the plurality of areas.   6. The encoding method according to claim 1, wherein a background area in the moving image is set as one of the plurality of areas.

Images