BRPI0304534B1 - method and apparatus for determining motion vector - Google Patents

Abstract

"motion vector calculation method". when a block (mb22) whose motion vector refers in direct mode contains a plurality of motion vectors, 2 motion vectors mv23 and mv24 which are used for interimaging prognosis of a current image (p23) to be encoded are determined by scaling a value obtained by averaging the plurality of motion vectors or selecting one from the plurality of motion vectors.

Description

Report of the Invention Patent for "METHOD AND APPARATUS FOR DETERMINING THE MOVEMENT VECTOR".

Technical Field The present invention relates to a moving image encoding and decoding method and particularly to a predictive encoding method referring to several preceding encoded images in the order of display or to several subsequent encoded images. in the display order or to multiple images both preceding and following in the display order. Background Art Generally, the volume of information is compressed by reducing redundancy in the temporal and spatial directions for moving image coding. Therefore, motion derivation and motion compensation are performed on a block-by-block basis referring to a preceding or following image and a coding is performed to a difference value between a predicted image obtained and a current image for interimaging prognostic coding to reduce temporal redundancy.

[003] In a H, 26L motion picture coding method, which is currently under standardization, an image with only intraimage prognostic coding (image I), an image for which interimage prognostic coding is performed by referring to refers to an image (hereinafter image P) and an image for which interimage prognostic coding is performed by referring to two preceding images in the display order or two following images in the display order or each of The preceding and following images in the display order (hereinafter, image B) are proposed. By way of example, H.26L is a new video coding standard designed to replace its H.263 predecessor, one of its main goals being to achieve 50% bit rate savings compared to H.263 and a Higher compression efficiency compared to H.263.

Figure 1 is an illustration showing an example of a reference relationship between each image according to the moving image encoding method and the reference images mentioned above.

In image 11, intraimage prognostic coding is performed without a reference image and in image P10, interimage prognostic coding is performed by referring to a preceding image in the display order, P7. In an image B6, interimage prognostic coding is performed by referring to two preceding images in the display order, in a B12 image, interimage prognostic coding is performed by referring to two subsequent images in the display order and a In image B18, interimage prognostic coding is performed by referring to each of the preceding and following images in the order of display.

[006] A forward mode is one of the dual prognostic prognostic modes that performs interimage prognostic coding by referring to each of the preceding and following images in the order of display. In direct mode, motion vectors for a block to be encoded are not encoded in the bitstream directly, and two motion vectors for actual motion compensation are calculated by referring to a motion vector of a block from the same location at an encoded image next to the image including the block to be encoded in display order and a prognostic block is generated.

[007] Figure 2 presents an example in which a coded image that is referred to in order to determine a direct mode motion vector contains a motion vector that refers to a preceding image in the display order. "P" indicated by a vertical line in figure 2 has nothing to do with an image type and it presents a simple image. In Figure 2, for example, an image P83, in which dual prediction is performed referring to images P82 and P84, is a current image to be encoded. If it is assumed that a block encoded in image P83 is an MB81 block, a motion vector of block MB81 is determined using a motion vector of a block of the same location MB82 in image P84 which is a coded reference image. back. Since block MB82 contains only one motion vector MV81 as a motion vector, two motion vectors MV82 and MV83 to be obtained are calculated directly by applying a scaling to an MV81 motion vector and a time interval. TR81 based on Equation 1 (a) and Equation 1 (b). MV82 = MV81 / TR81 x TR82 Equation 1 (a) MV83 = -MV81 / TR81 xTR83 Equation 1 (b) [008] In these equations, the time interval TR81 has a range between image P84 and image P82, ie a time interval between image P84 and a reference image indicated by motion vector MV81. Time interval TR82 has a time interval between image P83 and a reference image indicated by motion vector MV82. Time interval TR83 has a time interval between image P83 and a reference image indicated by motion vector MV83.

The direct mode includes two methods, the temporal prognosis already explained and the spatial prognosis and the spatial prognosis is explained below. In direct mode spatial prediction, for example, coding is performed on a 16 x 16 pixel-based macroblock and a motion vector, which is obtained by referring to an image closer to a current image to be encoded in the order display, is selected from the motion vectors in the three macroblocks in the vicinity of the current macroblock to be encoded and the selected motion vector is a motion vector for the current macroblock to be encoded. If three motion vectors refer to the same image, an average value is selected. If two of the three motion vectors refer to an image closer to a current image to be encoded in the display order, the remainder is considered as a "0" vector and an average value of these values is selected. If only 1 motion vector refers to an image closer to a current image to be encoded in the display order, this motion vector is selected. Thus, a motion vector is not coded for a current macroblock to be coded in direct mode and a motion prognosis is performed using a motion vector contained in another macroblock.

Figure 3A is an illustration showing an example of motion vector prediction method in the case where a preceding image in an image B in the display order is referred to using a conventional direct mode spatial prediction method. In this figure 3A, P indicates an image P, B indicates an image B and the numbers assigned to the image types in the four right images indicate an order in which each image is encoded. It should be assumed that a diagonally shaded macroblock in an image B4 is a current macroblock to be encoded. When a motion vector of a current macroblock to be coded is calculated using a direct mode spatial prediction method, first three coded macrobles (the shaded area with dashed lines) are selected from the macroblock adjoining. current to be encoded. The explanation of a method for selecting three neighboring macroblocks is omitted here. Motion vectors in the three coded macroblocks have already been calculated and stored. There is a case where the motion vector is obtained by referring to different images for each macroblock even if the macroblocks are in the same image. The reference indices in the reference images used to encode each macroblock may display which image is referenced from the three neighboring macroblock respectively. The details of the benchmarks will be explained later.

Now, for example, it is assumed that three neighboring macroblocks are selected for a current macroblock to be encoded shown in FIG. 3A and the motion vectors in each encoded macroblock are a motion vector a, b and c respectively. Here it is assumed that motion vector and motion vector b are obtained by referring to an image P with an image number 11 of "11" and motion vector c are obtained by referring to an image P with the image number 11 of "8". In this case, among these motion vectors a, b and c, motion vectors a and b that refer to an image closer to a current image to be encoded in order or display time are candidates for a vector and motion of a macroblock. current to be encoded. In this case, the motion vector c is considered as "0:" and an average value of these three motion vectors a, b and c is selected and determined as a motion vector of the current macroblock to be encoded.

However, an encoding method such as MPEG-4 can perform encoding for each macroblock in an image using a field structure and a frame structure. Therefore, in a coding method such as MPEG-4, there is a case where a macroblock encoded in the field frame and a macroblock encoded in the frame structure are mixed into a frame of reference frame. Even in such a case, if three macroblock in the vicinity of a current macroblock to be coded are encoded in the same structure as the current macroblock to be coded, it is possible to derive a motion vector from the current macroblock to be coded using the spatial prediction method mentioned. above in direct mode without any problems. That is, a case where three neighboring macroblocks are encoded in the frame structure for a current macroblock to be encoded in the frame structure, or in a case where three neighboring macroblocks are encoded in the field frame for a current macroblock to be encoded. in the field structure. The first case is as already explained. In the latter case, by using three motion vectors corresponding to the top fields of three neighboring macroblocks for a field above a current macroblock to be encoded and by using three motion vectors corresponding to the bottom fields of three neighboring macroblocks to be encoded. a lower field of the current macroblock to be encoded, a motion vector of the current macroblock to be encoded may be derived for the upper field and the lower field respectively using the method mentioned above.

However, in the direct mode temporal prognosis method, provided that the block mentioned above contains several motion vectors for the direct mode temporal prognosis when in a block with intra-image prognostic coding, direct mode motion compensation If a block to which the motion vector refers belongs to an image B such as B6 shown in Figure 1, a problem occurs where a motion vector calculation by a scaling based on Equation 1 cannot be applied. Additionally, there is a case where the motion vector value accuracy (half pixel precision and quarter pixel precision, for example) does not reach the predetermined precision since the division operation is performed after the calculation. of motion vector.

When a current macroblock to be encoded and one of the neighboring macroblocks are encoded in a different structure for spatial prediction, it is not specified which of the field structure and frame structure is used to encode the current macroblock to be encoded. , and a method for selecting a motion macroblock motion vector to be encoded from neighboring macroblock motion vectors encoded in both the field structure and the frame structure is not specified.

[0015] The first object of the present invention is to provide a method of predicting motion vector in a temporal direction with high precision in direct mode, even if a block whose motion vector is referred to belongs to an image B.

[0016] The second object of the present invention is to provide a high precision spatial direction motion vector prediction method, even if a block whose motion vector is referred to belongs to a B image.

In order to achieve the above objective, a motion vector calculation method according to the present invention is a motion vector calculation method for interim prognosis with reference to a plurality of images. . The motion vector calculation method comprises a reference step for referring to a plurality of preceding images in the display order or a plurality of subsequent images in the display order or a plurality of both preceding and following images. in the display order and a motion compensation step, with reference to a motion vector of a block of the same location in an image other than an image to which an interimage prognostic block belongs when it performs motion compensation of the motion block. interimage prognosis, to calculate a motion vector of the interimage prognostic block using at least one motion vector, which satisfies a predetermined condition, between motion vectors already calculated for the block located at the same location to which the motion vector refers . Therefore, according to the motion vector calculation method of the present invention, when in a block with interimage prognostic coding, motion compensation is performed by referring to a motion vector of a block located at the same location in another coded image and when a block to whose motion vector it refers and contains a plurality of motion vectors, motion compensation can be realized without contradiction by generating a motion vector used to scale between the plurality of motion vectors. Motion vectors.

For the motion vector calculation method according to the present invention, in the reference step, each of the images selected from a first image order and a second image order may be referred to. Here, the first image order is the order in which ID numbers are assigned to ascending sequence images giving priority to a preceding image in the display order, and the second image order is the order in which ID numbers are assigned to images in ascending sequence giving priority to a next image in the order of their display. In the motion compensation step, a motion vector, which refers to an image in the first image order of the block whose motion vector is referred to, may be used. For the method, even if a block whose motion vector refers to belongs to image B, the one used for block motion compensation with interimage prognosis can be determined as a motion vector by referring to an image in the first image. image order and scaling motion vector calculation method can be applied.

In addition, another method of motion vector calculation according to the present invention includes a designation step, a first selection step and a derivation step. The designation step is to designate one of a first reference index or a second reference index for the encoded image. Here, the first reference index and the second reference index are used to select at least one of a first reference image or a second reference image that is referred to when obtaining a block in a current image to be encoded by offset. movement from a plurality of encoded images stored in a storage unit. The first selection step is to select a motion vector indicating an average value of motion vectors, when performing motion compensation of a block in the current image to be encoded, there is a plurality of motion vectors containing the first indexes of motion. reference between the block motion vectors in the vicinity of the current image to be encoded. The derivation step is to derive a motion vector by referring to an image preceding the current image to be encoded in the order of display or an image following the current image to be encoded in the order of display or images preceding and following the current image to be encoded in the display order using the motion vector selected in the first selection step. Therefore, when motion compensation is performed for a block in the current image to be encoded and when there is a plurality of motion vectors containing the first reference index between block vectors in the vicinity of the block in the current image to be encoded , the motion vector of the current image to be encoded may be derived using a motion vector indicating an average value of motion vectors.

For the motion vector calculation method according to the present invention, a motion vector indicating an average value of the lowest first reference indices can be additionally selected from the motion vectors containing the first reference indices in the first selection step.

Additional information about the Technical Basis for this Application: Japanese Patent Application No. 2002-118598 filed April 19, 2002;

Japanese Patent Application No. 2002-121053 filed April 23, 2002;

Japanese Patent Application No. 2002 - 156266 filed May 29, 2002;

Japanese Patent Application No. 2002 - 177889 filed June 19, 2002;

Japanese Patent Application No. 2002 - 193027 filed July 2, 2002;

Japanese Patent Application No. 2002-204713 filed July 12, 2002;

Japanese Patent Application No. 2002 - 262151 filed September 6, 2002;

Japanese Patent Application No. 2002 - 290542 filed October 2, 2002;

Japanese Patent Application No. 2002 - 323096 filed November 6, 2002;

Japanese Patent Application No. 60/378643 filed May 9, 2002;

Japanese Patent Application No. 60/378954 filed May 10, 2002;

Are incorporated herein by reference.

Brief Description of the Drawings Figure 1 is a schematic diagram showing a reference relationship of the images of a conventional example.

[0035] Figure 2 is a schematic diagram showing an operation in a conventional direct mode.

Figure 3A is an illustration showing an example of a motion vector prediction method when a preceding temporal shape image is referred to in an image B using a conventionally straightforward spatial prediction method, [0037] A Figure 3B is an illustration showing an example of a reference list generated in each current image to be encoded.

Figure 4 is an explanatory illustration of image numbers and reference indices.

Figure 5 is an illustration showing a concept of an image coding signal format of a conventional image coding apparatus. Figure 6 is a block diagram showing an operation for coding according to first and second embodiments of this invention, Figure 7 is a schematic diagram showing an operation when a block to which motion vector refers in direct mode contains two motion vectors which refer to the preceding time in the order display

[0042] Figure 8 is a schematic diagram comparing a reference frame of images in display order and coding order.

[0043] Figure 9 is a schematic diagram showing an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the next time in the display order.

[0044] Figure 10 is a schematic diagram comparing an image reference ratio in display order and coding order.

Fig. 11 is a block diagram showing an operation to decode according to the fifth and sixth embodiments of the present invention.

[0046] Figure 12 is a schematic diagram showing an operation when a block to which motion vector refers in direct mode contains two motion vectors that refer to the preceding time in the display order.

Figure 13 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors which refer to the next time in the display order.

[0048] Figure 14 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors which refer to the next time in the display order.

Fig. 15 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the preceding time in the display order.

Fig. 16 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the preceding time in the display order.

Figure 17 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the preceding time in the display order.

Figure 18 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the preceding time in the display order.

Fig. 19 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors which refer to the next time in the order of display.

[0054] Figure 20 is a schematic diagram for presenting an operation when a block to whose motion vector refers in direct mode contains two motion vectors that refer to the next time in the display order.

Fig. 21 is a schematic diagram for presenting an operation when a block to which motion vector refers in direct mode contains two motion vectors which refer to the next time in the order of display.

Fig. 22 is a schematic diagram for presenting an operation when a block to which motion vector refers in direct mode contains two motion vectors which refer to the next time in the display order.

Fig. 23 is a schematic diagram for presenting an operation when a motion vector of a neighboring block is referred to in direct mode.

Fig. 24 is an illustration showing a bit stream.

Fig. 25 is an illustration showing a relationship between a current block to be encoded and a block in the vicinity of the current block to be encoded.

Fig. 26 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 27 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 28 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 29 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 30 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 31 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 32 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

Fig. 33 is an illustration showing a motion vector contained in a block in the vicinity of a current block to be encoded.

[0068] Figure 34 is an illustration showing a procedure for determining a motion vector to be used in direct mode.

Fig. 35 is an illustration showing a relationship between a current block to be encoded and a block in the vicinity of the current block to be encoded.

Fig. 36 is an illustration showing a procedure for determining a motion vector of a current block to be encoded using a value of a reference index.

Figure 37 is an illustration showing the double prognosis in direct mode when a motion vector referring to an image stored in a long term buffer is only one.

Fig. 38 is an illustration showing the double prognosis in direct mode when motion vectors referring to an image stored in long term buffer are two.

Fig. 39 is an illustration showing a process flow of a motion vector calculation method.

Fig. 40 is a block diagram showing a configuration of a moving image coding apparatus 100 according to the eleventh embodiment of the present invention.

Fig. 41A is an illustration showing an informed frame order for the moving image coding apparatus 100 in the image-by-image time order.

Fig. 41B is an illustration showing the case where the frame order shown in Fig. 41A is reordered in the coding order.

Fig. 42 is an illustration showing a structure of a list of reference images for explaining the first mode.

Fig. 43A is a flowchart showing an example of a motion vector calculation procedure using a direct mode spatial prediction method when a pair of macroblock to be encoded in a field structure and a pair of ma -blocks to be encoded in a frame structure are mixed.

Fig. 43B is an illustration showing an example of a location of neighboring macroblock pairs to which the present invention is applied when a current macroblock pair to be encoded is encoded in a frame structure.

Fig. 43C is an illustration showing an example of locating neighboring macroblock pairs to which the present invention is applied when a current macroblock pair to be encoded is encoded in a field structure.

Fig. 44 is an illustration showing a data configuration of a macroblock pair when encoding is performed in a frame structure and a data configuration of a macroblock pair when encoding is performed in a field structure.

Fig. 45 is a flowchart showing a detailed processing procedure in step S302 shown in Fig. 43.

[0083] Fig. 46 is an indicator graph showing a relationship between reference field indices and frame reference indices.

Fig. 47 is a flowchart showing a detailed processing procedure in step S303 shown in Fig. 43.

Fig. 48 is an illustration showing a position relationship between a current macroblock pair to be encoded and neighboring macroblock pairs in order to explain the first embodiment.

Fig. 49 is an illustration showing a position relationship between a current macroblock pair to be encoded and neighboring macroblock pairs in order to explain the first embodiment.

Fig. 50 is an illustration showing an example of a bit stream data configuration 700 generated by a bit stream generating unit 104.

Fig. 51 is a block diagram showing a configuration of a moving picture decoding apparatus 800 which decodes the bit stream 700 shown in Fig. 50.

Fig. 52A is an illustration showing an example of a physical format of a floppy disk that is a body of a storage medium.

Fig. 52B is an illustration showing an external view of the floppy disk viewed from the front, a section configuration and the floppy disk.

[0091] Fig. 52C is an illustration showing a configuration for writing and reading the above mentioned program on a floppy disk, FD.

Fig. 53 is a block diagram showing the entire configuration of the content delivery system implementing a content delivery service.

[0093] Fig. 54 is an illustration showing an example of a cell phone appearance.

Fig. 55 is a block diagram showing a cell phone configuration.

Fig. 56 is an illustration for presenting a device performing the encoding or decoding process set forth in the above embodiments and a system utilizing the device.

Best Mode for Carrying Out the Invention The present invention is for solving the problems of conventional technology and aims to propose a coding method and a moving image decoding method that can determine a motion vector used for motion compensation. without contradiction, even if a block whose motion vector is referenced in a direct mode is an image B. First, the reference indices will be explained here.

Figure 3B is an illustration showing an example of a reference image list 10 generated for each current image to be encoded. In the reference image list 10 shown in Figure 3B, the images are presented preceding and following an image B in the display order with an image B in the center and images to which image B may refer, image types, a 11, the first benchmark 12 and the second benchmark 13 are displayed. Image number 11 is, for example, a number having an order in which each image is encoded. The first reference index 12 is the first index having a relative position relationship between a current image to be encoded and neighboring images and, for example, is used primarily as an index when a current image to be encoded refers to a preceding image in the display order. A list of first benchmarks 12 is called a "benchmark list 0 (list 0)" or "first benchmark list". Also, the reference index is called the relative index. First, in the reference image list 10 shown in Figure 3B, the integer that is advanced by "1" is assigned a value from the first reference index 12 of "0" from the nearest to a current image. to be encoded in a time sequence to a reference image preceding a current image to be encoded in the order of display. Then, after a value advanced by "1" from "0" is assigned to all reference images following a current image to be encoded in the display order, the following values are assigned to reference images following the image. current to be encoded in display order from closest to the current image to be encoded in display order.

The second reference index 13 is the second index having a relative position relationship between a current image to be encoded and neighboring images and, for example, is used primarily as an index when a current image to be encoded refers to to a following image in the display order. A list of second benchmarks 13 is called a "benchmark list 1 (list 1)" or "second benchmark list". First, the integer that is advanced by "1" is assigned a value from the second reference index 13 that is "0" from the nearest to a current image to be encoded in the display order. Then, after an advanced value of "1" from "0" is assigned to all reference images following a current image to be encoded in the display order, the following values are assigned to reference images preceding an image. current to be encoded in the display order from the nearest value to a current image to be encoded in the display order. Therefore, it is seen in the reference image list 10 that as for the first reference index 12 and the second reference index, a reference image with the smaller reference index value is closest to the current image to be encoded in the reference image. Display order. A method for assigning a reference index number in the initial state is explained above, however, the method for assigning a reference index number can be changed on an image by image basis or on a slice by slice basis. In the method for assigning a reference index number, for example, a small number may be assigned to a distant image in the display order, however, such a reference index is used, for example, when coding efficiency is improved by referencing. to the distant image in the display order. In other words, since the reference indices in a block are presented by variable-length codewords and shorter length data are assigned to the smaller value indices by designating the smaller reference index for the refining reference image. coding efficiency, if it is referred to, the number of codes in the reference indices is reduced and the additional coding efficiency is improved.

[0099] Figure 4 is an explanatory illustration for image numbers and reference indices. Figure 4 shows an example of the reference image list and shows a reference image, an image number and a reference index used when coding image B in the center (indicated by a dashed line). Figure 4A presents the case designating benchmarks by the method for assigning benchmarks in the initial state explained using figure 3.

Figure 5 is a conceptual diagram of an image coding signal format of a conventional image coding apparatus. Image indicates a coding signal for an image, Header indicates a header coding signal included in the header of an image, Blocol indicates a coding signal in a direct mode coded block, Block2 indicates a coding signal in a coded block. By a non-direct interpolation prognosis, RidxO and Ridxl are the first reference index and the second reference index respectively, and MVO and MV1 are the first motion vector and the second motion vector respectively. Block coded Block 2 has two reference indices RidxO and Ridxl in a coding signal in this order to indicate two reference images to be used for motion compensation. In addition, the first motion vector MV1 and the second motion vector MV2 are encoded in the coding signal of coded block Block2 in this order. It can be judged by "PredType" which of the RidxO and / or Ridxl benchmarks is used. An image (the first reference image) referred to by the first MVO motion vector is indicated by the first reference index RidxO and an image (the second reference image) referred to by the second motion vector MV1 is indicated by the second reference index Ridx1. For example, when it is indicated that images are referred to bidirectionally by the motion vectors MVO and MV1, RidxO and Ridxl are used, when it is indicated that images are referred to unidirectionally by one of the motion vectors MVO or MV1, a RidxO or Ridxl corresponding to the motion vector is used and when direct mode is indicated, neither RidxO nor Ridxl is used.

[00101] The first reference image is specified by the first reference index and generally has the display time preceding a current image to be encoded and the second reference image is specified by the second reference index and usually has the following display time. the current image to be encoded. However, as the method for designating reference indices in Figure 4 presents, there is a case where the first reference image contains the display time following the current image to be encoded and the second reference image contains the display time preceding the current image to be encoded. The first reference index RidxO is a reference index indicating the first reference image referred to by the first MVO motion vector of Block 2 and the second reference index Ridx1 is a reference index indicating the second reference image referred to by the second reference vector. movement MV1 of block Block2.

On the other hand, a reference image designation for reference indices can be arbitrarily altered by explicitly indicating using a memory control signal in an encoded signal (RPSL in the Header in Figure 5). This makes it possible to change the reference image with the second reference index "0" to an arbitrary reference image. For example, as shown in Figure 4B, the designation of reference indices for image numbers may be changed.

Thus, since the reference image designation for reference indices can be changed arbitrarily and changing the designation of reference images for reference indices generally designates a lower reference index for an image which improves efficiency. If encoding is selected as a reference image, encoding efficiency can be enhanced by using a motion vector, which refers to an image whose reference index is the smallest, such as a motion vector used in direct mode. . (First Modality) A motion picture coding method according to the first embodiment of the present invention is explained using the block diagram in Figure 6.

[00105] A current moving image to be encoded is entered into a frame memory 101 in an image-by-image display order and reordered in an encoding order. Each image is divided into a group called a block, which is 16 (horizontal) x 16 (vertical) pixels in size, for example and the following processes are performed on a block by block basis.

A block read from frame memory 101 is reported to a motion vector detection unit 106. Here, a motion vector of a current block to be encoded is detected using a decoded image of an image. coded memory stored in frame memory 105 as a reference image. In this case, in a mode selection unit 107, an optimal prognostic mode is determined with reference to a motion vector obtained from motion vector detection unit 106 and a motion vector used in an encoded image stored in a unit. vector storage mode 108. A prediction mode obtained in mode selection unit 107 and a motion vector used in the obtained mode are reported to a difference calculation unit 109 and a prognostic residual image is generated by the computation of a difference of a current block to be coded and coding is performed on a predictive residual coding unit 102. In addition, the motion vector used in the mode obtained in mode selection unit 107 is stored in a data storage unit. motion vector 108 to be used for encoding by the following blocks and images. An above processing stream is an operation when an interimage prognostic coding is selected, however, a key 111 switches to an intraimage prognostic coding. Eventually, variable length coding is performed for control information such as a motion vector and image information such as image information output from the predictive residual coding unit 102 and a bit stream. eventually issued is generated by a bitstream generation unit 103.

[00107] A summary of the coding flow is explained above, however, the process details in motion vector detection unit 106 and mode selection unit 107 are explained below.

Motion vector detection is performed on a block by block basis or on an area by area basis (area is a divided block). Using the coded images preceding and following a current image to be coded in display order as reference images, a prognostic image and a prognostic mode showing a location that is optimally predicted in the image search area is generated by the decision by A motion vector.

[00109] A direct mode is one of the dual predictions that can perform interimage prediction coding prediction by referring to two preceding and / or next images in the display order. In direct mode, a current block to be encoded does not contain a motion vector directly and two motion vectors for real motion compensation are calculated by referring to a motion vector of a block of the same location in a coded image nearby in the display order and a predictive block is generated.

[00110] Figure 7 shows an operation when a coded block referred to to determine a direct mode motion vector contains two motion vectors that refer to two preceding images in the display order. A P23 image is a current image to be encoded and performs dual prediction referring to images P22 and P24. Assume that a block to be encoded is an MB21 block; and two required motion vectors are determined using a motion vector contained in an MB22 block, which is a block at the same location in the next coded reference image (the second reference image specified by the second reference index) P24. Since block MB22 contains motion vectors MV21 and MV22 as a motion vector, it is impossible to calculate two required motion vectors MV23 and MV24 by scaling directly similarly to Equation 1. Therefore, similarly to that Equation 2 presents a motion vector MV REF is calculated as a motion vector scaled from an average value of two motion vectors contained in block MB22 and a time interval TR_REF at the same time is calculated as from the average value. Then the motion vectors MV23 and MV24 are calculated by adjusting the scale of the motion vector MV_REF and the time interval TR REF based on Equation 3. In this case, the time interval TR21 indicates a time interval between the image P24. and image P21, i.e. an image referred to by motion vector MV21 and time interval TR22 indicates a time interval to an image referred to by motion vector MV22. In addition, time interval TR23 is a time interval to an image referred to by motion vector MV23 and time interval TR24 is a time interval to an image referred to by motion vector MV24. The time intervals between these images may be determined based, for example, on the information indicating the display time and the display order added for each image or the information difference. Note that a current image to be encoded refers to a next image in the example of Figure 7, however, the case referring to a non-next image may be treated in the same way. MV_REF = (MV21 + MV22) / 2 .... Equation 2 (a) TR_REF = (TR21 + TR22) / 2 .... Equation 2 (b) MV23 = MV_REF / TR_REF x TR23 .... Equation 3 ( a) MV24 = -MV_REF / TR_REF x TR24 .... Equation 3 (b) [00111] The above embodiment presents the coding method in which interimage prognostic coding can be performed using direct mode without contradiction even if a block whose motion vector refers in direct mode to an image B. In the encoding method, when a block to whose motion vector refers in direct mode contains several motion vectors that refer to an image Preceding in the display order, a motion vector is generated using the various motion vectors and two motion vectors to be used for actual motion compensation are determined by scaling. Note that when two motion vectors MV23 and MV24 in figure 7 are calculated, it is possible to use Equation 4 instead of Equation 2 as a method for averaging MV21 and MV22 motion vectors and averaging time intervals. TR21 and TR22 to calculate the motion vector MVREF and the time interval TR REF to have the scale adjusted.

First, as Equation 4 (a) presents, motion vector MV21 'is calculated by adjusting the scale of MV21 to match the time interval with motion vector MV22. Then the MV REF motion vector is determined by averaging the motion vectors MV21 'and MV22. Here the TR22 time slot is used directly as the TR_RF time slot. Note that the case of calculating a motion vector MV22 'by adjusting the scale of the motion vector MV22 rather than calculating the motion vector MV21' by adjusting the scale of a motion vector MV21 can be treated in the same way. MV21 '= MV21 / TR21 x TR22 .... Equation 4 (a) MV_RF = (MV21' + MV22) / 2 .... Equation 4 (b) TR_REF = TR22 .... Equation 4 (c) [00113 Note that when two motion vectors MV23 and MV24 in figure 7 are calculated, as a motion vector MV_REF and a time interval TR REF that have the scaled scale, a motion vector MV22 and a time interval TR22, which refers to a P22 image located closest to a P24 image whose motion vector it refers to, can be directly used as Equation 5 presents instead of using an average value of two motion vectors as Equation 2 presents. Likewise, as a motion vector MV REF and a time interval TR_REF, a motion vector MV21 and a time interval TR21, which refers to a temporarily distant located P21 image, can be directly used as the Equation 6 presents. This makes it possible to reduce the capacity of a motion vector storage unit in a coding apparatus since each block belonging to a P24 image whose motion vector it refers to can perform motion compensation by storing only one of two motion vectors. . MV_REF = MV22 .... Equation 5 (a) TR_REF = TR22 .... Equation 5 (b) MV_REF = MV21 .... Equation 6 (a) TR_REF = TR21 .... Equation 6 (b) [00114 ] Note that when two vectors and motion MV23 and MV24 in figure 7 are calculated, as a motion vector MV_REF and a time interval TR_REF that are scaled, a motion vector that refers to an image to be encoded previously. can be directly used instead of using an average value of two motion vectors as Equation 2 presents. Figure 8A shows a reference relationship in the moving image display order as Figure 7 shows and Figure 8B shows an example of an order in which images are reordered by the frame memory coding order 101 shown in Figure 8A. Figure 6. Here, an image P23 indicates a frame to be encoded in direct mode and an image P24 indicates an image whose motion vector refers to the encoding. When images are rearranged as shown in Fig. 8B, since a motion vector referring to an image to be encoded previously is directly used, a motion vector MV22 and a time interval TR22 are directly used as a motion vector MV_REF and as a timeframe TR REF as presented in Equation 5. Likewise, it is possible to directly use a motion vector that refers to an image to be encoded later. In this case, a motion vector MV21 and a time interval TR21 are directly applied as a motion vector MV_REF and as a time interval TR_REF as Equation 6 presents. This makes it possible to reduce the capacity of a motion vector storage unit in a coding apparatus as long as each block belonging to a P24 image, said motion vector can perform motion compensation by storing only one of the two motion vectors. .

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by scaling a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images.

Note that in Equation 2 (a) or 4 (b), when a motion vector MV_REF is calculated, after calculating the right side of Equation 2 (a) or 4 (b), the motion vector can be rounded. for a predetermined motion vector accuracy (for example, rounded to a value of 0.5 unit pixels for a half pixel precision motion vector). The accuracy of a motion vector is not limited to half pixel accuracy. In addition, the accuracy of a motion vector can be determined based on block, image based and sequence based, for example. Moreover, in Equations 3 (a), 3 (b) and 4 (a), when motion vectors MV23, MV24 and MV21 'are calculated, motion vectors can be rounded to a predetermined precision of a vector of movement after calculating the right side of Equations 3 (a), 3 (b) and 4 (a). (Second Modality) An overview of the coding process based on Figure 6 is totally the same as the first modality. Here, a detailed operation of the dual prognosis in direct mode is explained using figure 9.

[00118] Figure 9 shows an operation when a block referred to for determining a direct mode motion vector contains two motion vectors which refer to the next two images in the order of display. A P43 image is a current image to be encoded and performs dual prediction referring to images P42 and P44. Assume that a block to be encoded is an MB41 block, then two required motion vectors are determined using a motion vector from a block of the same location MB42 in the back coded reference image (the second reference image specified by the second one). reference index) Ρ44. Since block MB42 contains two motion vectors MV45 and MV46 as the motion vectors, two required motion vectors MV43 and MV44 cannot be calculated by directly scaling similarly to Equation 1. Therefore, as the Equation 7 presents, a motion vector MV_REF is determined as a motion vector scaled from an average value of two MB42 motion vectors and a time interval TR REF at this time is likewise determined at from an average value. Motion vectors MV43 and MV44 are calculated by scaling an MV_REF motion vector and a time interval TR_REF based on Equation 8. In this case, a time interval TR45 indicates a time interval between an image P44 and P45. i.e. even an image that is referred to by a motion vector MV45; and time interval TR46 indicates a time interval to an image that is referred to by a motion vector MV46. A time interval TR43 indicates a time interval up to an image that is referred to by a motion vector MV43; and a time interval TR44 indicates a time interval up to an image that is referred to by a motion vector MV44. The time intervals between these images may be determined based, for example, on information indicating the display time and the display order that is added for each image or information difference as explained in the first embodiment. Note that a current image to be encoded refers to a next image in the example of Figure 9, however, the case referring to an image that is not next can be treated in the same way. MV_REF = (MV45 + MV46) / 2 .... Equation 7 (a) TR_REF = (TR45 + TR46) / 2 .... Equation 7 (b) MV43 = - MV_REF / TR_REF x TR43 .... Equation 8 (a) MV44 = MV_REF / TR_REF χ TR44 .... Equation 8 (b) [00119] The above embodiment presents the coding method in which interimage prognostic coding can be performed using direct mode without contradiction even if a block , wherein said direct mode motion vector belongs to an image B. In the coding method, when a block whose said direct mode motion vector contains several motion vectors which refer to a following image in the order of display, A motion vector is generated using the various motion vectors and two motion vectors to be used for actual motion compensation are determined by scaling.

Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, it is possible to use Equation 9 instead of Equation 7 as a method for averaging the motion vectors MV45 and MV46 and for averaging the time intervals TR45 and TR46 to calculate motion vector MV_REF and time interval TR_REF to scale. First, as Equation 9 (a) presents, motion vector MV46 'is calculated by adjusting the scale of MV46 to match the time interval with motion vector MV45. Then the motion vector MV_REF is determined by averaging the motion vectors MV46 'and MV45. Here the TR41 time slot is used directly as the TR_REF time slot. Note that the case of calculating a motion vector MV45 'by adjusting the scale of the motion vector MV45 rather than calculating the motion vector MV46' by adjusting the scale of a motion vector MV46 can be treated in the same way. MV46 '= MV46 / TR46 x TR45 .... Equation 9 (a) MV_REF = (MV46' + MV45) / 2 .... Equation 9 (b) TR REF = TR45 .... Equation 9 (c) [ Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, such as an MV REF motion vector and a TR_REF time interval that have the scale adjusted, an MV45 motion vector and a TR45 time interval, the which refer to an image P45 located temporarily closest to an image P44 whose said motion vector can be directly used as Equation 10 presents instead and use an average value of two motion vectors as Equation 7 presents. Likewise, as a motion vector MV REF and a time interval TRREF, a motion vector MV46 and a time interval TR46, which all refer to a more distant, temporarily located image P46, can be directly used as the Equation. 11 presents. This method makes it possible to reduce the capacity of a motion vector storage unit in a coding apparatus as long as each block belonging to a P44 image, said motion vector of which can implement motion compensation by storing only one of two motion vectors. movement. MV_REF = MV45 .... Equation 10 (a) TR_REF = TR45 .... Equation 10 (b) MV_REF = MV46 .... Equation 11 (a) TR_REF = TR46 .... Equation 11 (b) [00122 ] Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, such as an MV REF motion vector and a TR REF time interval that have the scale adjusted, a motion vector that refers to an image to be The coded above can be directly used instead of using an average value of two motion vectors as Equation 7 presents. Fig. 10A shows an image reference relationship in the moving image display order as Fig. 9 shows and Fig. 10B shows an example of an order in which images are reordered in the frame memory coding order 101 shown in Fig. 10A. Figure 6. Here, an image P43 indicates an image to be encoded in direct mode and an image P44 indicates an image whose motion vector refers to encoding. When images are reordered as shown in Fig. 10B, since a motion vector referring to an image to be encoded previously is directly used, a motion vector MV46 and a time interval TR46 are directly used as a motion vector. motion MV_REF and as a time interval TR_REF as presented in Equation 11. Similarly, it is possible to directly use a motion vector that refers to an image to be encoded later. In this case, a motion vector MV45 and a time interval TR45 are directly applied as a motion vector MV_REF and as a time interval TR REF. This method makes it possible to reduce the capacity of a motion vector storage unit in a coding apparatus as long as each block belonging to a P44 image, whose said motion vector can perform motion compensation by storing only one of two motion vectors. movement.

Note that when an image that is referenced to determine a direct mode motion vector contains two motion vectors that refer to two subsequent images in the display order, it is possible to perform motion compensation by assuming that two Required motion vectors MV43 and MV44 are "0". This method makes it possible to reduce the capacity of a motion vector storage unit in a decoding apparatus and further makes it possible to omit a process for calculating a motion vector, provided that each block belonging to a P44 image, whose vector of Said motion does not have to store a motion vector.

Note that when an image that is referenced to determine a direct mode motion vector contains two motion vectors that refer to two subsequent images in the display order, it is possible to inhibit referring to a motion vector. move and apply only a different prognostic coding than direct mode. When the next two images in the display order are referred to as an image P44 shown in figure 9, it is conceivable that the correlation with a preceding image in the display order is low, because it is possible to generate a more accurate prognostic image by inhibiting the mode. select another prognostic method.

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by scaling a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images.

Note that in Equation 7 (a) or 9 (b), when a motion vector MV REF is calculated, after calculating the right side of Equation 7 (a) or 9 (b), the motion vector can be rounded to a predetermined accuracy of the motion vector. The accuracy of a motion vector includes half-pixel accuracy, one-third pixel accuracy, and quarter-pixel precision or the like. In addition, the accuracy of a motion vector can be determined, for example, block-based, image-based, and sequence-based. Furthermore, in Equations 8 (a), 8 (b) and 9 (a), when motion vectors MV43, MV44 and MV46 'are calculated, motion vectors can be rounded to a predetermined precision of a vector. of motion after calculating the right side of Equations 8 (a), 8 (b) and 9 (a). (Third Mode) A motion picture decoding method according to the third embodiment of the present invention is explained using the block diagram shown in Figure 11. However, it is assumed that the bit stream generated in the encoding method first mode is informed.

First, various information such as a predictive mode, motion vector information, and predictive residual coding data are extracted from the bitstream reported by a bitstream analyzer 601.

Prediction mode and motion vector information is output to a predictive mode / motion vector decode unit 608 and predictive residual coding data is output to a predictive residual decode unit 602. A predictive mode / motion compensation decoding unit 608 decodes the predictive mode and a motion vector used in the predictive mode. In motion vector decoding, a decoded motion vector stored in motion vector storage unit 605 is used. Decoded predictive mode and motion vector are output to a motion compensation decode unit 604. In addition, the decoded motion vector is stored in motion vector storage unit 605 for use to decode the motion vectors of the following blocks. In motion compensation decoding unit 604, a predictive image is generated based on the entered predictive mode and motion vector information using a decoded image stored in frame memory 603 as a reference image. A decoded image is generated by informing the prediction image generated above for an addition operation unit 606 and adding the informed image to the prognostic residual image generated in a prognostic residual decoding unit 602. The above embodiment has an operation for an interimage prognostic stream and bits, however, a key 607 switches to a decoding process for an intraimage prognostic bit stream.

[00130] A summary of a decoding stream is presented above, however, the detailed process in motion compensation decoding unit 604 is explained below.

Motion vector information is added based on block or area-based (a divided block). By using decoded images preceding and following a current image to be encoded in the display order as reference images, a prognostic image for performing motion compensation from the images is generated.

[00132] A direct mode is one of the dual predictions that performs interimage prediction coding by referring to each of the preceding and following images in the order of display. In direct mode, since a current block to be encoded reports a bit stream that does not contain a motion vector directly, two motion vectors for the actual motion compensation are calculated by referring to a motion vector of a block of the motion. same location on a nearby decoded image in the display order and a prognostic image is generated.

Fig. 7 shows an operation when a decoded image referred to as determining a direct mode motion vector contains two motion vectors which refer to two preceding images in the order of display. A P23 image is a current image to be decoded and performs dual prediction referring to images P22 and P24. When it is assumed that a block to be decoded is an MB21 block, two required motion vectors are determined using a motion vector from a block of the same location MB22 in the decoded back reference image (the second reference image specified by the second index). reference number) P24. Since block MB22 contains two motion vectors MV21 and MV22 as the motion vectors, the two required motion vectors MV23 and MV24 cannot be calculated by applying direct scaling similarly to Equation 1. Therefore, as the Equation 2, a motion vector MV_REF is determined as a motion vector scaled from an average value of two motion vectors of block MB22 and a time interval TR_REF at this time is also determined from a value medium. Then the motion vectors MV23 and MV24 are calculated by scaling an MV REF motion vector and a TR_REF time interval based on Equation 3. In this case, a time interval TR21 indicates a time interval between a P24 and P21, i.e. even an image that is referred to by a motion vector MV21 and a time interval TR22 indicates a time interval to an image that is referred to by a motion vector MV22. A time interval TR23 indicates a time interval up to an image that is referred to by a motion vector MV23; and a time interval TR24 indicates a time interval up to an image that is referred to by a motion vector MV24. The time intervals between these images can be determined based, for example, on the information indicating the display time and the display order added for each image or the information difference. Note that a current image to be encoded refers to a next image in the example of Figure 7, however, the case referring to a non-next image may be treated in the same way.

[00134] The above embodiment presents the decoding method in which an interimage prognostic decoding can be performed using the direct mode without contradiction even if a block whose said motion vector belongs to an image B. In the decoding method, when a block , wherein said direct mode motion vector contains several motion vectors that refer to a preceding image, one motion vector is generated using the various motion vectors and two motion vectors to be used for the actual motion compensation are determined by scaling.

Note that when two motion vectors MV23 and MV24 in figure 7 are calculated, it is possible to use Equation 4 instead of Equation 2 as a method for averaging the motion vectors MV21 and MV22 and for averaging the time intervals TR21 and TR22 to calculate motion vector MV REF and time interval TRREF to scale. First, as Equation 4 (a) presents, motion vector MV21 'is calculated by adjusting the scale of MV21 to match the time interval with motion vector MV22. Then the motion vector MV_REF is determined by averaging the motion vectors MV21 'and MV22. Here, the TR22 time slot is used directly as the TR REF time slot. Note that the case of calculating a motion vector MV22 'by adjusting the scale of the motion vector MV22 instead of calculating the motion vector MV2T by adjusting the scale of a motion vector MV21 can be treated in the same way.

Note that when two motion vectors MV23 and MV24 in figure 7 are calculated, as a motion vector MV_REF and as a time interval TR_REF that have the scaled scale, a motion vector MV22 and a time interval TR22, which refer to an image P22 located temporarily closest to an image P24, whose motion vector refers, can be directly used as Equation 5 presents instead of using an average value of two motion vectors as Equation 2 presents. Likewise, as a motion vector MV REF and as a time interval TR_REF, a motion vector MV21 and a time interval TR21, which refer to a temporarily distant located P21 image, can be directly used as the Equation 6 presents. This method makes it possible to reduce the capacity of a vector storage unit in a coding apparatus as long as each block belonging to a P24 image, said motion vector of which can perform motion compensation by storing only one of two motion vectors.

Note that when two motion vectors MV23 and MV24 in figure 7 are calculated, as a motion vector MV REF and as a time interval TR_REF that are scaled, a motion vector that refers to an image. to be encoded earlier can be directly used instead of using an average value of two motion vectors as Equation 2 presents. Figure 8A shows a reference relationship in the moving image display order as Figure 7 shows and Figure 8B shows an order in which a bit stream is entered, that is, a decoding order. Here, an image P23 indicates an image decoded in direct mode and an image P24 indicates an image whose motion vector refers to decoding.

When considering an order as shown in Figure 8B, since a motion vector referring to an image to be decoded previously is directly used, a motion vector MV22 and a time interval TR22 are directly applied as a motion vector. MV_REF and as a timeframe TR REF as Equation 5 presents. Likewise, it is possible to directly use a motion vector that refers to an image to be decoded later. In this case, a motion vector MV21 and a time interval TR21 are directly applied as a motion vector MV_REF and as a time interval TR_REF as Equation 6 presents. This makes it possible to reduce the capacity of a motion vector storage unit in a decoder as long as each block belonging to a P24 image, said motion vector can perform motion compensation by storing only one of two motion vectors. .

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by adjusting the scale of a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images. (Fourth Modality) An overview of the coding process based on Figure 11 is totally the same as the third modality. Here, a detailed operation of the dual mode direct prognosis is explained using Figure 9. However, it is assumed that the stream and bits generated in the image coding method of the first embodiment are informed.

[00140] Figure 9 shows an operation when an image referenced to determine a direct mode motion vector contains two motion vectors which refer to two subsequent images in the order of display. A P43 image is a current image to be decoded and performs dual prediction referring to images P42 and P44. When it is assumed that a block to be decoded is an MB41 block, two required motion vectors are determined using a motion vector from a block of the same location MV42 in the decoded back reference image (the second reference image specified by the second index). reference number) P44. Since block MB42 contains two motion vectors MV45 and MV46 as motion vectors, the two motion vectors MV43 and MV44 cannot be calculated by directly scaling similarly to Equation 1. Therefore, as Equation 7, An MV REF motion vector is determined as a motion vector scaled from an average value of two vectors and MB42 block motion, and a TRREF time interval is then also determined from an average value. Then the motion vectors MV43 and MV44 are calculated by scaling a motion vector MV_REF and a TR REF time interval based on Equation 8. In this case, a TR45 time interval indicates a time interval between a image P44 and P45, i.e. even an image that is referred to by a motion vector MV45; and time interval TR46 indicates a time interval to an image that is referred to by a motion vector MV46. A time interval TR43 indicates a time interval up to an image that is referred to by a motion vector MV43; and a time interval TR44 indicates a time interval up to an image that is referred to by a motion vector MV44. Note that a current image to be decoded refers to a next image in the example of Figure 9, however, the case referring to an image that is not adjacent can be treated in the same way.

[00141] The above embodiment presents the decoding method in which a decoding by interimage prognosis can be performed using direct mode without contradiction even if a block whose said direct mode motion vector belongs to an image B. decoding method, when a block whose said direct mode motion vector contains several motion vectors that refer to a following image in the order of display, a motion vector is generated using the various motion vectors and two Motion vectors to be used for actual motion compensation are determined by adjusting the scale.

Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, it is possible to use Equation 7 instead of Equation 9 as a method for averaging motion vectors MV45 and MV46 and for averaging time intervals TR45 and TR46 to calculate motion vector MV REF and time interval TR_REF to scale. First, as Equation 9 (a) presents, motion vector MV46 'is calculated by adjusting the scale of MV46 to match the time interval with motion vector MV45. Then the motion vector MV_REF is determined by averaging the vectors MV46 'and MV45. Here the TR45 time slot is used directly as the TR RF time slot. Note that the case of calculating an MV45 'motion vector by adjusting the scale of the MV45 motion vector instead of calculating the MV46' motion vector by adjusting the scale of an MV46 motion vector can be treated in the same way.

Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, as a motion vector MV_REF and as a time interval TR_REF that have the scaled scale, a motion vector MV45 and a time interval TR45, which refer to an image P45 located temporarily closest to an image P44 whose said motion vector can be directly used, as equation 10 presents, rather than using an average value of two motion vectors as equation 7 presents. Likewise, as a motion vector MV_REF and as a time interval TR_REF, a motion vector MV46 and a time interval TR46, which all refer to a more distant, temporarily located image P46, can be directly used as the Equation. 11 presents. This method makes it possible to reduce the capacity of a motion vector storage unit in a decoder as long as each block belonging to a P44 image, said motion vector can implement motion compensation by storing only one of two motion vectors. movement.

Note that when two motion vectors MV43 and MV44 in figure 9 are calculated, as a motion vector MV_REF and as a time interval TR_REF that have the scaled, a motion vector that refers to an image to being decoded earlier can be directly used, rather than using an average value of two motion vectors as Equation 7 presents. Figure 10A shows a reference relationship in the moving image display order as Figure 9 shows and Figure 10B shows an order in which a bit stream is informed, that is, a decoding order. Here, an image P43 indicates an image that is decoded in direct mode and an image P44 indicates an image whose motion vector refers to decoding. When considering an order as shown in Figure 10B, since a motion vector referring to an image to be decoded previously is directly used, a motion vector MV46 and a time interval TR46 are directly applied as a motion vector. MV_REF and as a time interval TR_REF as Equation 10 presents. This method makes it possible to reduce the capacity of a motion vector storage unit in a decoder as long as each block belonging to a P44 image, said motion vector can perform motion compensation by storing only one of two motion vectors. movement.

Note that when a block that is referenced to determine a motion vector in direct mode contains two motion vectors that refer to the next two images in the order of display, it is possible to perform motion compensation by assuming that two Required motion vectors MV43 and MV44 are "0". This method makes it possible to reduce the capacity of a motion vector storage unit in a decoding apparatus and further make it possible to omit a process for calculating a motion vector provided that each block belonging to a P44 image, whose motion vector is not have to store a motion vector.

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by adjusting the scaling of a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying by a constant number. Here a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images (Fifth Modality) [00147] The encoding / decoding method can be performed not only by the method encoding / decoding presented in the first mode up to the fourth mode above, but also by a motion vector calculation method shown below.

[00148] Figure 12 shows an operation when a coded or decoded block referenced to calculate a direct mode motion vector contains two motion vectors which refer to the two preceding images in the order of display. A P23 image is a current image to be encoded or decoded. When it is assumed that a block to be encoded or decoded is an MB1 block, two required motion vectors are determined using a motion vector of a block of the same location MB2 in the encoded or decoded back reference image (the second reference image specified by second reference index) P24. Note that in Figure 12, MB1 block is a process current block, MB1 and MB2 blocks are in the same location in other images, and MV21 motion vectors are the first forward motion vectors that the reference image is specified by first reference index and MV22 is the backward motion vector that the reference image is specified by the second reference index and these motion vectors are used for encoding or decoding block MB2 and refer to images P21 and P22, respectively. P21, P22 and P24 images are encoded or decoded images. A time interval TR21 is a time interval between picture P21 and picture P24; a time interval TR22 is a time interval between picture P22 and picture P24; a time interval TR21 'is a time interval between P21 and picture P23; and a time interval TR24 'is a time interval between picture P23 and picture P24.

In the motion vector calculation method, as shown in Fig. 12, only the forward motion vector (the first motion vector) previously encoded or decoded MV21 is used between the motion vectors of block MB2 in the reference image P24 and motion vectors MV21 'and MV24' of block MB1 are calculated by the following equations.

[00150] MV21 '= MV21 x TR217TR21 [00151] MV24' = -MV21 x TR247TR21 [00152] Then the double prognosis is performed from images P21 and P24 using motion vectors MV21 'and MV24'. Note that a motion vector of block MB1 can be calculated using only one motion vector (the second motion vector) MV22 encoded or decoded later between motion vectors of block MB2 in reference image P24, rather than calculating vectors MB21 'and MV24' motion block using only the MV21 motion vector. In addition, as shown in the first embodiment to the fourth embodiment, a motion vector of block MB1 can be determined using both motion vectors MV21 and MV22. When selecting one of the motion vectors MV21 and MV22, a motion vector of a previously encoded or decoded block may be selected and it may be arbitrarily set in a coding apparatus and a decoding apparatus. Motion compensation is possible when image P21 is in short-term reference image buffer or long-term reference image buffer. Explanation will be given to the short term reference image buffer and to the long term reference image buffer later.

[00153] Figure 13 shows an operation when a coded or decoded block referenced to calculate a direct mode motion vector contains two motion vectors which refer to two subsequent images in the order of display. A P22 image is a current image to be encoded or decoded. When it is assumed that a block to be encoded or decoded is an MB1 block, two required motion vectors are determined using a motion vector from a block of the same location MB2 in the encoded or decoded back reference image (the second reference image ) P23. Note that in Figure 13, block MB1 is a current processing block, blocks MB1 and MB2 are blocks located at the same location in images, and motion vectors MV24 and MV25 are backward motion vectors used for encoding or decoding the MB2 block and refer to images P21 and P22 respectively. P21, P23, P24 and P25 images are encoded or decoded images. A time interval TR24 is a time interval between image P23 and image P24, a time interval TR25 is a time interval between image P23 and image P25, a time interval TR24 'is a time interval between P22 and image P24 and a time interval TR21 'is a time interval between image P21 and image P22.

In a motion vector calculation method as shown in Fig. 13, only the backward motion vector MV24, which refers to image P24 of block MB2 in reference image P23, is used and Motion vectors MV21 'and MV24' are calculated by the following equations. MV21 '= -MV24 x TR217TR24 MV24' = MV24 x TR247TR24 [00155] Then the double prognosis is performed from images P21 and P24 using motion vectors MV21 'and MV24'.

Note that as shown in Fig. 14, when only a backward motion vector MV25, which points to image P25 of block MB2 in reference image P23, motion vectors MV21 'and MV24 are used. 'are calculated by the following equations. Here, a time interval TR24 is a time interval between picture P23 and picture P24; a time interval TR25 is a time interval between picture P23 and picture P25; a time interval TR25 'is a time interval between picture P22 and picture P25; and a time interval TR21 'is a time interval between picture P21 and picture P22. MV21 '= -MV25 x TR217TR25 MV25' = MV25 x TR257TR25 [00157] Then the double prognosis is performed from images P21 and P24 using motion vectors MV21 'and MV24'.

Fig. 15 shows an operation when a coded or decoded block referenced to calculate a direct mode motion vector contains two motion vectors that refer to a preceding image in the display order. A P23 image is a current image to be encoded or decoded. When it is assumed that a block to be encoded or decoded is an MB1 block, two required motion vectors are determined using a motion vector from a block of the same location MB2 in the encoded or decoded back reference image (the second reference image specified by the second reference index) P24. Note that in figure 15, block MB1 is a current processing block, blocks MB1 and MB2 are blocks located at the same location in the other images. Motion vectors MV21A and MV21B are forward motion vectors used for MB2 block encoding or decoding and both refer to image P21. P21, P22 and P24 images are encoded or decoded images. Time intervals TR21A and TR21B are a time interval between picture P21 and picture P24; a time interval TR21 'is a time interval between picture P21 and picture P23; and a time interval TR24 'is a time interval between P23 and image P24.

[00159] In a motion vector calculation method as shown in Figure 15, only the forward motion vector MV21A, which points to image P21 of block MB2 in reference image P24, is used and the vectors MV21A 'and MV24' are calculated by the following equations.

MV21A '= M V21A x TR217TR21A MV24' = -MV21A x TR247TR21A

Then, the double prognosis is performed from images P21 and P24 using motion vectors MV21A 'and MV24'.

Note that a motion vector of block MB1 can be calculated using only a forward motion vector MV21B, which points to image P21 of block MB2 in reference image P24. In addition, as shown in the first embodiment to the fourth embodiment, a motion vector of block MB1 can be determined using both forward motion vectors MV21A and MV21B. When selecting one of the forward motion vectors MV21A and MV21B, a motion vector of a previously encoded or decoded block (described above in a bit stream) can be selected and it can be arbitrarily set by a coding apparatus and by a decoder. Here, the previously encoded or decoded motion vector means the first motion vector. Motion compensation is possible when image P21 in the short-term reference image buffer or long-term reference image buffer. Explanation will be given to short-term reference image buffer and to long-term reference image buffer later.

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by scaling a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images.

Note that in the equations mentioned above for calculating motion vectors MV21 ', MV24', MV25 'and MV21A', motion vectors can be rounded to a predetermined precision of a motion vector after calculating the right side of the motion. equations. Motion vector accuracy includes half-pixel precision, one-third pixel accuracy, and one-quarter pixel accuracy or the like. In addition, the accuracy of a motion vector can be determined, for example, block-based, image-based, and sequence-based. (Sixth Modality) In this sixth embodiment, a method for calculating a current motion vector by scaling only one of two forward motion vectors, which refer to two preceding images in the display order, is 14, 15 and 16. In this case, a reference image used to determine a current motion vector in direct mode contains the two forward motion vectors. Note that block MB1 is a current block to be processed; MB1 and MB2 blocks are blocks in the same place in other images; and motion vectors MV21 and MV22 are forward motion vectors used for MB2 block encoding and decoding and refer to images P21 and Ρ22, respectively. P21, P22 and P24 images are encoded or decoded images. A time interval TR21 is a time interval between picture P21 and picture P24; a time interval TR22 is a time interval between picture P22 and picture P24; a time interval TR21 'is a time interval between P21 and picture P23; and a time interval TR22 'is a time interval between P22 and picture P23.

As for the first method, when an MB2 block in a reference image P24 contains the forward motion vector MV21 referring to the image P22 and a forward motion vector MV22 referring to the image P23 as shown in Figure 16, a motion vector MV22 'of block MB1 is calculated using only one motion vector MV22 which refers to an image P22 next to a current image P23 in the order of display by a following equation. MV22 '= MV22 x TR227TR22 [00166] Then motion compensation is performed from image P22 using motion vector MV22'.

As the second method, when an MB2 block of a reference image P24 contains a forward motion vector MV21 referring to an image P21 and a forward motion vector MV22 referring to an image P22 as shown. In Figure 17, a motion vector MV21 'of block MB1 is calculated using only one motion vector MV21 which refers to an image P21 being far from a current image P23 in the order of display by a following equation. MV21 '= MV21 x TR217TR21 [00168] Then motion compensation is performed from image P21 using motion vector MV21'.

[00169] The first and second methods make it possible to reduce the capacity of a motion vector storage unit as long as the MB2 block belonging to an image P24, said motion vector of which can perform motion compensation by storing only one of Two motion vectors.

Note that motion compensation can be performed from a nearby image P22 in the order of display using the forward motion vector MV21 in the same way as in the first embodiment. An MVN motion vector (not shown in this figure) used for motion compensation is calculated by a following equation. MVN = MV21 x TR227TR21 As the third method, as shown in Figure 18, a motion compensation block is obtained from images P21 and P22 respectively using the motion vectors MV21 'and MV22' calculated above and an image Average is used as a tween image in motion compensation.

[00172] The third method increases the calculated amount, however, it improves the accuracy of motion compensation.

In addition, it is possible to obtain a motion compensation block from image P22 using the motion vectors mentioned above MVN and MV22 'and to use an average image as an interpolation image in motion compensation.

Note that in this embodiment, the case where a motion vector used in direct mode is calculated by adjusting the scale of a referenced motion vector using a time interval between images is explained, however, the motion vector can be calculated by multiplying a reference motion vector by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images.

Note that in the equations mentioned above for calculating motion vectors MV21 ', MV22' and MVN, motion vectors can be rounded to a predetermined precision of a motion vector after calculating the right side of the equations. Motion vector accuracy includes half-pixel precision, one-third pixel accuracy, and one-quarter pixel accuracy or the like. In addition, the accuracy of a motion vector can be determined, for example, block-based, image-based, and sequence-based. (Seventh Modality) In the sixth embodiment above, the case when a reference image used to determine a motion vector of a current block to be encoded or decoded contains two forward motion vectors in direct mode is described. The two forward motion vectors refer to the two preceding images in the display order. However, when the reference image contains two backward motion vectors that refer to the next two images in the display order, it is likewise possible to calculate a current motion vector by scaling only one of the two motion vectors. backward (the second motion vectors whose reference image is specified by the second reference indices). Explanation will be given using figures 17 to 20 below. Note that MB1 block is a current process block, MB1 and MB2 blocks are blocks of the same location in other images, and motion vectors MV24 and MV25 are backward motion vectors (the second motion vectors whose reference image is specified by the second reference indices) used for MB2 block encoding or decoding. P21, P23, P24 and P25 images are encoded or decoded images. A time interval TR24 is a time interval between picture P23 and picture P24; a time interval TR25 is a time interval between picture P23 and picture P25; a time interval TR24 'is a time interval between P22 and picture P25; and a time interval TR25 'is a time interval between P22 and image P25.

As the first method, when an MB2 block of a reference image P23 contains two backward motion vectors MV24 referring to an image P24 and MV25 referring to an image P25 as shown in Figure 19, a vector MV24 'of the MB1 block is calculated using only one backward motion vector MV24 which refers to a picture P24 being temporarily close to a current picture P22 by a following equation. MV24 '= MV24 x TR247TR24 [00178] Then motion compensation is performed from image P24 using motion vector MV24'.

Note that motion compensation can be performed from a nearby image P23 in the display order using a backward motion vector MV24 in the same manner as in the first embodiment. An MVN1 motion vector (not shown in this figure) used for motion compensation is calculated by a following equation. MVN1 = MV24 x TRN1 / TR24 As with the second method, when an MB2 block of a reference image P23 contains two backward motion vectors MV24 referring to an image P24 and MV25 referring to an image P25 as shown in Fig. 20, a motion vector MV25 'of block MB1 is calculated using only one backward motion vector MV25 which refers to a picture P25 distant from a current picture P23 in the order of display by a following equation. MV25 '= MV25 x TR257TR25 [00181] Then motion compensation is performed from image P25 using motion vector MV25'.

The first and second methods make it possible to reduce the capacity of a motion vector storage unit since the MB2 block belonging to a P23 image whose motion vector is referred to can implement motion compensation by storing only one of Two motion vectors.

Note that motion compensation can be performed from a nearby image P23 in the display order, using a vector and backward motion MV25 in the same way as in the first mode. An MVN2 motion vector (not shown in this figure) used for motion compensation is calculated using a following equation. MVN2 = MV25 x TRN1 / TR25 As the third method, as shown in Figure 21, a motion compensation block is obtained from images P24 and P25, respectively, using the calculated motion vectors MV24 'and MV25'. above and an average image is used as an interpolation image in a motion compensation.

[00185] The third method increases the amount of calculation, however, it improves the accuracy of motion compensation.

Note that it is possible to obtain a motion compensation block from image P24 using the motion vectors MVN1 and MVN2 mentioned above and use an average image as an interpolation image in motion compensation.

Furthermore, as shown in Fig. 22, when a reference image referenced to determine a motion vector of a direct motion current motion vector contains a backward motion vector that refers to a following image. In display order, for example, a motion vector MV24 'is calculated using a following equation. MV24 '= MV24 x TR247TR24 [00188] Then motion compensation is performed from image P24 using motion vector MV24'.

Note that motion compensation can be performed from a nearby image P23 in the display order using a backward motion vector MV25 in the same manner as in the first embodiment. An MVN3 motion vector (not shown in this figure) used for motion compensation is calculated by a following equation. MVN3 = MV24 x TRN1 / TR24 [00190] Note that in this embodiment, the case when a current motion vector is calculated by scaling the backward motion vector is explained when containing two backward motion vectors, which are refer to the next two images in the display order and when containing a backward motion vector, which refers to a following image in the display order. However, a current motion vector can be calculated by referring to a motion vector of a neighboring block in the same image without using a backward motion vector and when intra-image coding is performed, a current motion vector can be calculated by referring to a motion vector of a neighboring block in the same image.

Firstly, the first calculation method will be described.

[00192] Figure 23 shows a position relationship between a motion vehicle to be referred to and a current block. An MB1 block is a current block and refers to a motion vector of a block including three pixels located at A, B, and C. Note that when a pixel C cannot be referenced since it is outside a frame or it was not encoded / decoded, a motion vector of a block including a pixel D is used instead of a block including pixel C. By calculating an average value of the motion vectors of three current blocks including pixels A, B and C to be referred to, a motion vector actually used in direct mode is determined. By calculating an average value of the motion vectors of the three blocks, additional information showing which motion vector is selected need not be described in a bit stream. Accordingly, it is possible to obtain a motion vector expressing the motion close to the actual motion of block MB1. In this case, motion compensation can be performed by forward reference only (reference to first reference image) using the determined motion vector and by bidirectional reference (reference to first reference image and second reference image) using a vector of parallel motion with the determined motion vector.

Next, the second calculation method will be described.

Under the second calculation method, an average value is not selected as in the first calculation method and a motion vector used in real direct mode is determined by selecting a motion vector whose coding efficiency is the highest. of the motion vectors of the three blocks including pixels A, B and C. In this case, motion compensation can be performed by forward reference only (reference to the first reference image) using the determining motion vector and by bidirectional reference. (reference to first reference image and second reference image) using a motion vector parallel to the determined motion vector. Information indicating a motion vector with the highest coding efficiency, for example, is as shown in Fig. 24A, added to a block header area in a bit stream generated by a bit stream generating unit 103. with information indicating a direct mode output from a mode selection unit 107. Note that as shown in Fig. 24B, information indicating a motion vector with the highest encoding efficiency may be added to a header area of A croblock. Here, the information indicating a motion vector with the highest coding efficiency is, for example, a number identifying a block including a current pixel to be referred to and a given identification number for each block. When a block is identified by the identification number, a motion vector with the highest efficiency may be indicated by using only one of the motion vectors used to encode a block corresponding to an identification number and when the motion vectors are more than 1, a motion vector with the highest coding efficiency may be indicated by using multiple motion vectors. Or, a motion vector with the highest efficiency may be indicated by using an identification number given for each block for each motion vector in the double direction (reference to first reference image and second reference image). This selection method always makes it possible to select a motion vector that makes coding efficiency higher. However, since additional information showing which motion vector is selected needs to be described in a bitstream, extra amount of code for the additional information is required. In addition, the third method of calculation is explained.

Under the third method of calculation, a motion vector referring to a reference image with the lowest reference index is determined as a motion vector used in a real direct mode. The lowest reference index generally means a motion vector that refers to a close image in the display order or a motion vector with the highest encoding efficiency. Therefore, this motion vector selection method makes it possible to improve coding efficiency, since a motion vector used in direct mode is generated using a motion vector that refers to a closer image in the display order or a motion vector with the highest coding efficiency.

Note that as all three motion vectors refer to the same reference image, an average value of the three motion vectors can be used. On the other hand, when two of the three motion vectors refer to a reference image with the smallest reference index value, for example, the same of the two motion vectors can always be selected. As an example, referring to Fig. 23, there are three blocks including pixels A, B and C respectively and when the reference index values of blocks including pixels A and B are the smallest and the same reference image is referred to, a motion vector in the block including pixel A may be selected. However, when the reference index values of blocks including pixels A and C are the smallest and the same reference image is referred to, a motion vector in a BL1 block including pixel A located closest to a block may be selected. .

Note that the mean value mentioned above can be an average value of components in the horizontal and vertical direction of each motion vector and can be an average value of the value (absolute value) of each motion vector.

In the case as shown in Figure 25, an average value of motion vectors may be an average value of motion vectors contained in 5 blocks: a block located of a BL1 block in a following reference image; blocks including pixels A, B and C, respectively; +++ is a block including a pixel D shown in Figure 25. As described above, when a block of the same location, which is close to a current pixel to be encoded, of block BL1 in a following reference image is used, A process for calculating an average value of motion vectors is made easier by using a block including pixel D to make the number of blocks an odd number. Note that when multiple blocks are in an area of the same location as block BL1 in a following reference image, motion compensation can be performed for block BL1 by using a motion vector in a block that occupies the largest overlapping area with block BL1, or by dividing block BL1 corresponding to the area of the various blocks in the following reference image and motion compensation can be performed based on the split block.

[00199] An explanation will be additionally given using concrete examples.

As shown in Figures 26 and 27, when all blocks including pixels A, B and C contain a motion vector that refers to an image preceding a current image to be encoded, any one of the first to The third calculation methods mentioned above can be used.

Likewise, as shown in Figures 28 and 29, when all blocks including pixels A, B and C contain a motion vector that refers to an image following a current image to be encoded, either from the first to the third calculation methods can be used.

In the following, the case shown in figure 30 is explained. Figure 30 shows the case where each of the blocks including pixels A, B and C respectively contain each of the motion vectors, one refers to the image preceding a current image to be encoded and another refers to a image following the current image to be encoded.

According to the first calculation method, a forward motion vector used for BL1 motion compensation is selected by an average value of the MVAf, MVBf, and MVCf vectors and a backward motion vector used for compensation. Block BL1 is selected by an average value of the vectors MVAb, MVBb and MVCb. Here, the motion vector MVAf is a forward motion vector of a block containing pixel A, the motion vector MVAb is a backward motion vector of a block containing pixel A, the motion vector MVBf is a vector forward motion of a block containing pixel B, the motion vector MVBb is a backward motion vector of a block containing pixel B, the motion vector MVCf is a forward motion vector of a block containing pixel C, motion vector MVCb is a backward motion vector of a block containing pixel C. Motion vectors such as motion vector MVAf are not limited to the case by referring to an image as shown in the figure. The same applies to the following explanation.

According to the second calculation method, a motion vector to be used in real direct mode is determined by selecting a motion vector with the highest coding efficiency of the forward reference motion vectors MVAf, MVBf. and MVCf and by selecting a motion vector with the highest coding efficiency among the back reference motion vectors MVAb, MVBb and MVCb. In this case, motion compensation can be performed by forward reference only using a vector with the highest coding efficiency among the forward reference motion vectors MVAf, MVBf, and MVCf and by double prediction using a parallel motion vector. Determined motion vector. Note that in order to achieve the highest coding efficiency, motion compensation can be performed by selecting a block and using the forward and backward reference motion vectors of the selected block instead of selecting against the vectors. forward and backward reference movements respectively. In this case, as long as the information indicating a selection can be reduced compared to the case where selecting information indicating a block containing a pixel having a selected forward motion vector with respect to the highest coding efficiency and a block containing contains a pixel having a selected backward motion vector with respect to the highest coding efficiency, the coding efficiency may be improved. The block selection can be from the following: 1. A block includes a pixel having a forward reference motion vector that refers to a reference image with the lowest reference index value; 2. A block has the smallest value which is a sum of a reference index value of an image referred to by a forward reference motion vector of a block including each pixel and a reference index value of an image referred to by a reference motion vector behind a block including each pixel; 3. A block selects an average reference index value from an image referred to by a forward reference motion vector and includes a pixel having a forward reference motion vector with the selected average value and a backward motion vector. is included in the block; and 4. A block selects an average reference index value in an image referred to by a backward reference motion vector and includes a pixel having a backward motion vector with the selected mean value and a forward motion vector. included in the block. Note that when each of the backward motion vectors refers to the same image, the selection of method 1 and method 3 is appropriate.

According to the third method of calculation, one of the forward reference motion vectors MVAf, MVBf and MVCf, which refers to a reference image with the lowest reference index value, is a vector of forward reference movement (the first reference) used in direct mode. Or, one of the back reference motion vectors MVAb, MVBb and MVCb, which refers to a reference image with the lowest reference index value, is a backward reference motion vector (the second reference). used in direct mode. Note that in the third calculation method, the forward motion vector referring to the reference image with the lowest reference index is a forward motion vector of a BL1 block and the backward motion vector referring to the The reference image with the lowest reference index is a backward motion vector of block BL1, however, two motion vectors BL1 and BL2 can be derived using one of a forward motion vector or a backward motion vector. referring to a reference image with the lowest reference index and motion compensation can be performed using the derived motion vector.

Next, the case shown in Figure 31 is explained. Fig. 31 presents a case where pixel A contains each of the motion vectors, one refers to the preceding image and the other to a following image, pixel B contains only one motion vector to refers to a preceding image and pixel C contains only one motion vector that refers to a following image.

Thus, when there is a block including a pixel containing only one motion vector referring to the image unidirectionally, assuming that a motion vector referring to an image in another direction is 0, the calculation method in the Figure 30 mentioned above can be used for motion compensation. Specifically, using the first or third calculation method in figure 30, the calculation can be performed by assuming MVCf = MVBb = 0. That is, in the first calculation method, when a forward motion vector of block BL1 is calculated, assuming whereas a motion vector MVCf of pixel C referring to a preceding image is 0, an average value of motion vectors MVAf, MVBf and MVCf is calculated. On the other hand, when a backward motion vector of block BL1 is calculated, assuming a motion vector MVBb, which refers to a following image, of pixel B is 0, an average value of motion vectors MVAb, MVBb and MVCb is calculated.

According to the third method of calculation, assuming a motion vector MVCf, which refers to a preceding image, of pixel C and a motion vector, which refers to a following image, of the pixel B is 0, a motion vector, which refers to a reference image with the lowest reference index value, from block BL1 is calculated. For example, when a block including a pixel A refers to an image with the first benchmark "0" and a block including a pixel B refers to an image with the first benchmark of "1", the lowest value of the benchmark is "0". Therefore, since only the motion vector MVBf, which refers to a preceding image, the block including the pixel refers to an image with the lowest first reference index, the motion vector MVBf is selected as a vector. forward movement of block BL1. In addition, for example, when both pixels A and C refer to a following image with the lowest second reference index, for example "0", assuming a motion vector MVBb, which refers to a following image. , of a pixel B is 0, an average value of the motion vectors MVAb, MVBb and MVCb is calculated. The motion vector resulting from the calculation is established as a forward motion vector of block BL1.

In the following, a case shown in Fig. 32 is explained. Fig. 32 presents a case when pixel A contains each of the motion vectors; one refers to the preceding image and another refers to a following image. Pixel B contains only a motion vector that refers to a preceding image and pixel C does not contain a motion vector and is encoded within the image.

When a block including a current pixel C to be encoded is intra-image encoded, assuming that the motion vectors referring to the preceding and following images of the block are both 0, the calculation method in figure 30 mentioned above may be used for motion compensation. Specifically, the calculation can be performed assuming MVCf = MVCb = 0. Note that in Figure 30, MVBb is 0.

Finally, a case shown in Fig. 33 is explained. Fig. 33 presents a case where a pixel C is encoded by the direct mode.

When blocks including a referenced current pixel contain a direct mode encoded block, motion compensation of the BL1 block may be performed using a motion vector used to encode a direct mode encoded block and using the calculation method shown herein. in figure 30.

Note that it is determined which of a forward or backward reference motion vector is used in relation to an image to be referred to, an image to be encoded, and the time information included in each image. Therefore, when a motion vector is derived after differentiating between a forward reference and a back reference, a motion vector of each block is judged whether it is a forward reference or a back reference from the time information contained in. each picture.

In addition, an example of a calculation method combining the calculation methods mentioned above is explained. Fig. 34 is an illustration showing a procedure for determining a motion vector to be used in direct mode. Fig. 34 is an example of a method for determining a motion vector using reference indices. Note that RidxO and Ridxl presented in figure 34 are reference indices explained above. Fig. 34A shows a procedure for determining a motion vector using the first reference index RidxO and Fig. 34B shows a procedure for determining a motion vector using the second reference index Ridx1. First, figure 34A is explained.

In step S3701, there are three blocks including pixels A, B and C respectively and the number of blocks referring to an image using the first reference index RidxO is calculated.

When the number of blocks calculated in step S3701 is "0", the number of blocks referring to an image using the second reference index Ridxl is further calculated in one step S3702. When the number of blocks calculated in step S3702 is "0", motion compensation is performed bidirectionally for a current block to be coded assuming that a motion block of the current block to be coded is "0" in a step S3703. On the other hand, when the number of blocks calculated in step S3702 is "1" or more, a motion vector of a current block to be encoded is determined in step S3704 by the number and blocks containing the second reference index Ridxl. For example, motion compensation of a current block to be encoded is performed using the motion vector determined by the number of blocks containing the second reference index Ridx1.

When the number of blocks calculated in step S3701 is "1", a motion vector containing the first reference index RidxO is used in a step S3705.

When the number of blocks calculated in step S3701 is "2", a motion vector corresponding to an average value of three motion vectors is used in a step S3706, assuming that a block that does not contain the first reference index. RidxO contains a motion vector of MV = 0 from the first RidxO reference index.

When the number of blocks calculated in step S3701 is "3", a motion vector corresponding to an average value of three motion vectors is used in one step S3707. Note that motion compensation in step S3704 can be performed bidirectionally using a motion vector. Here, bidirectional motion compensation can be performed after calculating a motion vector in the same direction as a motion vector and a motion vector in the opposite direction to the first motion vector, for example by scaling the first motion vector , or can be performed using a motion vector in the same direction as the first motion vector and a motion vector of "0". Next, figure 34B is explained.

[00220] The number of blocks containing the second reference index Ridxl is calculated in one step S3711.

When the number of blocks calculated in step S3711 is "0", the number of blocks containing the first reference index Ridxl is additionally calculated in one step S3712. When the number of blocks calculated in step S3712 is "0", motion compensation is performed bidirectionally for a current block to be coded assuming that a motion block of the current block to be coded is "0" in a step S3713 . On the other hand, the number of blocks calculated in step S3712 is "1" or more, a motion vector of a current block to be encoded is determined in one step S3714 by the number of blocks containing the first reference index RidxO. For example, motion compensation of a current block to be encoded is performed using the motion vector determined by the number of blocks containing the first reference index RidxO.

When the number of blocks calculated in step S3711 is "1", a motion vector containing the second reference index Ridxl is used in a step S3715.

When the number of blocks calculated in step S3711 is "2", a motion vector corresponding to an average value of three motion vectors is used in one step S3716, assuming that a block that does not contain the second reference index Ridxl contains a motion vector of MV = 0 from the second reference index Ridxl.

When the number of blocks calculated in step S3711 is "3", a motion vector corresponding to an average value of three motion vectors is used in one step S3717. Note that motion compensation in step S3714 can be performed bidirectionally using a motion vector. Here, bidirectional motion compensation can be performed after calculating a motion vector in the same direction as a first motion vector and a motion vector in the opposite direction to the first motion vector, for example by adjusting the scale of the first vector. motion, or can be performed using a motion vector in the same direction as the first motion vector and a motion vector of "0".

Note that Figs 34A and 34B are respectively explained, but both methods may be used or one of these methods may be used. However, when one of these methods is used, for example, when a process starting from step 3704 shown in figure 34A is used and a process up to step S3704 is used, a process after step S3711 shown in figure 34B may be used. . When a process up to step S3704 is used, provided that a process after step S3712 is not used, a motion vector can be uniquely determined. When both processes of FIGS. 34A and 34B are used, either process may be used first, or both processes may be used together. When a block in the vicinity of a current block to be encoded is encoded in direct mode, a reference index referred to by a motion vector used to encode the encoded block in direct mode may be assumed to be contained in a block encoded in direct mode. and located in the vicinity of a current block to be encoded.

[00226] The detailed explanation of the method for determining a motion vector will be given below using concrete examples of a block. Fig. 35 is an illustration showing motion vector types contained in each block referred to by a current block BL1 to be encoded. In Fig. 35A, a block containing a pixel A is an intra-image coded block and a block containing a pixel B includes a motion vector and motion compensation is performed for the block using a motion vector and a block containing a pixel C is a block including two motion vectors and motion compensation is performed bidirectionally. The block containing pixel B contains a motion vector indicated by the second reference index Ridxl. Since the block containing pixel A is a block to be encoded within the image, it does not contain a motion vector. In other words, it also does not contain a benchmark.

In step S3701, the number of blocks containing the first reference index RidxO is calculated. As shown in Fig. 35, since the number of blocks containing the first RidxO benchmark is 2, assuming that a block that does not contain the first RidxO benchmark contains a motion vector MV = 0 of the first RidxO benchmark, A motion vector corresponding to an average value of three motion vectors is used in one step S3706. Bidirectional motion compensation can be performed for a current block to be encoded using only the above motion vector, or it can be performed using the second reference index Ridxl and another motion vector as shown below.

[00228] In step S3711, the number of blocks containing the second reference index Ridxl is calculated. As shown in Fig. 35, since the number of blocks containing the second reference index Ridxl is 1, a motion vector containing the second reference index Ridxl is used in step S3715.

In addition, an example of another calculation method combining the calculation methods mentioned above is explained. Fig. 38 is an illustration showing a procedure for determining a motion vector of a current block to be encoded using reference index values showing an image referred to by a motion vector contained in blocks including pixels A, B and C, respectively. Figures 36A and 36B are illustrations showing a procedure for determining a motion vector based on the first reference index RidxO and figures 36C and 36D are illustrations showing a procedure for determining a motion vector based on the second reference index RidxO. Figure 36A shows a procedure based on the first RidxO benchmark, on the other hand Figure 36C shows a procedure based on the second RidxO benchmark. Figure 36B shows a procedure based on the first RidxO benchmark, on the other hand Figure 36D shows a procedure based on the second RidxO benchmark. Accordingly, only figures 36A and 36B are explained below. To begin with, figure 36A is explained.

In step S3801, it is judged whether the smallest first RidxO benchmark of the first effective RidxO benchmarks can be selected.

When it is possible to select the lowest first RidxO reference index from the first effective RidxO reference indices, the motion vector selected in step S3802 is used.

When the first effective RidxO reference indices include the several lowest RidxO reference indices in step S3801, a motion vector contained in a block selected by priority in a step S3803 is used. Here, priority, for example, determines a motion vector to be used for motion compensation of a current block to be encoded in alphabetical order of pixels contained in the blocks.

[00233] When there is no first effective RidxO benchmark in step S3801, the process that is different from steps S3802 and S3803 is used in step S3804. For example, a process following a step S3711 explained in figure 374B may be used. Next, figure 36B is explained. The different point between figures 36A and 36B is that a process in steps S3803 and S3804 in figure 36A is changed to a step S3813 in figure 36B.

In step S3811, it is judged whether the lowest first RidxO benchmark of the first effective RidxO benchmarks can be selected.

When it is possible to select the lowest first RidxO reference index from the first effective RidxO reference indices, the motion vector selected in step S3812 is used.

[00236] When there is no first effective reference index RidxO in step S3811, the process that is different from S3812 is used in step S3813. For example, a process following a step S3711 explained in figure 374B may be used.

Note that the first effective RidxO reference index is indicated by "O" in Figure 35B and is a reference index showing having a motion vector. In Figure 35B, the places where "x" is written indicates that reference indices are not designated. In a step S3824 in figure 36C and a step S3833 in figure 36D, a process after step S3701 explained in figure 34A can be used.

[00238] The detailed explanation of the method for determining a motion vector will be given below using concrete examples of a block and Figure 35.

In step S3801, it is judged whether the lowest first RidxO benchmark of the first effective RidxO benchmarks can be selected.

In the case presented in figure 35 there are two first effective RidxO benchmarks, however, when it is possible to select the lowest first RidxO benchmark from the first effective RidxO benchmarks in step S3801, a motion vector selected in step S3802 is used.

When the first effective RidxO reference indices include the several lowest RidxO reference indices in step S3801, a motion vector contained in a block selected by priority in a step S3803 is used. Here, priority, for example, determines a motion vector to be used for motion compensation of a current block to be encoded in alphabetical order of pixels contained in the blocks. When blocks including pixels B and C respectively contain the same first RidxO reference index, the first RidxO reference index of the block including pixel B is employed by priority and motion compensation is performed for a current block BL1 to be encoded. using a motion vector corresponding to the first RidxO reference index of the block containing pixel B. In this case, motion compensation may be performed for the current block BL1 to be bidirectionally encoded using only the determined motion vector, or it may be be performed using the second reference index Ridxl and another motion vector as shown below.

In a step S3821, it is judged whether the smallest second Ridxl reference index of the second effective Ridxl reference indices can be selected.

In the case shown in Fig. 35, provided that the second effective reference index Ridxl is one, a motion vector corresponding to the second reference index Ridxl of a block containing pixel C is used in a step S3822.

Note that in the above explanation, as for the block containing no reference indices, assuming the block contains a motion vector of "0", an average value of three motion vectors is selected, however, assuming that the block contains a motion vector of "0", an average value of three vectors can be selected, or an average value of block motion vectors containing reference indices can be selected.

Note that a motion vector used for motion compensation of a current block to be encoded can be determined by a different priority than explained above, for example, in the order of pixels B - A - C, which are contained in the blocks. .

Thus, by determining a motion vector used for motion compensation of a current block to be encoded by using a reference index, a motion vector can be uniquely determined. Moreover, according to the example mentioned above, the coding efficiency can be improved. In addition, as long as it is not necessary to judge whether a motion vector is a forward reference or a back reference using time information, a process for determining a motion vector can be simplified. When referring to a prediction mode for each block and a motion vector used for motion compensation or the like, there is a set of patterns, however, as mentioned above, provided that a process is done by a series of streams, it It is useful.

Note that in this embodiment, the case of calculating a motion vector used in direct mode by scaling a motion vector referred to using a time interval between images is explained, however, a calculation can be performed by multiplying. by a constant number. Here, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images. Note that a calculation method using the RidxO and Ridxl reference indices is not just a method using an average value and the calculation methods can be combined with other calculation methods. For example, in the third calculation method mentioned above, when motion vectors contained in blocks having pixels A, B and C respectively referring to the same image whose reference index is the smallest are several, it is not necessary to calculate a value. The mean of these motion vectors and a motion vector obtained from averaging these motion vectors can be used as a motion vector of the BL1 block used in direct mode. Or, for example, a motion vector whose coding efficiency is highest can be selected from the various motion vectors with the lowest reference index.

In addition, a forward motion vector and a backward motion vector of block BL1 can be independently or dependently calculated. For example, a forward motion vector and a backward motion vector can be calculated from the same motion vector.

On the other hand, a forward motion vector or a backward motion vector, both obtained from the calculation, can be used as a motion vector of block BL1.

In this embodiment, a MB reference block in a reference image contains a forward motion vector (the first) referring to a reference image stored in the long image buffer. term as the first reference image and a backward motion vector (the second) referring to a reference image stored in the short-term image buffer as the second reference image.

[00252] Fig. 37 is an illustration showing a double prognosis in direct mode when only one reference image is stored in long term image buffer.

The point different from the eighth embodiment of the various embodiments mentioned above is that a forward motion vector MV21 of an MB2 block in a reference image refers to a reference image stored in long term image buffer.

Short-term image buffer is a buffer for temporarily storing reference images, and for example, images are stored in the order in which images are stored in a buffer (ie, encoding / decoding order). ). When there is not enough storage capacity for images that are recently stored in buffer, the most previously stored images are deleted in buffer.

In long-term image buffering, images are not always stored in long-term image buffering in the order of time as in short-term image buffering. For example, as an order to store the images, the time order of the images can be matched and the address order in a buffer in which the images are stored can be matched. Therefore, it is impossible to scale an MV21 motion vector to an image stored in long-term image buffer based on a time interval.

A long-term image buffer is not for storing a reference image temporarily as short-term image buffering, but for storing a reference image continuously. Therefore, a time interval corresponding to a motion vector stored in long-term image buffering is much broader than a time interval corresponding to a motion vector corresponding to a motion vector stored in long-image buffering short term.

[00257] In Figure 37, a boundary between long-term image buffer and short-term image buffer is indicated by a vertical dotted line as shown in the figure and the information about the images relative to the left side. The vertical dotted line image is stored in the long-term image buffer and information about the images on the right side of the vertical dotted line is stored in the short-term image buffer. Here, an MB1 block in an image P23 is a current block. An MB2 block is a reference block of the same location as MB1 block in a P24 image. An MB21 forward motion vector of block MB2 in reference image P24 is the first motion vector referring to an image P21 stored in long-term image buffer as the first reference image and a backward motion vector. MV25 of block MB2 in reference image P24 is the second motion vector referring to an image P25 stored in short-term image buffer as the second reference image.

As mentioned above, a time interval TR21 between images P21 and P24 corresponds to a forward motion vector MV21 referring to an image stored in the long term image buffer, a time interval TR25 between images P24 and P25 correspond to a backward motion vector MV25 referring to an image stored in the short term image buffer and the time interval TR21 between images P21 and P24 may become much longer. wider than the TR25 time interval between images P24 and P25 or may be undefined.

Therefore, a motion vector of block MB1 in the current image P23 is not calculated by scaling a motion vector of block MB2 in reference image P24 as in the previous embodiments, but the motion vector is calculated using a following method. MV21 = MV21 'MV24' = 0 [00260] The upper equation shows that the first MV21 motion vector stored in long-term image buffer is used directly as the first MV21 motion vector in a current image.

[00261] The equation below shows that since the second motion vector MV24 ', which refers to the image 24 stored in the short-term image buffer, the block MB1 in the current image to the image P23 is sufficiently smaller. than the first motion vector MV21 ', MV24' is insignificant. The second motion vector MV24 'is treated as "0".

As described above, a reference block MB contains a motion vector referring to a reference image stored in long-term image buffer as the first reference image; and a motion vector referring to a reference image stored in short-term image buffer as the second reference image. In this case, the double prognosis is performed using the motion vector stored in long-term image buffer outside the block motion vectors in the reference image directly as a motion vector of a block in a current image.

Note that a reference image stored in long-term buffer may be the first reference image or second reference image and a motion vector MV21 referring to a reference image stored in the buffer image. Long term can be a backward motion vector. In addition, when the second reference image is stored in long-term image buffer and the first reference image is stored in short-term image buffer, a motion vector in a current image is calculated by adjusting the scale of a motion vector referring to the first reference image.

[00264] This makes it possible to perform dual prediction without using the time that is considerably longer in long-term or undefined image buffering.

Note that dual prediction can be performed not directly using a motion vector referenced but using a motion vector by multiplying by a constant number.

In addition, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images. (Ninth Mode) [00267] In this mode, the double prognosis in direct mode is presented. In this case, a reference block MB in a reference image contains two forward motion vectors referring to a reference image stored in a long term image buffer.

[00268] Fig. 38 is an illustration showing the double prognosis in direct mode when a reference block MB in a reference image contains two forward motion vectors referring to a reference image stored in the buffer. Long-term imaging.

The different point of the ninth embodiment of the eighth embodiment is that both motion vectors MV21 and MV22 of an MB2 block in a reference image refer to an image stored in long term image buffer.

In figure 38, a boundary between long-term image buffer and short-term image buffer is indicated by a dotted vertical line as shown in the figure and the information about the images on the left side of the line. vertical dotted image is stored in long-term image buffer and information about the images on the right side of the dotted vertical line is stored in short-term image buffer. Both MB21 block motion vectors MV21 and MV22 in a reference image P24 refer to an image stored in long-term image buffer. Motion vector MV21 corresponds to a reference image P21 and motion vector MV22 corresponds to a reference image P22.

[00271] A time interval TR22 between images P22 and P24 may be much wider than the time interval TR25 between images P24 and P25 or may be undefined corresponding to motion vector MV22 referring to image P22 stored in the long-term image buffer.

[00272] In figure 38. the images are stored in the order of images P22 - P21 in that order in the long term image buffer. Here, image P21 corresponds to a motion vector MV21 and image P22 corresponds to a motion vector MV22. In Fig. 38, a motion vector of an MB1 block in a current image is calculated as follows. MV22 '= MV22 MV24' = 0 [00273] The above equation shows that an MV22 motion vector referring to an image P22 to which the smallest order is assigned is used directly as an MB22 motion vector 'from block MB1 in a current picture P23.

[00274] The equation below shows that since MB1's backward motion vector MV24 'in current image P23 stored in short-term image buffer is sufficiently smaller than motion vector MV21', MV24 'is insignificant. The backward motion vector MV24 'is treated as "0".

As described above, by directly utilizing a motion vector referring to an image to which the smallest order is designated outside the motion vectors of a block in a reference image stored in long term image buffer , the double prognosis can be made without using time, which is considerably greater in long-term or indefinite imaging buffer.

Note that the double prognosis can be made not directly using a motion vector referred to but using an motion vector by multiplying by a constant number.

In addition, a constant used for multiplication can be variable when encoding or decoding is performed based on multiple blocks or based on multiple images.

Additionally, when both motion vectors MV21 and MV22 of block MB2 in a reference image refer to an image stored in long-term image buffer, a motion vector referring to the first reference image may be selected. For example, when MV21 is a motion vector referring to the first reference image and MV22 is a motion vector referring to the second reference image, the MV21 motion vector referring to a P21 image and a motion vector. Motion "0" referring to an image P24 is used as motion vectors of an MB1 block. (Tenth Modality) [00279] In this embodiment, a method of calculating a direct mode motion vector presented in the fifth embodiment until the ninth embodiment is explained. This method of calculating a motion vector is applied for encoding or decoding an image. Here, a current block to be encoded or decoded is called a MB current block. A block from the same location as the current MB block in a reference image is called a reference block.

Fig. 39 is an illustration showing a process flow of a motion vector calculation method of this embodiment.

First, it is judged whether a MB reference block in a back reference image by a current MB block contains a motion vector (step S1). If reference block MB does not contain a motion vector (Not in step S1), the double prognosis is executed assuming that a motion vector is "0" (step S2) and a process for a motion vector calculation is completed. .

If reference block MB contains a motion vector (Yes in Step S1), it is judged whether the reference block contains a forward motion vector (step S3).

[00283] If the reference block does not contain a forward motion vector (Not in step S3), since reference block MB contains only a backward motion vector, the number of backward motion vectors is judged (step S14). When the number of backward motion vectors of reference block MB is "2", the double prognosis is performed using two scaled backward motion vectors based on one of the calculation methods mentioned in figures 17, 18, 19. and 20.

On the other hand, when the number of backward motion vectors of reference block MB is "1", the only backward motion vector contained in reference block MB is scaled and motion compensation is performed. using the scaled backward motion vector (step S16). After completing the double prognosis in step S15 or S16, the motion vector calculation method process is complete.

On the other hand, if reference block MB contains a forward motion vector (Yes in step S3), the number of forward motion vectors of reference block MB is judged (step S4).

When the number of MB reference block forward motion vectors is "1", it is judged whether a reference image corresponding to MB reference block forward motion vector is stored in long image buffer term or short-term image buffer (step S5).

When the reference image corresponding to the MB reference block forward motion vector is stored in the short-term image buffer, the MB reference block forward motion vector is scaled and double predicted. is performed using the scaled forward motion vector (step S6).

When the reference image corresponding to the forward motion vector of reference block MB is stored in the long term image buffer, the double prognosis is performed based on the motion vector calculation method shown in figure 37 assuming that a backward motion vector is 0 and using the forward motion vector of reference block MB directly without scaling (step S7). After completing the double prognosis in step S6 or S7, a process for the motion vector calculation method is completed.

[00289] When the number of forward motion vectors of reference block MB is "2", the number of forward motion vectors corresponding to a reference image stored in long-term image buffer is judged (step S8). ).

When the number of forward motion vectors corresponding to a reference image stored in long-term image buffer is "0" in step S8, a motion vector that is temporarily close to a current image containing the block. The current MB is scaled and the double prognosis is performed using the scaled forward motion vector based on the motion vector calculation method shown in Figure 14 (step S9).

When the number of forward motion vectors corresponding to a reference image stored in long-term image buffer is "1" in step S8, a motion vector in an image stored in short-image buffer The term is scaled and the double prognosis is performed using the scaled motion vector (step S10).

When the number of forward motion vectors corresponding to a reference image stored in long-term image buffer is "2" in step S8, it is judged whether the same image in long-term image buffer is referred to by both two forward motion vectors (step S11). If the same image in the long term image buffer is referred to by both two forward motion vectors (Yes in step S11), the double prognosis is performed using a motion vector previously encoded or decoded in the image referred to by the two forward motion vectors. forward motion in long-term image buffering based on the motion vector calculation method shown in Figure 13 (step S12).

If the same image in the long-term image buffer is not referred to by both two forward motion vectors (Not in step S11), the double prognosis is performed using a forward motion vector corresponding to an image for which a small order is assigned to the long term image buffer (step S13). In long-term image buffering, since reference images are stored independent of the actual time of the images, a forward motion vector to be used for dual prognosis is selected according to an order assigned to each reference image. . There is a case where the order of the reference images stored in the long term image buffer coincides with the time of the images, however, it may merely coincide with a buffer address. In other words, the order of the images stored in the long-term image buffer does not necessarily coincide with the time of the images. After completing the double prognosis in steps S12eS13, the process of motion vector calculation method is completed. (Eleventh Modality) The detailed explanation of the eleventh embodiment according to the present invention will be given below using illustrations.

Fig. 40 is a block diagram showing a configuration of a moving image coding apparatus 1100 according to the eleventh embodiment of the present invention. The motion picture coding apparatus 1100 is an apparatus that can encode motion pictures by applying a direct mode spatial prediction method even if a block encoded in a field structure and a block encoded in a frame structure are mixed and includes a frame memory 1101, a difference calculation unit 1102, a prognostic difference coding unit 1103, a bitstream generation unit 1104, a prognostic difference decoding unit 1105, a addition 1106, a frame memory 1107, a motion vector detection unit 1108, a mode selection unit 1109, an encoding control unit 110, a key 1111, a key 1112, a key 1113, a key 1114 , a key 1115 and a motion vector storage unit 1116.

[00296] Frame memory 1101 is an image memory for storing informed images based on the image. The difference calculation unit 1102 calculates the prognostic error, which is the difference between an informed image from frame memory 1101 and a reference image obtained from a decoded image based on a motion vector and outputs the same. The prediction difference coding unit 1103 performs frequency conversion for the prediction error obtained in the difference calculation unit 1102, quantizes and outputs it. Bitstream generation unit 1104 converts to a coded bit stream format issued after performing variable length coding of the coded result from the prediction difference coding unit 1103 and generates a bit stream by adding additional information such as as the header information in which information related to the coded prognostic error is described. The prognostic difference decoding unit 1105 performs variable length coding and inverse quantization of the encoded result from the prognostic difference coding unit 1103 and thereafter performs inverse frequency conversion such as IDCT conversion after execute and decode the coded result to emit the prognostic residue. Addition operation unit 1106 adds a prognostic residue as a decoded result to the reference image mentioned above and outputs a reference image presenting the same image as an image informed by encoded and decoded image data. Frame memory 1107 is an image memory storing reference images based on images.

The motion vector detection unit 1108 derives a motion vector for each encoding of a current frame to be encoded. The mode selection unit 1109 selects whether the calculation of a motion vector is performed in direct mode or in another mode. The coding control unit 1110 reorders the informed images stored in frame memory 1101 in the input order to the coding order. Additionally, the coding control unit 1110 judges which of a field frame or frame structure is used for coding for each predetermined size of a current frame to be coded. Here, the default size is a size of two macroblocks (for example, 16 (horizontal) x 16 (vertical) pixels) vertically combined (hereafter, pair of macroblocks). If a field structure is used for encoding, a pixel value is read from each other horizontal scan line corresponding to interlacing from frame memory 1101, if a frame base is used for encoding, each pixel value in the The informed image is read sequentially from frame memory 1101 and each read pixel value is placed in memory to configure a current pair of macroblocks to be encoded corresponding to the field structure or frame structure. Motion vector storage unit 1116 stores a motion vector of an encoded macroblock and frame reference indices referred to by the motion vector. Reference indices are stored for each macroblock of the encoded macroblock pairs.

In the following, an operation of the motion picture coding apparatus configured as above is explained. The images to be informed are reported to frame memory 1101 based on the image in order of time. Fig. 41A is an illustration showing an order of the frames reported to the moving image coding apparatus 100 based on the image in time order. Fig. 41B is an illustration showing an order of the images by rearranging the order of the images shown in Fig. 41A to the coding order. In figure 41A, vertical lines indicate images and the number indicated on the lower right side of each image shows the image types (I, P and B) with the first letters of the alphabet and the image numbers in the order of time with the following numbers. . Fig. 42 is an illustration showing a structure of a frame list 300 to explain the eleventh embodiment. Each image reported to frame memory 1101 is reordered in coding order by coding control unit 1110. Images are reordered in coding order based on the reference ratio of an interimage prognostic coding and the coding order, an image. used as a reference image is previously encoded to an image referring to an image.

For example, it is assumed that a P image uses one of three preceding and neighboring I or P images as a reference image. On the other hand, it is assumed that an image B uses one of three preceding and neighboring I or P images and one of the next and neighboring I or P images as a reference image. Specifically, an image P7 which is reported after images B5 and B6 in Fig. 41A is rearranged into placed before images B5 and B6 since image P7 is referred to by images B5 and B6. Similarly, an image P10 informed after images B8 and B9 is reordered and placed before images B8 and B9 and an image P13 informed after images B11 and B12 is rearranged and placed before images B11 and B12. Accordingly, the result of rearranging an order of images shown in Fig. 41A is as shown in Fig. 41B.

It is assumed that each frame reordered image 1101 is read based on the macroblock pair and each macroblock pair is 16 (horizontal) x 16 (vertical) pixels in size. Here, a pair of macroblocks combine two macroblocks vertically. Therefore, a pair of macroblocks is 16 (horizontal) x 32 (vertical) pixels in size. A process for encoding a B11 image is explained below. Note that in this embodiment it is assumed that the coding control unit 1110 controls the reference indices, ie a list of reference frames.

Since frame B11 is frame B, interimage prediction coding is performed using the bi-directional reference. It is assumed that picture B11 uses two of the preceding pictures P10, P7, P4 and a following picture P13 as a reference picture. In addition, it is assumed that the selection of two images from these four images can be specified based on macroblock. Here, it is assumed that reference indices are assigned using an initial condition method. In other words, a list of reference frames 300 during image coding B11 is as shown in Fig. 42. With respect to a reference image in this case, the first reference image is specified by the first reference index in Fig. 42 and The second reference image is specified by the second reference index.

In a process for picture B11, it is assumed that coding control unit 1110 controls switch 1113 to be on, keys 1114 and 1115 to be off. Therefore, a pair of macroblock in image B11 read from frame memory 1101 is reported to motion vector detection unit 1108, mode selection unit 109, and difference calculation unit 1102. In unit motion detection sensor 108 by utilizing the decoded data from images P10, P7, and P4 stored in frame memory 1107 as a reference image, the first motion vector, and the second motion vector of each macroblock contained in a pair. macroblock is derived. In mode selection unit 1109, a coding mode for a pair of macroblocks is determined using the motion vectors derived in motion vector detection unit 1108. Here, it is assumed that the coding mode for an image B can be selected, for example, from intraimage coding, interimaging prognostic coding using one-way motion vector, interimaging prognostic coding using two-way motion vector and a direct mode. When encoding modes other than direct mode are selected, it is determined which of a frame structure or a field structure is used to encode a pair of macroblock at the same time.

[00303] Here, a motion vector calculation method using a direct mode spatial prognostic method is explained. Fig. 43A is a flowchart showing an example of a motion vector calculation procedure using a direct mode spatial prediction method when a pair of macroblocks to be encoded in a field structure and a pair of macroblocks to be encoded in a Frame structure are mixed.

Fig. 43B is an illustration showing an example of a location of neighboring macroblock pairs to which the present invention is applied when a current macroblock pair to be encoded is encoded in a frame structure. Fig. 43C is an illustration showing an example of locating neighboring macroblock pairs to which the present invention is applied when a macroblock pair to be encoded is encoded in a field structure. A pair of diagonally shaded macroblock in FIGS. 43B and 43C is a current pair of macroblock to be encoded.

When a current macroblock pair to be encoded is encoded using a direct mode spatial prediction, three macroblock pairs in the vicinity of the current macroblock pair to be encoded are selected. In this case, the current macroblock pair can be encoded in a field structure or a frame structure. Therefore, the coding control unit 1110 first determines which of a field structure or frame structure is used to encode a pair of current macroblocks to be encoded. For example, when the number and pairs of neighboring macroblock encoded in a field structure is large, a current pair of macroblock encoded in a field structure and if the number of neighboring macroblock pairs encoded in a frame structure is large, A pair of current macroblocks is encoded in a field structure. Thus, by determining which of a field structure and a frame structure is used to encode a current pair of macroblocks to be encoded by using information relative to neighboring blocks, it is not necessary to describe information showing which structure is used to encode a current pair of macroblocks to be encoded in a bit stream. Additionally, as long as a structure is predicted from neighboring macroblock pairs, it is possible to select a suitable structure.

Next, motion vector detection unit 1108 calculates a motion vector of a current macroblock pair to be encoded according to the determination of encoding control 1110. First, motion vector detection unit 1108 checks which of a field structure or frame structure is determined to be used for encoding by control unit 110 (S301) and when the frame structure is determined to be used for encoding, a motion vector of a pair of current macroblock to be encoded is derived using frame structure (S302) and when the field structure is determined to be used for encoding, a motion vector of the current macroblock pair to be encoded is derived using field structure (S303) ).

Fig. 44 is an illustration showing a data configuration of a macroblock pair when coding is performed using a frame structure and a macroblock pair configuration when coding is performed using a field structure. In Figure 44, a white circle indicates a pixel in odd-numbered horizontal scan lines, and a black circle shaded with a dashed oblique line pattern indicates a pixel in even-numbered horizontal scan lines. When a pair of macroblocks are cut from each frame presenting an informed image, the pixels in the odd-numbered horizontal scan lines and the pixels in the even-numbered horizontal scan lines are alternately placed in a vertical direction as shown in Figure 44. When the above mentioned macroblock pair is encoded in the frame structure, a process is performed for each MB1 macroblock and for each MB2 macroblock for the macroblock pair and a motion vector is calculated for each MB1 and MB2 macroblock. forming a pair of macroblocks. When the macroblock pair is encoded in the field structure, the macroblock pair is divided into TF and BF macroblocks. Here, macroblock TF indicates an upper field and macroblock BF indicates a lower field when interlacing a pair of macroblocks in a horizontal scan line direction and the two fields form a pair of macroblocks. Then a motion vector is calculated for both fields respectively.

Based on such a macroblock pair, the case in which a current macroblock pair to be encoded is encoded in a frame structure is explained as shown in Fig. 43B. Fig. 45 is a flowchart showing a detailed processing procedure in step S302 shown in Fig. 43. Note that in Fig. 45, a pair of macroblock is indicated as MBP and a macroblock is indicated as MB.

The mode selection unit 1109 first calculates a motion vector of an MB1 macroblock (an upper macroblock), which is one of the macroblocks forming a current macroblock pair to be encoded, using a direct mode spatial prediction. First, mode selection unit 1109 calculates the smallest index value in the images referred by neighboring macroblock pairs for the first and second indexes respectively (S501). In this case, however, when a neighboring macroblock pair is encoded in the frame structure, the value is determined using only one macroblock adjacent to a current macroblock to be encoded. Next, it is checked if the neighboring macroblock pairs are encoded in the field structure (S502) and if the encoding is performed using the field structure, the number of fields for which the lowest indices are assigned in the referred fields is additionally checked. by the two macroblocks forming neighboring macroblock pairs from a list of reference frames in Fig. 42 (S503).

When the result of the verification of step S503 shows that the smallest index is assigned to any field referred to by the two macroblocks (ie fields to which the same index is assigned), an average value of motion vectors of the two macroblocks. is calculated and made to be a motion vector of the neighboring macroblock pair. This is since, when considering based on an interlaced structure, two macroblocks of the macroblock pairs with the field structure are adjacent to a current macroblock to be encoded with the frame structure.

When the result of the verification of step S503 shows that the smallest index is assigned to only one field referred to by a macroblock; A macroblock motion vector is determined as a neighboring macroblock pair vector (S504A). When the lowest index is not assigned to any of the mentioned fields, a motion vector of the neighboring macroblock pair is assumed to be "0" (S505).

In the above cases, from the motion vectors of neighboring macroblocks, by using only motion vectors referring to the fields to which the lowest index is assigned, it is possible to select a motion vector with the efficiency of higher coding. A process in one step S505 shows that there is no proper motion vector for prognosis.

When the result of the verification of step S502 shows that neighboring macroblock pairs are encoded in the frame structure between neighboring macroblock pairs, a motion vector of a macroblock adjacent to a current macroblock to be encoded is determined as a motion vector of the neighboring macroblock pair (S506).

[00314] The mode selection unit 1109 repeats the processes from steps S501 through S506 above for the three selected neighboring macroblock pairs. As a result, a motion vector is calculated for each of the three neighboring macroblock pairs as for a macroblock of a current macroblock pair to be encoded, for example, an MB1 macroblock.

Next, mode selection unit 1109 checks whether the number of neighboring macroblock pairs referring to a frame with the lowest index or to a frame field between three neighboring macroblock pairs is 1 (S507).

In this case, the mode selection unit 1109 unifies the reference indices of the three neighboring macroblock pairs to a reference frame index or a reference field index and compares them. In a list of reference frames shown in Fig. 42, reference indices are merely assigned to each frame, however, provided that the relationship between the reference frame indices and the reference field indices to which the indices are assigned. For each field is constant, you can convert one from the reference frame list or from a reference field list to other key figures by calculation.

[00317] Fig. 46 is an indicator graph showing a relationship between the reference field indices and the reference frame indices.

As shown in Fig. 46, there are several tables indicated by the first field f1 and the second field f2 in chronological order in a list of reference fields and reference frame indices such as 0, 1 and 2 are assigned to each frame. based on the frames including a current block to be encoded (the frames shown in figure 46). In addition, reference field indices such as 0, 1 and 2 are assigned to the first field f1 and the second field f2 of each frame based on the first field f1 of a frame including a current block to be encoded (when the first field is a current field to be encoded). Note that reference field indices are assigned from the first field f1 and the second field f2 of a frame next to a current field to be encoded. Here, if the current block to be encoded is the first field f 1, the indexes are assigned giving priority to the first field f1 and if the current block to be encoded is the second field f2, the indexes are assigned giving priority to the second field. field f2.

For example, when a neighbor macroblock encoded in the frame structure refers to a frame with a frame reference index "1" and a neighbor macroblock encoded in the field structure refers to the first field f1 with an index With reference field "2", the neighboring macroblocks mentioned above are treated as if they refer to the same image. In other words, when a precondition that a reference frame index referenced by a neighboring macroblock is equal to half of the reference field index (rounding down after the decimal point) assigned to a reference field of another macroblock. neighbor is satisfied, neighboring macroblocks are treated as if they refer to the same image.

For example, when a current block to be encoded included in the first field f1 indicated by Δ in Fig. 46 refers to the first field f1 with reference field index "2" and a neighbor macroblock with structure frame refers to a frame with the reference frame index "1", the neighboring blocks mentioned above are treated as if they referenced the same image as long as the precondition mentioned above is met. On the other hand, when a neighbor macroblock refers to the first field with a reference field index "2" and another neighbor macroblock refers to a frame with a reference frame index "3", neighboring blocks are treated. as if they did not refer to the same image since the precondition is not satisfied.

As mentioned above, if the result of the verification of step S507 shows that the number is 1, a motion vector of a neighboring macroblock pair referring to a field of a frame with the lowest index or a field in the frame is determined as a motion vector of a current macroblock to be encoded (S508). If the result of checking from step S507 shows that the number is not 1, it is further checked whether the number of neighboring macroblock pairs of three neighboring macroblock pairs referring to a frame with the lowest index or a field in the frame is 2. or more (S509). So if the number is 2 or more, assuming a motion vector of neighboring macroblock pairs not referring to a frame with the lowest index or a field in the frame is "0" (S510), an average value of three Movement vectors of neighboring macroblock pairs is determined as a motion vector of a current macroblock to be encoded (S511). If the result of checking step S509 is less than 2, provided that the number of neighboring macroblock pairs referring to the frame with the lowest index or the field in the frame is "0", a motion vector of a current macroblock to be encoded is determined as "0" (S512).

As a result of the above process, a motion vector MV1 can be obtained as a result of the calculation for a macroblock forming a pair of macroblocks to be encoded, for example MB1. The mode selection unit 109 performs the above process for a motion vector with the second reference index and performs motion compensation for the double prognosis using the two motion vectors obtained. However, when neither neighboring macroblock contains the first or second motion vector, motion compensation is performed not by using a motion vector in the direction indicated by a motion vector not contained in neighboring macroblocks, but using a motion vector with One way only. In addition, the same process is repeated for the other macroblocks in the current macroblock pair to be encoded, for example MB2. As a result, it is equal to perform direct mode motion compensation for each of the two macroblocks in a current pair of macroblocks to be encoded.

In the following, a case in which a pair of current macroblock to be encoded is encoded in the frame structure as shown in Fig. 43C is explained. Fig. 47 is a flowchart showing a detailed processing procedure in step S303 shown in Fig. 43. Mode selection unit 1109 calculates a motion vector MVt using a direct mode spatial prediction for a macroblock forming a pair of macroblocks a be encoded, for example, a TF macroblock corresponding to a field above the macroblock pair. First, mode selection unit 1109 calculates the smallest index value in the image referred to by neighboring macroblock pairs (S601). However, when macroblock pairs are processed by the field structure, only one macroblock of a field (an upper field or a lower field) equal to the current macroblock to be encoded is considered. Next, it is checked whether the neighboring macroblock pairs are encoded by the frame structure (S602) and if the encoding is performed using the frame structure, it is further judged whether the frames referred to by the two macroblocks in the neighboring macroblock pair are frames with The smallest index based on the index value assigned to each frame is a list of 300 reference frames (S603).

If the result of the verification of step S603 shows that the lowest index is assigned to either of the frames referred to by the two macroblocks, an average value of the motion vectors of the two macroblocks is calculated and the result of the calculation is determined as a vector. neighboring macroblock pair of motion (S604). If the result of checking from step S603 shows that one or both of the frames referenced are not frames with the lowest index, it is further checked whether a frame referenced by either macroblock contains the lowest index (S605). If the scan result shows that the smallest index is assigned to a frame referred to by one of the macroblocks, a macroblock motion vector is determined as a neighbor macroblock pair motion vector (S606). On the other hand, if the result of the verification of step S605 shows that none of the macroblock refers to one when with the lowest index, a motion vector of the neighboring macroblock pair is determined as "0" (S607). In the above cases, from the motion vectors of neighboring macroblocks, by using only motion vectors referring to the frames for which the lowest index is assigned, it is possible to select a motion vector with the highest coding efficiency. A process in one step S607 shows that there is no proper motion vector for prognosis.

When the result of the verification of step S602 shows that neighboring macroblock pairs are encoded in the field structure in the neighboring macroblock pair, the motion vectors of the neighboring macroblock pair as a whole is determining as a motion vector. of the macroblock pair corresponding to a current macroblock in a current macroblock pair to be encoded (S608). Mode select unit 109 processes from steps S601 through S608 for three pairs of selected neighboring macroblocks. As a result, it is equal to obtain a motion vector for three neighboring macroblock pairs respectively as for a macroblock of the current macroblock pair to be encoded, for example a TF macroblock.

Next, motion vector detection unit 108 checks whether the number of neighboring macroblock pairs referring to a frame with the lowest index among three neighboring macroblock pairs is 1 (S609). If 1, a motion vector of a neighboring macroblock pair referring to a frame with the lowest index is determined as a motion vector of the current macroblock to be encoded (S610). If the result of checking from step S609 shows that the number is not 1, it is further checked whether the number of neighboring macroblock pairs referring to a frame with the lowest index among the three neighboring macroblock pairs is two or more (S611 ). So if the number is two or more, assuming a motion vector of neighboring macroblock pairs not referring to a frame with the lowest index is "0" (S612), an average value of three pair motion vectors neighboring macroblock is determined as a motion vector of the current macroblock to be encoded (S613). If the result of the verification of step S611 is less than 2, since the number of neighboring macroblock pairs referring to a frame with the lowest index is "0", a motion vector of the current macroblock to be encoded. is determined as "0" (S614).

[00327] As a result of the above process, a motion vector MVt can be obtained as a calculation result for a macroblock forming a current macroblock pair to be encoded, for example, a TF macroblock corresponding to an upper field. The mode selection unit 109 repeats the above process also for the second motion vector (corresponding to the second reference index). As for a TF macroblock, two motion vectors can be obtained by the above process and motion compensation is performed using the two motion vectors. However, when neither neighboring macroblock contains the first or second motion vector, motion compensation is performed not by using a motion vector in the direction indicated by a motion vector not contained in neighboring macroblocks, but using a motion vector in One way only. This is because when a neighboring macroblock pair refers only unidirectionally, it is conceivable that coding efficiency becomes higher when a neighboring macroblock pair also refers to only one direction.

In addition, the same process is repeated for another macroblock in the current macroblock pair to be encoded, for example, BF corresponding to a field below. As a result, it is equal to perform direct mode motion compensation for each of the two macroblocks in the current pair of macroblocks to be encoded, for example, TF and BF macroblocks.

Note that in the above cases, when a coding structure for a current macroblock pair to be coded and a coding structure for a neighboring macroblock pair are different, a calculation is performed by a process such as calculating an average value. of the motion vectors of the two macroblocks in the neighboring macroblock pair, however, the present invention is not limited to the above cases. For example, only when a coding structure for a current macroblock pair to be coded and a neighboring macroblock pair are the same, can a motion vector of the neighboring macroblock pair be used and when a coding structure for a pair is of the current macroblock to be encoded and for a neighboring macroblock pair are different, a motion vector of the neighboring macroblock pair whose coding structure is different is not used. Specifically, 1. When a current macroblock pair to be encoded is encoded in the frame structure, only a motion vector of a neighboring macroblock pair encoded in the frame structure is used. In this case, when none of the motion vectors of the neighboring macroblock pair encoded in the frame structure refers to a frame with the lowest index, a motion vector of the current macroblock pair to be encoded is set to "0". When a neighboring macroblock pair is encoded in the field structure, a motion vector of the neighboring macroblock pair is determined as "0".

[00330] Next, 2. When a current macroblock pair to be encoded is encoded in the field structure, only a motion vector of a neighboring macroblock pair encoded in a field structure is used. In this case, when none of the motion vectors of the neighboring macroblock pair encoded in the field structure refers to a frame with the lowest index, a motion vector of the current macroblock pair to be encoded is set to "0". When a neighboring macroblock pair is encoded in the frame structure, a motion vector of the neighboring macroblock pair is determined as "0". Thus, after calculating a motion vector of each neighboring macroblock pair, 3. When the number of motion vectors obtained by referring to a frame with the lowest index or a field in the frame between these motion vectors is only one, the motion vector is determined as a motion vector of a current macroblock pair in direct mode and if the number is not 1, an average value of three motion vectors is determined as a motion vector of the current macroblock pair in direct mode. direct.

Additionally, in the above cases, which of a field structure and a frame structure is used to encode a current macroblock pair to be encoded is determined based on the majority of a coding frame of encoded neighboring macroblock pairs, however. , the present invention is not limited to the above case. An encoding structure may be fixed, for example, a frame structure is always used for direct mode coding, or a field structure is always used for direct mode coding. In this case, for example, when the field structure and frame structure are changed for each current frame to be encoded, this can be described in a bitstream header as a whole or in a frame header of each frame. Swapping is performed, for example, sequence-based, GOP-based, image-based and slice-based, in which case this can be described in a corresponding header of a bit stream or the like. Needless to say, even in the above cases, only when the coding structures used for a current macroblock pair to be encoded and a neighboring macroblock pair are the same, a motion vector of the current macroblock pair to be encoded in the mode. The direct stream can be calculated by a method using a motion vector of the neighboring macroblock pair. In addition, when transmitting as packets, a header part and a data part may be separated and transmitted respectively. In this case, the header part and the data part are never included in a bit stream. However, as for packets, although a transmission order may be more or less out of sequence, a header part corresponding to a corresponding data part is only transmitted in another packet and there is no difference even if it is not included in a bit stream. Thus, while fixing which of the frame structure and field structure is used, a process for determining a coding structure by utilizing neighbor information becomes unnecessary and a process can be simplified.

In addition, in direct mode, after processing a pair of current macroblocks using both the frame structure and the field structure, a higher coding efficiency coding structure can be selected. In this case, it can be described which of the frame structure and field structure is selected in a header part of a pair of coded macroblocks. Needless to say, even in the above cases, only when the coding structures used for a current macroblock pair to be encoded and a neighboring macroblock pair are the same, a motion vector of the current macroblock pair to be encoded in the Direct mode can be calculated by a method using a motion vector of the neighboring macroblock pair. By using such a method, information showing which of the frame structure and field structure is used becomes necessary in a bitstream, however, it is possible to reduce the residual data for motion compensation and the coding efficiency can be reduced. be perfected.

In the above explanation, it is the case that motion compensation is performed for neighboring macroblock pairs based on the size of the macroblock, however, motion compensation can be performed based on a different size. In this case, as shown in Figs. 48A and 48B, a motion vector containing pixels located at a, b and c is used as a motion vector of the neighboring macroblock pair for each macroblock of a current macroblock pair to be encoded. Fig. 48A shows the case processing an upper macroblock and Fig. 48B shows the case processing an upper macroblock. When the structures (a frame structure / field structure) for a current macroblock pair and neighboring macroblock pairs are different, a process is performed using a block including pixels located at a, and a block including pixels located at a ', b' and c 'as shown in figures 49A and 49B. Here, locations a ', b' and c 'are a block included in another macroblock in the same pair of macroblock corresponding to pixel locations a, b and c. For example, in the case of Fig. 49A, when the coding structures (a frame structure / a field structure) for a current macroblock pair and neighboring macroblock pairs are different, a motion vector of a block on the side The left side of a higher current macroblock to be encoded is determined using motion vectors of BL1 and BL2. In the case of Fig. 49B, when the coding structures (a frame structure / a field structure) for a current macroblock pair and neighboring macroblock pairs are different, a motion vector of a block on the left side of a macroblock The upper current to be encoded is determined using the motion vectors of BL3 and BL4. By using such a processing method, even if motion compensation is performed for the size-based neighbor macroblock using a different size than a macroblock, a direct mode process can be performed with regard to the difference in frame structure and size. field structure.

In addition, when motion compensation is performed for the neighboring macroblock based on the different size of a macroblock by calculating an average value of motion vectors of a block included in the macroblock, the calculated value may be a vector. of motion of the macroblock. Even if motion compensation is performed for size-based neighboring macroblocks using a different size than a macroblock, a direct mode process can be performed with respect to the difference of a frame structure and a field structure.

By the way, as mentioned above, a motion vector is derived and interimage prognostic coding is performed based on the derived motion vector. As a result, the derived motion vector in motion vector detection unit 108 and the coded prognostic error image are stored in a macroblock-based bit stream. However, as for a motion vector of a direct mode encoded macroblock, it is merely described that it is encoded in direct mode and the motion vector and reference indices are not described in a bit stream. Fig. 50 is an illustration showing an example of a stream and bit data configuration 700 generated by a bitstream generation unit 104. As shown in Fig. 50, in bitstream 700 generated by the stream generation unit bit 104, a Header is provided for each Image. The Header contains, for example, items such as an RPSL item presenting the change of a reference frame list 10 and an item displaying an image type of the image and not shown in this figure and when a method of assigning the first reference index 12 and second reference index 113 in the frame list 10 is changed from the initial parameters, an assignment method after the change is described in item RPSL.

On the other hand, the coded prognostic error is recorded based on the macroblock. For example, when a macroblock is encoded using a forward mode spatial prediction, a macroblock motion vector is not described in a Blocol item and information showing that an encoding mode is direct mode is described in a "PredType" item. . Here, the Blocol item is an item in which a prediction error corresponding to the macroblock is described and the item "PredType" has a macroblock coding mode. When it is selected that one of the frame structure and field structure is used for coding from the coding efficiency point of view mentioned above, the information showing the choice between frame structure and field structure is described. Subsequently, the coded prognostic error is described in a CodeRes item. When another macroblock is a macroblock encoded in an image prediction coding mode, it is described in the "PredType" item in a Block2 item that an encoding mode for the macroblock is the inter prediction coding mode. Here the CodeRes item has a coding mode described and the "PredType" item is an item in which a prediction error corresponding to the macroblock is described. In this case, the first reference index 12 of the macroblock is further described in a RidxO item and the second reference index 13 is further described in a Ridxl item other than the coding mode. Reference indices in a block are represented by variable-length codewords and the shortest code length is assigned the smallest value. Subsequently, a motion vector of the macroblock during forward frame reference is described in an MVO item and a motion vector during backward frame reference is described in an MV1 item. Then the coded prognostic error is described in the CodeRes item.

Fig. 51 is a block diagram showing a configuration of a motion picture decoding apparatus 800 which decodes the bit stream 700 shown in Fig. 50. The moving image decoding apparatus 800 is a decoding apparatus bit stream decoding motion picture 700 in which a prediction error including a direct mode encoded macroblock is described and includes a bit stream analysis unit 701, a prognostic difference decoding unit 702, a 703 decoding mode, a motion compensation decoding unit 705, a motion vector storage unit 706, a frame memory 707, an addition operation unit 708, keys 709 and 710, and a decoding unit vector set 711. Bitstream analysis unit 701 extracts various data from the reported bitstream 700. Here, the various data data includes information such as information relating to a coding mode and information relating to a motion vector or the like. Information extracted with respect to a coding mode is output to mode decoding unit 703. On the other hand, information extracted with respect to a motion vector is output to motion vector decoding unit 705. Additionally, extracted prognostic difference coding data is output to the prognostic difference decoding unit 702. The prognostic difference decoding unit 702 decodes the informed prognostic difference coding data and generates a prognostic difference image. The generated prognostic difference image is output to key 709. For example, when key 709 is connected to a terminal b, a prognostic difference image is output to addition operation unit 708.

The mode decoding unit 703 controls keys 709 and 710 by referring to encoding mode information extracted from a bit stream. When a coding mode is intra-picture coding mode, mode decoding unit 703 controls to connect switch 709 with terminal a and to connect switch 710 with terminal c. In addition, mode decoder unit 703 outputs encoding mode information to motion compensation decode unit 705 and motion vector decode unit 711. Motion vector decode unit 711 decodes a encoded motion vector entered from bit stream analysis unit 701. The decoded reference image number and the decoded motion vector are stored in motion vector storage unit 706 and output to the compensation decode unit motion vector icon at the same time.

When a coding mode is direct mode, mode decoder unit 703 controls to connect switch 709 to terminal b and to connect switch 710 to terminal d. In addition, mode decoder unit 703 outputs encoding mode information to motion compensation decode unit 705 and motion vector decode unit 711. Motion vector decode unit 711 determines a motion vector to be used in direct mode using a motion vector of the neighboring macroblock pair and a reference image number stored in motion vector storage unit 706, when an encoding mode is direct mode. Since the method for determining a motion vector is the same as the contents explained for the operation of mode selection unit 109 shown in Fig. 40, the explanation will be omitted here.

Based on the decoded reference image number and decoded motion vector, the motion compensation decoding unit 705 obtains a macroblock-based motion compensation image from frame memory 707. The compensation image motion rate obtained is output to addition operation unit 708. Frame memory 707 is a memory storing frame-based decoding images. Addition operation unit 708 adds the informed prediction difference image to a motion compensation image and generates a decoded image. The generated decoded image is output to frame memory 707.

As mentioned above, according to this embodiment, even if a neighboring macroblock pair encoded by a frame structure and a neighboring macroblock pair encoded by a field structure are mixed into the encoded neighboring macroblock pairs corresponding to one pair. Since current macroblocks to be encoded include a direct mode spatial prediction method, a motion vector can be easily calculated.

It is noted that in the above embodiment, the case where each image is processed based on the pair of macroblocks (connecting two macroblocks vertically) using a frame structure or a field structure is explained, however, a process can be performed by exchanging a frame structure and a field structure on a different basis, for example based on the macroblock.

Also, in the above embodiment, it is the case that a macroblock in an image B is processed in direct mode, however, a macroblock in an image P can be similarly processed. When encoding and decoding an image P, each block performs motion compensation from only one image and a list of reference frames is only one. Therefore, in order to perform the process equal to this mode on an image P, the process by calculating two motion vectors of a current block to be encoded / decoded (the first frame of reference and the second frame of reference) in this mode can be changed to a process calculating a motion vector.

In addition, in the above embodiment, the case where prognostic generation of a motion vector used in direct mode is performed using motion vectors of three neighboring macro-pair pairs is explained, however, the number of pairs of neighboring macroblocks to be used may differ. For example, a case using only a motion vector of a neighboring macroblock pair located on the left side is conceivable. (Twelfth Mode) [00345] By storing a program for performing a configuration of the image encoding method and the image decoding method mentioned in the above embodiment on a storage medium such as a floppy disk, it is possible to easily execute the process mentioned in the above embodiment on a standalone computer system.

Fig. 52 is an illustration explaining a storage medium storing a program for performing the encoding method and the image decoding method from the first to the eleventh embodiments above in a computer system.

Figure 52B shows an external view of the floppy disk viewed from the front, a cross-sectional configuration and the floppy disk, and Figure 52A shows an example of a physical shape of a floppy disk as a body of the media. recording. A floppy disk FD is contained in a shell F and several Tr tracks are formed concentric on the disc surface from the inner radius to the inner radius and each track is divided into 16 sectors if in the angular direction. Therefore, as for the floppy disk storing the program mentioned above, the image encoding method and an image decoding method such as the above program are stored in an area allocated on the floppy disk FD mentioned above.

Fig. 52C shows a configuration for writing and reading the above mentioned program on floppy disk FD. When the above mentioned program is stored on floppy disk FD, the image encoding method and the image decoding method as the above mentioned program are recorded via a floppy disk controller from a Cs computer system. When the aforementioned encoding or decoding method is built into the computer system by the program on the floppy disk, the program is read from the floppy disk and transferred to the computer system.

It is noted that in the above explanation a floppy disk is used as a storage medium, however it is possible to perform the same way using an optical disk. In addition, a storage medium is not limited to a floppy disk and a medium capable of storing a program such as a CD-ROM, a memory card and a ROM cassette can perform the same way.

Applications of the image coding method and the image decoding method presented in the above embodiments and a system using the applications will be further explained.

Fig. 53 is a block diagram showing an overall configuration of an ex100 content delivery system for performing the content delivery service. The area for providing the communication service is divided into cells of desired size and cell locations ex107 through ex110 which are fixed wireless stations are placed in each cell.

In this ex100 content delivery system, for example, the ex101 Internet is connected to devices such as an ex111 computer, an ex112 Digital Personal Assistant (PDA), an ex113 camera, an ex114 mobile phone, and a cell phone with a camera ex115 via the ex102 internet service provider, an ex104 phone network, and ex107 through ex110 cell locations.

However, the ex100 content delivery system is not limited to the configuration as shown in Fig. 53 and may be connected with a combination of any of them. In addition, each device [00354] can be connected directly to the ex104 telephone network, not through cell locations ex107 through ex110.

Camera ex113 is a device such as a digital camcorder capable of shooting moving images. The cell phone may be a cell phone of a PDC (Personal Digital Communication) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Broadband Code Division Multiple Access) system or a GSM System (Global System for Mobile Communications), a PHS (Personal Handheld Telephone System) or the like.

An ex103 streaming server is connected to the ex113 camera via the ex109 cell site and the ex104 telephone network and live or similar distribution using the ex113 camera based on encoded data transmitted from the user becomes possible. The camera ex113 or the server for transmitting the data may encode the data. In addition, motion picture data filmed by a camera 116 may be transmitted to streaming server ex103 via computer ex111. The ex116 camera is a device such as a digital camera capable of shooting moving and still images. The camera ex116 or computer ex111 can encode motion picture data. An ex117 LSI included with the ex111 computer or ex116 camera performs encoding processing. Software for encoding and decoding images can be integrated into any type of storage medium (such as a CD-ROM, a floppy disk and a hard disk) which is a recording medium that can be read by the ex111 computer or the like. . Additionally, an ex115 mobile phone with an ex115 camera can transmit moving image data. This image data is the LSI encoded data included with the ex115 mobile phone.

The ex100 content delivery system encodes content (such as a live music video) shot by users using camera ex113, camera ex116 or similar in the same manner as above and transmits it to the server. The ex103 streaming server, while the ex103 streaming server distributes the content data stream to clients upon their request. Customers include the ex111 computer, the ex112 PDA, the ex113 camera, the ex114 mobile phone, and so on capable of decoding the encoded data mentioned above. In the ex100 content delivery system, customers can thus receive and reproduce the encoded data and additionally can receive, decode and reproduce the data in real time in order to perform the personal broadcast.

When each device in this system performs encoding and decoding, the image coding apparatus or the image decoding apparatus shown in the above embodiment may be used. A cell phone is explained as an example.

Fig. 54 is an illustration showing the mobile phone ex115 using the image coding method and the image decoding method explained in the above embodiments. The ex115 mobile phone has an ex201 antenna for communicating with the ex110 cell site via radio waves, an ex203 camera unit such as a CCD camera capable of shooting moving and still images, an ex202 display unit such as a video camera. liquid crystal for displaying the data obtained by decoding the images and the like filmed by the camera unit ex203 and received by the antenna ex201, a body unit including an ex204 operation key set, an ex208 voice output unit such as a loudspeaker. speakerphone, an ex205 voice input unit such as a voice input microphone, an ex207 storage medium for storing encoded or decoded data such as motion picture or camera shot data, e-mail data received and still or moving picture data, and an ex206 slot unit to connect the ex207 storage media to the mobile phone ex115. The storage medium ex207 stores itself a flash memory, a type of electrically erasable Programmable Read-Only Memory (EEPROM) which is a non-volatile memory that can be electrically erased and rewritten in a plastic enclosure such as a SD card.

Next, the ex115 cell phone is explained with reference to figure 55. On the ex115 cell phone, an ex311 main control unit for general control of each body unit unit including the ex202 video unit and operation keys ex204 is connected with an ex310 power supply circuit unit, an ex304 operation input control unit, an ex312 image encoding unit, an ex303 camera interface unit, an LCD (Crystal Video) control unit Ex302, an ex309 image decoding unit, an ex308 multiplexing / splitting unit, an ex307 write and read unit, an ex306 modem circuit unit, and an ex305 voice processing unit with each other across the bus synchronous ex313.

When a call end key or an activation key is turned on by the operation of a user, the ex310 power supply circuit unit supplies its components with power from a battery pack to activate the Digital cell phone with an ex115 camera to put it in the ready state.

On the ex115 mobile phone, the ex305 voice processing unit converts the voice signals received by the ex205 voice input unit into conversion mode for digital voice data under the control of the ex311 main control unit including a CPU, ROM and RAM, the ex306 modem circuit unit performs digital voice data broadcast spectrum processing and the ex301 communication circuit unit performs digital to analog conversion and data frequency transformation to transmit same via the ex201 antenna. In addition, on the ex115 mobile phone, the ex301 communication circuit unit amplifies the data received by the ex201 antenna in talk mode and performs frequency transformation and analog to digital conversion to data, the ex306 modem circuit unit performs reverse data spread spectrum processing and voice processing unit ex305 converts it to analog voice data so as to output it via voice output unit 208.

Additionally, when transmitting an e-mail in data communication mode, e-mail text data informed by operating the body unit's operation keys ex204 is sent to the ex311 main control unit via the ex304 operating input control unit. On the ex311 main control unit, after the ex306 modem circuit unit performs text data diffusion spectrum processing and the ex301 communication circuit unit performs digital to analog conversion and frequency transformation, data is transmitted to cell location ex110 via antenna ex201.

When image data is transmitted in data communication mode, image data filmed by camera unit ex203 is provided to image encoding unit ex312 via interface unit with camera ex303. When they are not transmitted, it is also possible to display the image data shot by camera unit ex203 directly on video unit 202 via interface unit with camera ex303 and LCD control unit ex302.

The image coding unit ex312, which includes the image coding apparatus as explained in the present invention, compresses and encodes the image data provided from the camera unit ex203 by the coding method used for the image coding apparatus. image encoding as presented in the above-mentioned embodiment in order to transform it into encoded image data and send it to the multiplexing / splitting unit ex308. At this time, the mobile phone ex115 sends the voices received by the voice input unit ex205 during shooting by the camera unit ex203 to the multiplexing / separation unit ex308 as digital voice data via the voice processing unit ex305.

The multiplexing / separating unit ex308 multiplexes the encoded image data provided from the image coding unit ex312 and the voice data supplied from the speech processing unit ex305 by a predetermined method, the unit modem circuitry ex306 performs diffusion spectrum processing of the multiplexed data obtained as a result of multiplexing and the ex301 communication circuitry unit performs digital to analog conversion and data frequency transformation to transmit via the ex201 antenna .

With respect to receiving data from a moving image file that is linked with a Network (Web) page or the like in data communication mode, the ex306 modem circuit unit performs reverse spectrum processing. broadcast data received from cell location ex110 via antenna ex201 and sends multiplexed data obtained as a result of processing to multiplexing / splitting unit ex308.

In order to decode the multiplexed data received via the ex201 antenna, the ex308 multiplexing / splitting unit separates the multiplexed data into an image data bitstream and a voice data bitstream and provides the data. encoded images for the image decoding unit ex309 and voice data for the speech processing unit ex305, respectively, via the synchronous bus ex313.

Next, the image decoding unit ex309, which includes the image decoding apparatus as explained in the present invention, decodes the image data stream and bits by the decoding method corresponding to the encoding method as shown in mentioned mode for generating the reproduced motion picture data and provides this data to the ex202 video unit via the ex302 LCD control unit and thus the motion picture data included in a motion picture file linked with a Network, for example, are displayed. At the same time, the ex305 voice processing unit converts the voice data into analog voice data and supplies this data to the ex208 voice output unit and thus the voice data included in the one page motion picture file Network, for example, are played.

[00370] The present invention is not limited to the above mentioned system and at least the image coding apparatus or the image decoding apparatus in the above mentioned embodiments may be incorporated into a digital broadcasting system as shown in Fig. 56. terrestrial or satellite digital has been in the news lately. More specifically, a bit stream of video information is transmitted from an ex409 broadcast station or communicated by an ex410 broadcast satellite via radio waves. Upon reception, the ex410 broadcast satellite transmits radio waves for broadcast, an ex406 household antenna with a satellite broadcast reception function receives the radio waves and an ex401 television (receiver) or set-top box ( STB) ex407 decodes the bitstream for playback. The image decoding apparatus as presented in the above mentioned embodiment may be implemented in the playback device ex403 to read and decode the bit stream recorded on an ex402 storage medium which is a recording medium such as a CD and a DVD. In this case, the reproduced video signals are displayed on an ex404 monitor. It is also designed to implement the image decoding apparatus in the ex407 set-top box connected with an ex405 cable to a cable television or to the ex406 satellite and / or terrestrial antenna to reproduce them on an ex408 monitor on the ex401 television. . The picture decoder can be incorporated into the television, not the decoder box. Or, an ex412 car having an ex411 antenna can receive signals from the ex410 satellite or ex107 cell location to reproduce moving images on a video device such as an ex413 car navigation system.

Additionally, the image coding apparatus as presented in the above-mentioned embodiment may encode image signals for recording on a storage medium. As a concrete example, there is an ex420 recorder, such as a DVD recorder for recording image signals on an ex421 DVD disc and a disc recorder for recording them on a hard disk. They can be recorded to an ex422 SD card. If the ex420 recorder includes the picture decoding apparatus as shown in the above mentioned mode, the picture signals recorded on the ex421 DVD disc or ex422 SD card can be played back for display on the ex408 monitor.

Like the structure of the ex413 car navigation system, the structure without the ex203 camera unit and camera, the ex303 camera interface unit and the ex312 picture coding unit outside the components shown in figure 55 is conceivable. The same goes for the ex111 computer, the ex401 television (receiver) and others.

In addition, three types of implementations may be designed for a terminal such as the above mentioned mobile phone ex114; a send / receive terminal implemented with both an encoder and a decoder; a send terminal implemented with only one encoder; and a receive terminal implemented with a decoder only.

As described above, it is possible to use the motion picture encoding method or the motion picture decoding method in the aforementioned embodiments in any of the above mentioned devices and systems and using this method, the effects described in the embodiments above can be obtained.

From the invention thus described, it will be obvious that the embodiments of the invention may be varied in various ways. Such variations are not to be construed as departing from the spirit and scope of the invention and all such modifications as would be obvious to those skilled in the art are intended for inclusion within the scope of the following claims.

Industrial Applicability The image coding apparatus according to the present invention is useful as an image coding apparatus embedded in a personal computer with communication function, a PDA, a broadcasting station for digital broadcasting and a a cell phones or similar.

The image decoding apparatus according to the present invention is useful as an image decoding apparatus equipped for a communication function personal computer, a PDA, an STB for receiving digital broadcast and a cell phone or the like.

Claims (4)

Method for determining the motion vector for directly determining a motion vector of a current block (MB1) included in a current image (P23) which is included in encoded form in a bit stream, characterized by It comprises the steps of: specifying (106,608), when a co-located block (MB2) has exactly two motion vectors (MV21, MV22), only one motion vector out of the two motion vectors, the co-block located being located in a second motion compensated image (P24) at the same position as the current block is located in the current image, the second compensated motion image (P24) being an image after the current image in the display order; and determining (106,608) the motion vector of the current block using the determined motion vector and not the other outside the two motion vectors, where, in the case where (i) a first motion vector (MV21) of the two motion vectors co-located block motion refers to a first reference image (P21) preceding the second compensated motion image (P24) in the display order and (ii) a second motion vector (MV22) of the two motion vectors of the co-located block refers to a second reference image (P22) preceding the second compensated motion image (P24) in the display order, only the first motion vector is specified in the specification of a motion vector, the first motion vector being encoded or decoded before the second motion vector, and wherein the first reference image and the second reference image are different images and the second compensated motion image is a B-image. Method according to claim 1, characterized in that all blocks included in the second motion compensated image (P24) are encoded using a frame structure. 3. Apparatus for determining the motion vector which directly determines a motion vector of a current block (MB1) included in a current image (P23) which is included in encoded form in a bit stream characterized by the fact that comprising: a specific unit configured to specify (106,608), when a co-located block (MB2) has exactly two motion vectors (MV21, MV22), only one motion vector out of the two motion vectors, block co -located being located on a second motion compensated image (P24) in the same position as the current block is located on the current image, the second compensated motion image (P24) being an image after the current image in the display order; and a unit of determination configured to determine the motion vector of the current block using the determined motion vector, in which case (i) a first motion vector (MV21) of the two motion vectors of the co-located block refers to a first reference image (P21) preceding the second compensated motion image (P24) in the display order and (ii) a second motion vector (MV22) of the two motion vectors of the co-located block refers to if a second reference image (P22) preceding the second compensated motion image (P24) in the display order, only the first motion vector is specified in specifying a motion vector, the first motion vector being encoded, or decoded before the second motion vector, and wherein the first reference image and the second reference image are different images and the second compensated motion image is a B-image. Apparatus according to claim 3, characterized in that all blocks included in the second compensated motion picture (P24) are encoded using a frame structure.