JP3884472B2 - Video encoding method and video decoding method - Google Patents

Description

  The present invention relates to a moving image encoding method, a moving image decoding method, a moving image encoding device, a moving image decoding device, and a method for encoding and decoding each picture constituting a moving image with a frame structure or a field structure, and It relates to a program for implementing it in software.

  In moving picture coding, the amount of information is generally compressed using redundancy in the spatial direction and temporal direction of a moving picture. Here, inter-picture predictive coding is used as a method of using temporal redundancy. In inter-picture predictive coding, when a certain picture is coded, a picture that is forward or backward in display time order is used as a reference picture. Then, the amount of motion from the reference picture is detected, and the amount of information is compressed by removing the redundancy in the spatial direction for the difference value between the motion compensated picture and the picture to be encoded.

  H. currently being standardized. In a moving picture coding scheme called H.264, a picture that is subjected to intra-picture prediction coding using only a picture to be coded without having a reference picture is called an I picture. Here, a picture means one encoding unit that includes both a frame and a field. Also, a picture that is inter-picture prediction encoded with reference to one already processed picture is called a P picture, and a picture that is inter-picture predictively encoded with reference to two already processed pictures at the same time is called a B picture. Call it.

  FIG. 17 is a schematic diagram showing the prediction relationship of each picture in the moving picture coding method. In FIG. 17, the vertical line indicates one picture, and the picture type (I, P, B) is shown at the lower right of each picture. In addition, the arrows in FIG. 17 indicate that the picture at the start of the arrow performs inter-picture predictive encoding using the picture at the end of the arrow as a reference picture. For example, the second B picture from the beginning is encoded by using the first I picture and the fourth P picture from the beginning as reference pictures.

  H. In the H.264 system, an encoding mode called a direct mode can be selected for encoding a B picture. This direct mode has two types of methods, a temporal method and a spatial method. In temporal direct mode, the current block itself does not have a motion vector, and the motion vector of another encoded picture is used as a reference motion vector, and scaling processing is performed based on the display temporal positional relationship between pictures. Thus, the motion vector used in the encoding target block is predicted and generated (see, for example, Patent Document 1).

  FIG. 18 is a schematic diagram showing a motion vector prediction generation method in the temporal direct mode, where P is a P picture, B is a B picture, and numbers attached to picture types indicate the display order of the pictures. ing. Each picture P1, B2, B3, and P4 has display order information T1, T2, T3, and T4, respectively. Here, a case where the block BL0 of the picture B3 shown in FIG. 18 is encoded in the temporal direct mode will be described.

  In this case, the motion vector MV1 of the block BL1 in the same position as the block BL0 in the picture P4 that is an already encoded picture located near the display time of the picture B3 is used. The motion vector MV1 is a motion vector used when the block BL1 is encoded, and refers to the picture P1. In this case, the motion vector used when coding the block BL0 is the motion vector MV_F for the picture P1 and the motion vector MV_B for the picture P4. At this time, assuming that the magnitude of the motion vector MV1 is MV, the magnitude of the motion vector MV_F is MVf, and the magnitude of the motion vector MV_B is MVb, MVf and MVb are obtained by Expression 1a and Expression 1b, respectively.

MVf = (T3-T1) / (T4-T1) × MV (Formula 1a)
MVb = (T3-T4) / (T4-T1) × MV (Formula 1b)

  Using the motion vector MV_F and the motion vector MV_B obtained by performing the scaling process from the motion vector MV1 in this way, motion compensation of the block BL0 is performed from the picture P1 and the picture P4 which are reference pictures.

  On the other hand, in the spatial direct mode, the encoding target block itself does not have a motion vector as in the temporal direct mode, and refers to the motion vector of the encoded block located spatially around the encoding target block. , Encoding is performed using it.

  FIG. 19 is a schematic diagram showing a motion vector prediction generation method in the spatial direct mode, where P is a P picture, B is a B picture, and the numbers attached to the picture types indicate the display order of each picture. ing. Here, a case where the block BL0 of the picture B3 shown in FIG. 19 is encoded in the spatial direct mode will be described.

In this case, among the motion vectors MVA1, MVB1, and MVC1 of the encoded block including the three pixels A, B, and C around the block BL0 that is the encoding target, the most in display time from the encoding target picture. A motion vector that refers to a nearby already encoded picture is determined as a motion vector candidate for the encoding target block. If there are three determined motion vectors, the median value is selected as the motion vector of the encoding target block. If there are two, the average value of them is obtained and used as the motion vector of the encoding target block. If there is only one, the motion vector is used as the motion vector of the encoding target block. In the example shown in FIG. 19, the motion vectors MVA1 and MVC1 are obtained with reference to the picture P2, and the motion vector MVB1 is obtained with reference to the picture P1. Therefore, the average value of the motion vectors MVA1 and MVC1 that refer to the picture P2, which is the already encoded picture closest to the encoding target picture, is obtained, and the first motion vector of the encoding target block is obtained. Let it be a certain MV_F. The same applies to obtaining MV_B, which is the second motion vector.
Japanese Patent Laid-Open No. 11-75191

  H. In the H.264 encoding method, in addition to encoding a single picture with a frame as it is for a progressive picture, one picture used for an interlaced picture has two pictures, a top field and a bottom field. It is permissible to use field coding that divides the field into a number of fields.

  FIG. 20 is a schematic diagram illustrating display order information included in fields in an interlaced image and a progressive image, and two vertical lines having the same frame number indicate fields. In the interlaced image, display order information is assigned so that the top field and the bottom field are equally spaced as shown in FIG. On the other hand, in the progressive image, as shown in FIG. 20B, the two fields have the same display order information, so that an accurate display order relationship can be expressed. In the following, an image in which two fields belonging to the same frame have the same display order information is referred to as a progressive image, and an image other than that is referred to as an interlaced image. It is possible to give the same display order information to two fields belonging to.

  Therefore, when field coding is performed with interlaced images and progressive images, when the temporal direct mode is selected, the motion vector is scaled using the method described in the background art using the display order information of each field. It will be. At this time, there are cases where two pictures to be referred to become a top field and a bottom field belonging to the same frame. This case will be described below separately for each case of an interlaced image and a progressive image.

  FIG. 21 is a schematic diagram showing a motion vector prediction generation method in the temporal direct mode for interlaced images, where P is a P picture, B is a B picture, and the number attached to the picture type is the number of each picture. The display order is shown. Here, a case will be described in which block BL0 of picture B2 shown in FIG. 21 is field-encoded in the temporal direct mode.

  In this case, the motion vector MV1 of the block BL1 in the same position as the block BL0 in the picture P3 that is the backward reference picture of the picture B2 is used. This motion vector MV1 is a motion vector used when the block BL1 is encoded, and refers to the top field of the same picture P3. In this case, the motion vector MV_F and the motion vector MV_B used when the block BL0 is encoded can be obtained as follows using the above equations 1a and 1b.

MVf = (4-5) / (6-5) × MV = −MV
MVb = (4-6) / (6-5) × MV = −2MV

  FIG. 22 is a schematic diagram showing a motion vector prediction generation method in a temporal direct mode in a progressive image, where P is a P picture, B is a B picture, and the number attached to the picture type is the number of each picture. The display order is shown. Here, a case will be described in which block BL0 of picture B2 shown in FIG. 22 is field-encoded in the temporal direct mode.

  In this case, the motion vector MV1 of the block BL1 in the same position as the block BL0 in the picture P3 that is the backward reference picture of the picture B2 is used. This motion vector MV1 is a motion vector used when the block BL1 is encoded, and refers to the top field of the same picture P3. In this case, the motion vector MV_F and the motion vector MV_B used when the block BL0 is encoded cannot be obtained in the above equations 1a and 1b because the denominator becomes 0 as follows.

MVf = (3-5) / (5-5) × MV calculation impossible MVb = (3-5) / (5-5) × MV calculation impossible

  As described above, when field coding is performed with a progressive image, the temporal direct mode is selected, and if the two pictures to be referenced are the top field and the bottom field belonging to the same frame, the motion is caused by the scaling process. The vector cannot be predicted and generated.

  Similarly, when field coding is performed on an interlaced image and progressive image, when the spatial direct mode is selected, the display order information of each field is used to display from the encoding target picture as described above. The motion vector that refers to the closest encoded picture that is closest is determined as a motion vector candidate of the encoding target block. At this time, there is a case where the picture referred to by the motion vector becomes a top field and a bottom field belonging to the same frame.

  FIG. 23 is a schematic diagram illustrating a motion vector prediction generation method in a spatial direct mode in a progressive image, where P is a P picture, B is a B picture, and the number attached to the picture type is the number of each picture The display order is shown, and T at the end indicates a top field and B indicates a bottom field. Here, a case will be described in which block BL0 of picture B3_T shown in FIG. 23 is field-encoded in the spatial direct mode.

  In this case, the motion vectors MVA1, MVB1, and MVC1 of the encoded block including the three pixels A, B, and C around the block BL0 to be encoded refer to the fields P2_T, P1_B, and P2_B, respectively. . Of these, the fields P2_T and P2_B are the top field and the bottom field belonging to the same frame, and thus have the same display order information. Therefore, it is impossible to specify which of the fields P2_T and P2_B is closest to the encoding target picture in terms of display time. Therefore, the motion vector of the encoding target block cannot be predicted and generated.

  Therefore, the present invention has been made in view of the above circumstances, and when performing field encoding and decoding of a moving image, a moving image code capable of reliably obtaining a motion vector even when the direct mode is selected. An object of the present invention is to provide a conversion method and a moving image decoding method.

  In order to achieve the above object, a moving picture coding method according to the present invention is a method of coding each picture constituting a moving picture with a frame structure or a field structure, and refers to an already coded picture. A motion vector calculation step for calculating a motion vector for each block constituting the picture, a mode determination step for determining the encoding mode of the block to be processed, and the encoding mode determined in the mode determination step are displayed in display time. The motion vector of the block to be processed is obtained by performing the scaling process of the reference motion vector based on the display temporal positional relationship between the reference pictures, using the motion vector of the encoded picture in the vicinity as a reference motion vector. If the encoding mode is predicted and generated, the motion of the processing target block Scaling determination step for determining whether or not it is possible to predict and generate a Kuttle, and based on the determination result of the scaling determination step, the encoding mode determined in the mode determination step is used as it is or after being updated And a motion compensation step for performing motion compensation.

  As a result, the motion vector of an encoded picture that is close in display time is used as a reference motion vector, and the reference motion vector is scaled based on the display temporal positional relationship between the reference pictures to thereby process the block to be processed. When encoding is performed in a temporal direct mode in which motion vectors are predicted and generated, even if scaling processing is not possible, processing such as changing the encoding mode may be performed to encode the processing target block it can.

  The moving picture coding method according to the present invention is a method of coding each picture constituting a moving picture with a frame structure or a field structure, and constructs a picture by referring to an already coded picture. A motion vector calculation step for calculating a motion vector for each block, and a motion vector of an already encoded block located in the spatial periphery of the processing target block, is already closest to the processing target picture in display time. Whether or not the motion vector of the processing target block can be predicted and generated when predicting and generating the motion vector of the processing target block based on the motion vector referring to the encoded picture The motion vector cannot be generated in the prediction determination step for determining If it is determined, characterized in that it comprises a recent picture determination step of determining by using information other than display order information picture to closest to the target picture.

  As a result, among the motion vectors of the already encoded block located in the spatial periphery of the processing target block, the motion vector that refers to the already encoded picture that is closest in display time from the processing target picture. Based on the display order information of the picture, when encoding is performed in the spatial direct mode for predicting and generating the motion vector of the processing target block, it is not possible to generate and predict the motion vector. It is possible to encode a processing target block by predicting and generating a motion vector by performing a process of determining a picture closest to the encoding target picture.

  The moving picture decoding method according to the present invention is a method for decoding each picture constituting a moving picture with a frame structure or a field structure, and constructs a picture by referring to an already decoded picture. A motion vector calculation step for calculating a motion vector for each block, a mode extraction step for extracting the decoding mode of the processing target block, and a decoding mode in which the decoding mode extracted in the mode extraction step is close in display time Decoding by predicting and generating a motion vector of the processing target block by performing a scaling process of the reference motion vector based on a display temporal positional relationship between reference pictures, using a motion vector of a converted picture as a reference motion vector In the normalization mode, the motion vector of the target block is predicted and generated. A scaling determination step for determining whether or not it can be performed, and motion compensation is performed by using the decoding mode extracted in the mode extraction step as it is or based on the determination result of the scaling determination step. And a motion compensation step to be performed.

  As a result, when the extracted encoding mode is the temporal direct mode and the scaling process is not possible, the decoding target block is decoded by performing a process such as changing the decoding mode. can do.

  The moving picture decoding method according to the present invention is a method for decoding each picture constituting a moving picture with a frame structure or a field structure, and constructs a picture by referring to an already decoded picture. A motion vector calculation step for calculating a motion vector for each block and a motion vector of an already decoded block located in the spatial periphery of the processing target block, which is already closest to the processing target picture in display time Whether or not the motion vector of the processing target block can be predicted and generated when the motion vector of the processing target block is predicted and generated based on the motion vector referring to the decoded picture. The motion vector cannot be generated in the prediction determination step for determining If it is determined, characterized in that it comprises a recent picture determination step of determining by using information other than display order information picture to closest to the target picture.

  As a result, when decoding is performed in the spatial direct mode, even if the motion vector cannot be predicted and generated based on the display order information of the picture, the picture closest to the decoding target picture The motion block is decoded by predicting and generating a motion vector.

  Furthermore, the present invention can be realized not only as such a moving image encoding method and a moving image decoding method, but also includes the characteristic steps included in such a moving image encoding method and a moving image decoding method. It can also be realized as a moving image encoding device and a moving image decoding device provided as means, or as a program for causing a computer to execute these steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  In addition, the moving image encoding method of the present invention can have any one of the following configurations (1) to (11).

  (1) A method of encoding each picture constituting a moving picture with a frame structure or a field structure, and calculating a motion vector for each block constituting the picture with reference to the already coded picture A calculation step, a mode determination step for determining the encoding mode of the block to be processed, and the motion mode of an encoded picture whose encoding mode determined in the mode determination step is close in display time as a reference motion As a vector, in the case of an encoding mode in which the motion vector of the processing target block is predicted and generated by performing scaling processing of the reference motion vector based on the display temporal positional relationship between reference pictures, the processing target block Whether or not a motion vector can be predicted and generated A scaling judgment step, based on the scaling judgment step of judging the result, the motion compensation step of performing motion compensation by using the coding mode determined by said mode determining step it is or Update.

  (2) In the scaling determination step, when the display order information included in the two pictures referred to in the scaling process is the same, the scaling process is performed to predict and generate a motion vector of the processing target block. Judge that it is not possible.

  (3) In the scaling determination step, when the two pictures referred to in the scaling process are a top field and a bottom field belonging to the same frame, and both the two fields have the same display order information, It is determined that the motion vector of the processing target block cannot be predicted and generated by performing the scaling process.

  (4) In the motion compensation step, when it is determined that the motion vector cannot be generated in the scaling determination step, encoding is performed using the motion vector of the processing target block calculated in the motion vector calculation step. Change to the encoding mode to perform motion compensation.

  (5) In the motion compensation step, when it is determined that the motion vector cannot be generated in the scaling determination step, the prediction is generated for the processing target block without performing the scaling processing. Motion compensation is performed using the coding mode determined in the mode determination step as a vector of a predetermined value set in advance.

  (6) When at least one of the predetermined vectors is a 0 vector, and the motion compensation step determines that the motion vector cannot be generated in the scaling determination step, the scaling processing is not performed. In addition, at least one of the motion vectors generated by the prediction of the processing target block is set as a 0 vector, and motion compensation is performed using the coding mode determined in the mode determination step.

  (7) In the motion compensation step, when it is determined in the scaling determination step that the motion vector cannot be generated, the motion of an already encoded block located in the spatial periphery of the processing target block Based on the vector, motion compensation is performed by changing to a coding mode in which the motion vector of the processing target block is predicted and generated and coded.

  (8) A method of encoding each picture constituting a moving picture with a frame structure or a field structure, and calculating a motion vector for each block constituting the picture with reference to the already coded picture A motion vector referring to an already encoded picture that is closest to the processing target picture in display time among the motion vectors of the already encoded block located in the spatial vicinity of the processing target block A prediction determination step for determining whether or not the motion vector of the processing target block can be predicted and generated when the motion vector of the processing target block is generated based on the prediction, and the prediction If it is determined in the determination step that the motion vector cannot be generated, the processing target And a recent picture determination step of determining by using information other than display order information picture to nearest from puncturing.

  (9) In the prediction determination step, among the motion vectors of the already encoded block, there are a plurality of motion vectors referring to an already encoded picture that is closest in display time from the processing target picture. The motion of the processing target block when there are a top field and a bottom field belonging to the same frame, and both of the two fields have the same display order information. It is determined that a vector cannot be predicted and generated.

  (10) In the nearest picture determination step, when it is determined in the prediction determination step that the motion vector cannot be generated, among the top field and the bottom field belonging to the same frame and having the same display order information, A field having the same attribute as the processing target field is determined as a field closest to the processing target field.

  (11) In the nearest picture determination step, when it is determined in the prediction determination step that the motion vector cannot be generated, among the top field and the bottom field belonging to the same frame and having the same display order information, A field encoded later is determined as a field closest to the processing target field.

  Further, the moving picture decoding method of the present invention can have any one of the following configurations (12) to (22).

  (12) A method of decoding each picture constituting a moving picture with a frame structure or a field structure, wherein a motion vector is calculated for each block constituting the picture with reference to the already decoded picture A calculation step, a mode extraction step for extracting a decoding mode of the block to be processed, and a decoding mode extracted in the mode extraction step with reference to a motion vector of a decoded picture that is close in display time As a vector, when the decoding target mode predicts and generates a motion vector of the processing target block by performing scaling processing of the reference motion vector based on a display temporal positional relationship between reference pictures, the processing target block Whether the motion vector can be predicted and generated And scaling determining step that, based on the scaling judgment step of determination results, and a motion compensation step of performing motion compensation by using the decoding mode extracted in the mode extraction step as it is or Update.

  (13) In the scaling determination step, when the display order information included in the two pictures referred to in the scaling processing is the same, the scaling processing is performed to predict and generate a motion vector of the processing target block. Judge that it is not possible.

  (14) In the scaling determination step, when the two pictures referred to in the scaling process are a top field and a bottom field belonging to the same frame, and both the two fields have the same display order information, It is determined that the motion vector of the processing target block cannot be predicted and generated by performing the scaling process.

  (15) In the motion compensation step, when it is determined that the motion vector cannot be generated in the scaling determination step, decoding is performed using the motion vector of the processing target block calculated in the motion vector calculation step. Change to the decoding mode to perform motion compensation.

  (16) In the motion compensation step, when it is determined that the motion vector cannot be generated in the scaling determination step, the prediction is generated for the processing target block without performing the scaling processing. Motion compensation is performed using the decoding mode extracted in the mode extraction step as a vector having a predetermined value set in advance.

  (17) At least one of the predetermined vectors is a zero vector, and the scaling process is not performed in the motion compensation step when it is determined in the scaling determination step that the motion vector cannot be generated. Furthermore, motion compensation is performed using the decoding mode extracted in the mode extraction step with at least one of the motion vectors predicted and generated for the processing target block as a 0 vector.

  (18) In the motion compensation step, when it is determined in the scaling determination step that the motion vector cannot be generated, the motion of an already decoded block located in the spatial periphery of the processing target block Based on the vector, motion compensation is performed by changing to a decoding mode in which a motion vector of the processing target block is predicted and generated and decoded.

  (19) A method of decoding each picture constituting a moving picture with a frame structure or a field structure, wherein a motion vector is calculated for each block constituting the picture with reference to the already decoded picture Of the motion vectors of the already decoded block located in the spatial vicinity of the processing target block and the calculation target block, the motion vector referring to the already decoded picture that is closest in display time from the processing target picture A prediction determination step for determining whether or not the motion vector of the processing target block can be predicted and generated when the motion vector of the processing target block is generated based on the prediction, and the prediction When it is determined in the determination step that the motion vector cannot be generated, the processing pair Recent determined using information other than display order information picture to nearest from the picture and a picture determining step.

  (20) In the prediction determination step, among the motion vectors of the already decoded block, there are a plurality of motion vectors referring to the already decoded picture that is closest in display time to the processing target picture. The motion of the processing target block when there are a top field and a bottom field belonging to the same frame, and both of the two fields have the same display order information. It is determined that a vector cannot be predicted and generated.

  (21) In the nearest picture determination step, when it is determined in the prediction determination step that the motion vector cannot be generated, a top field and a bottom field belonging to the same frame and having the same display order information The field having the same attribute as the processing target field is determined as the field closest to the processing target field.

  (22) In the nearest picture determination step, when it is determined in the prediction determination step that the motion vector cannot be generated, a top field and a bottom field belonging to the same frame and having the same display order information The field decoded later is determined as the field closest to the processing target field.

  As is apparent from the above description, according to the moving picture encoding method of the present invention, when encoding is performed in the temporal direct mode or the spatial direct mode, a motion vector is reliably generated and a processing target block is generated. Can be encoded.

  Further, according to the moving picture decoding method according to the present invention, when decoding is performed in the temporal direct mode or the spatial direct mode, it is possible to reliably generate a motion vector and decode the processing target block. .

  Embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an embodiment of a moving picture coding apparatus using a moving picture coding method according to the present invention.

  As shown in FIG. 1, the moving image encoding apparatus includes a picture memory 101, a prediction residual encoding unit 102, a code sequence generation unit 103, a prediction residual decoding unit 104, a picture memory 105, a motion vector detection unit 106, a motion A compensation encoding unit 107, a motion vector storage unit 108, a direct mode availability determination unit 109, a difference calculation unit 110, an addition calculation unit 111, and switches 112 and 113 are provided.

  The picture memory 101 stores moving images input in units of pictures in order of display time. The motion vector detection unit 106 uses the encoded reconstructed image data as a reference picture, and detects a motion vector indicating a position predicted to be optimal in the search area in the picture. The motion compensation encoding unit 107 determines a block encoding mode using the motion vector detected by the motion vector detection unit 106, and generates predicted image data based on the encoding mode. This encoding mode indicates how the macroblock is encoded.

  The motion vector storage unit 108 stores the motion vector detected by the motion vector detection unit 106. The direct mode enable / disable determining unit 109 determines whether or not the scaling process can be performed when the encoding mode determined by the motion compensation encoding unit 107 is the temporal direct mode. Confirm. Further, the direct mode availability determination unit 109 determines whether or not a motion vector used in an encoding target block can be predicted and generated when the encoding mode is a spatial direct mode. The difference calculation unit 110 calculates a difference between the image data read from the picture memory 101 and the prediction image data input from the motion compensation encoding unit 107, and generates prediction residual image data.

  The prediction residual encoding unit 102 performs encoding processing such as frequency conversion and quantization on the input prediction residual image data to generate encoded data. The code string generation unit 103 performs variable-length encoding or the like on the input encoded data, and adds motion vector information input from the motion compensation encoding unit 107, encoding mode information, and the like. Thus, a code string is generated.

  The prediction residual decoding unit 104 performs decoding processing such as inverse quantization and inverse frequency transform on the input encoded data, and generates decoded differential image data. The addition operation unit 111 adds the decoded differential image data input from the prediction residual decoding unit 104 and the predicted image data input from the motion compensation encoding unit 107 to generate reconstructed image data. The picture memory 105 stores the generated reconstructed image data.

Next, the operation of the moving picture coding apparatus configured as described above will be described.
FIG. 2 is an explanatory diagram showing the order of pictures in the picture memory 101, and is an explanatory diagram showing (a) the input order and (b) the rearranged order. Here, the vertical line indicates a picture, and the symbol shown at the lower right of each picture is that the first alphabet is the picture type (I, P, or B), and the second and subsequent numbers are picture numbers in display time order. Is shown. A P picture uses a neighboring I picture or P picture in front in display time order as a reference picture, and a B picture has a neighboring I picture or P picture in front in display time order and rear in display time order. One neighboring I picture or P picture is used as a reference picture.

  For example, as shown in FIG. 2A, the input image is input to the picture memory 101 in units of pictures in order of display time. When the picture type to be encoded is determined, the pictures input to the picture memory 101 are rearranged in the order in which they are encoded, for example, as shown in FIG. This rearrangement to the coding order is performed based on the reference relationship in the inter-picture predictive coding, and the picture used as the reference picture is rearranged so that it is coded before the picture used as the reference picture. .

  Each picture that has been rearranged in the picture memory 101 is read in units of macroblocks divided into groups of horizontal 16 × vertical 16 pixels, for example. Also, motion compensation and motion vector detection are performed in units of blocks divided into groups of, for example, horizontal 8 × vertical 8 pixels.

  The subsequent operation will be described in the case where the picture to be encoded is a B picture.

  For B pictures, inter-picture prediction coding using two-way reference is performed. For example, when encoding the picture B11 in the example shown in FIG. 2 (a), the reference pictures ahead in the display time order are the pictures P10, P7, P4, and the reference pictures behind the display time order are the picture P13. It becomes. Here, consider a case where a B picture is not used as a reference picture when another picture is encoded.

  The macroblock of the picture B11 read from the picture memory 101 is input to the motion vector detection unit 106 and the difference calculation unit 110.

  The motion compensation encoding unit 107 determines whether each block in the macroblock is encoded with a frame structure or a field structure. For encoding with the frame structure or the field structure, for example, there is a method of obtaining the variance of the pixel values in the block between the frame structure and the field structure and selecting the one with the smaller variance. Each picture can be encoded in either a frame structure or a field structure in units of pictures.

  The motion vector detection unit 106 uses each reference picture stored in the picture memory 105 as a frame unit or a field unit according to the determined frame structure encoding or field structure encoding, and blocks each block in the macroblock. In contrast, a forward motion vector and a backward motion vector are detected. Here, the reconstructed image data of the pictures P10, P7, and P4 stored in the picture memory 105 is used as a forward reference picture, and the reconstructed image data of the picture P13 is used as a backward reference picture. The motion vector detection unit 106 outputs the detected motion vector to the motion compensation encoding unit 107.

  The motion compensation encoding unit 107 determines a macroblock encoding mode using the motion vector detected by the motion vector detection unit 106. Here, the encoding mode of the B picture is, for example, intra-picture encoding, inter-picture predictive encoding using a forward motion vector, inter-picture predictive encoding using a backward motion vector, and inter-picture using a bi-directional motion vector. It is assumed that it is possible to select which method to encode from predictive encoding and direct mode. As for the direct mode, it is assumed that the temporal direct mode or the spatial direct mode is designated in advance. In determining the encoding mode, generally, a method is selected in which the encoding error is reduced with a small amount of bits.

  Next, an operation of determining the encoding mode by the direct mode availability determination unit 109 performed when encoding in the direct mode is selected will be described. The operation of determining the encoding mode can be performed by any one of methods 1 to 3 described below.

(Method 1)
FIG. 3 is a flowchart showing the operation of determining the encoding mode by the method 1. When the motion compensation encoding unit 107 selects to perform encoding in the direct mode, the motion compensation encoding unit 107 notifies the direct mode availability determination unit 109 to that effect. The direct mode availability determination unit 109 that has received the notification first determines whether or not the temporal direct mode is designated (step S101). As a result, when it is determined that the temporal direct mode is selected, the direct mode availability determination unit 109 determines whether or not field coding is selected (step S102). As a result, when it is determined that field coding is not selected, the direct mode availability determination unit 109 instructs the motion compensation coding unit 107 to perform coding in the temporal direct mode (step S103). . On the other hand, if it is determined that the field encoding is selected, the direct mode availability determination unit 109 determines whether the motion vector used in the encoding target block can be predicted and generated by the scaling process. (Step S104). That is, it is determined whether or not two pictures to be referenced belong to the same frame and are a top field and a bottom field having the same display order information. As a result, when scaling processing is possible (when the condition determination in step S104 is NO), the direct mode availability determination unit 109 instructs the motion compensation encoding unit 107 to perform encoding in the temporal direct mode. (Step S103). On the other hand, when the scaling process is not possible (when the condition determination in step S104 is YES), the direct mode availability determination unit 109 instructs the motion compensation encoding unit 107 to perform encoding in a mode other than the direct mode (step S105). ).

  Also, as a result of the determination (step S101), even when it is determined that the mode is not the temporal direct mode (that is, the spatial direct mode), the direct mode availability determination unit 109 determines whether or not field coding is selected. Is determined (step S106). As a result, when it is determined that the field encoding is not selected, the direct mode availability determination unit 109 instructs the motion compensation encoding unit 107 to perform encoding in the spatial direct mode (step S107). .

  As a result of the determination (step S106), when it is determined that field encoding is selected, the direct mode availability determination unit 109 performs encoding based on the display order information of the picture in the spatial direct mode. It is determined whether or not the motion vector used in the block can be predicted and generated (step S108). That is, of the motion vectors of the three encoded blocks including the three pixels around the block to be encoded, the motion vectors closest to the encoding target picture (field) in terms of display time have already been encoded. It is determined whether or not there are a plurality of motion vectors referring to a picture, and the plurality of referenced pictures belong to the same frame and are a top field and a bottom field having the same display order information. At this time, when the above condition is satisfied, it is determined that a motion vector cannot be predicted and generated.

  As a result of the determination (step S108), when it is determined that a motion vector can be predicted and generated (when the condition determination in step S108 is NO), the direct mode propriety determination unit 109 performs motion compensation encoding. The unit 107 is instructed to encode in the spatial direct mode (step S107).

  On the other hand, when it is determined that a motion vector cannot be predicted and generated (when the condition determination in step S108 is YES), the direct mode availability determination unit 109 determines whether the top field and the bottom field have the same display order information. Of these, the motion compensation encoding unit 107 is instructed to set the field having the same attribute as the encoding target field to be the field closest to the encoding target field in display time (step S109). Here, the field having the same attribute is a top field if the encoding target field is a top field, and a bottom field if the encoding target field is a bottom field. In addition, the direct mode availability determination unit 109 instructs the motion compensation encoding unit 107 to perform encoding in the spatial direct mode (step S107).

(Method 2)
FIG. 4 is a flowchart showing the operation of determining the encoding mode by the method 2. Note that, since it is determined that the field encoding is selected and the processing other than the processing when it is determined that the scaling processing is not possible (steps S201 to S204, S206 to S209), the description is omitted. To do.

  If it is determined that the field coding is selected and further scaling processing is not possible, the direct mode availability determination unit 109 sets the motion vector to “0” with respect to the motion compensation encoding unit 107. The encoding in the direct mode is instructed (step S205).

(Method 3)
FIG. 5 is a flowchart showing the operation of determining the coding mode by the method 3. Since processes other than the process when it is determined that the field encoding is selected and the scaling process is not possible (steps S301 to S306 and S308) are the same as the method 1, the description thereof is omitted.

  When it is determined that field coding is selected and further scaling processing is not possible, the direct mode availability determination unit 109 can predict and generate a motion vector used in a block to be encoded in the spatial direct mode. It is determined whether or not (step S307). Subsequent operations are the same as those in Method 1.

  In the above methods 1 to 3, when it is determined that the motion vector cannot be predicted and generated in the spatial direct mode (steps S109, S209, and S308), the following processing is performed. To obtain the methods 1 ′ to 3 ′. FIG. 6 is a flowchart showing the operation of determining the encoding mode by the method 1 '. Since the methods 2 'to 3' are the same as the method 1 ', description and illustration are omitted.

(Method 1 ')
The direct mode propriety determination unit 109 selects a field encoded later among the top field and the bottom field having the same display order information (that is, a field encoded closest in time from the encoding target field). The motion compensation encoding unit 107 is instructed to set the field closest to the display time from the encoding target field (FIG. 6, step S110).

  Next, the motion compensation encoding unit 107 generates predicted image data in the encoding mode determined by the direct mode availability determination unit 109 as described above. The operation will be described below for each determined encoding mode.

(Encoding in normal temporal direct mode)
In this case, the motion compensation encoding unit 107 performs motion compensation using the same method as in the temporal direct mode described with reference to FIG. 18 in the background art. That is, a motion vector of a block at the same position as the block to be encoded in the encoded picture is used as a reference motion vector, and the motion compensation encoding unit 107 uses the reference motion vector as a motion vector storage unit 108. More specifically, a scaling process is performed based on the reference motion vector and the display temporal positional relationship between pictures to predict and generate a motion vector to be used in the encoding target block. Then, using this motion vector, the motion compensation encoding unit 107 performs bi-directional prediction from two reference pictures to generate predicted image data.

(Encoding in temporal direct mode with motion vector set to “0”)
The motion compensation encoding unit 107 generates prediction image data by performing bi-directional prediction from two reference pictures using “0” as a direct motion vector without performing motion vector prediction generation by scaling processing. .

  The value of the motion vector used at this time is not limited to “0”, and may be a predetermined value that can be determined without requiring scaling. In the above example, both motion vectors for two reference pictures are described as “0”. However, the present invention is not limited to this, and at least one of the motion vectors for two reference pictures is “0”. It is good also as.

(Encoding in other than direct mode)
The motion compensation encoding unit 107 performs bi-directional prediction from two reference pictures using the motion vector detected by the motion vector detection unit 106 to generate predicted image data.

(Encoding in spatial direct mode)
In this case, the motion compensation encoding unit 107 performs motion compensation using the same method as the spatial direct mode described with reference to FIG. 19 in the background art. That is, among the motion vectors of the three encoded blocks including the three pixels around the block to be encoded, refer to the already encoded picture that is closest in display time from the encoding target picture. The motion vector used in the encoding target block is predicted and generated using the motion vector.

  At this time, among the motion vectors of the three blocks, there are a plurality of motion vectors that refer to already encoded pictures that are closest in display time from the encoding target picture (field), and the reference When the plurality of pictures belong to the same frame and are the top field and the bottom field having the same display order information, the motion compensation encoding unit 107 uses the same display order information based on an instruction from the direct mode availability determination unit 109 One of the top field and the bottom field having the field is the field closest to the encoding target field in terms of display time.

  That is, when the instruction from the direct mode availability determination unit 109 is the instruction described in the methods 1 to 3, the same attribute as that of the encoding target field among the top field and the bottom field having the same display order information is used. A certain field is a field closest to the encoding target field in display time. For example, in the example shown in FIG. 23, the field P2_T that is the same top field as the encoding target field B3_T among the fields P2_T and P2_B having the same display order information is closest to the encoding target field in terms of display time. A field is assumed. Therefore, the motion vector MVA1 referring to the field P2_T is determined as the first motion vector candidate of the encoding target block.

  In addition, when the instruction from the direct mode availability determination unit 109 is the instruction described in the methods 1 ′ to 3 ′, the top field and the bottom field having the same display order information are encoded later. The field is the field closest to the encoding target field in terms of display time. For example, in the example shown in FIG. 23, if the field P2_B among the fields P2_T and P2_B is encoded later, the field P2_B encoded later among the fields P2_T and P2_B having the same display order information is encoded. The field closest to the target field in terms of display time. Therefore, the motion vector MVC1 referring to the field P2_B is determined as a candidate for the first motion vector MV_F of the encoding target block. The same applies to obtaining MV_B, which is the second motion vector.

  When there are three motion vectors determined in this way, the median value is selected as the motion vector of the encoding target block. If there are two, the average value of them is obtained and used as the motion vector of the encoding target block. If there is only one (example shown in FIG. 23), the motion vector is set as the motion vector of the encoding target block. Using the motion vector obtained in this way, the motion compensation encoding unit 107 performs motion compensation from the reference picture to generate predicted image data.

  Next, the motion compensation encoding unit 107 outputs the predicted image data generated as described above to the difference calculation unit 110 and the addition calculation unit 111. Note that, when the motion compensation encoding unit 107 selects intra-picture encoding, the predicted image data is not output. The motion compensation encoding unit 107 connects the switch 112 to the side to which the signal is directly input from the picture memory 101 when the intra-picture encoding is selected, and when the inter-picture predictive encoding is selected. The switch 112 is controlled to be connected to the signal input side from the difference calculation unit 110. In addition, the motion compensation encoding unit 107 outputs the determined encoding mode to the code string generation unit 103.

  The difference calculation unit 110 to which the prediction image data is input from the motion compensation encoding unit 107 calculates a difference between the prediction image data and the image data of the macroblock of the picture B11 read from the picture memory 101, and performs prediction. Residual image data is generated and output to the prediction residual encoding unit 102.

  The prediction residual encoding unit 102 to which the prediction residual image data is input performs encoding processing such as frequency conversion and quantization on the prediction residual image data, generates encoded data, and generates a code string. Output to the unit 103. The code string generation unit 103 to which the encoded data is input performs variable-length encoding or the like on the encoded data, and further, motion vector information and encoding mode information input from the motion compensation encoding unit 107 Etc., a code string is generated and output. For macroblocks encoded in the direct mode, motion vector information is not added to the encoded sequence.

  Thereafter, the same processing is performed for the remaining macroblocks of the picture B11.

  As described above, when field encoding is selected and encoding is performed in the temporal direct mode, it is determined whether or not scaling processing is possible. When it is determined that the scaling process is not possible, the process such as changing the encoding mode is performed, so that the scaling process cannot be performed and the encoding cannot be performed.

  In addition, when field coding is selected and coding is performed in the spatial direct mode, can a motion vector used in a block to be coded be predicted and generated based on display order information of a picture? Judgment of whether or not. If it is determined that a motion vector cannot be predicted and generated, which of the top field and the bottom field having the same display order information is closest to the encoding target field in terms of display time. Therefore, there is no possibility that a motion vector cannot be predicted and generated and encoding cannot be performed.

  FIG. 7 is a block diagram showing a configuration of an embodiment of a moving picture decoding apparatus using the moving picture decoding method according to the present invention.

  The moving picture decoding apparatus includes a code stream analysis unit 201, a prediction residual decoding unit 202, a picture memory 203, a motion compensation decoding unit 204, a motion vector storage unit 205, a direct mode availability determination unit 206, an addition operation unit 207, And a switch 208.

  The code string analysis unit 201 extracts various data such as decoding mode information and motion vector information used at the time of coding from the input code string. The prediction residual decoding unit 202 decodes the input prediction residual encoded data, and generates prediction residual image data. The motion compensation decoding unit 204 generates motion compensated image data based on decoding mode information, motion vector information, and the like. The motion vector storage unit 205 stores the motion vector extracted by the code string analysis unit 201.

  The direct mode availability determination unit 206 determines whether or not the scaling process can be performed when the decoding mode extracted by the code stream analysis unit 201 is the temporal direct mode, and determines the decoding mode. I do. In addition, the direct mode availability determination unit 206 determines whether or not a motion vector used in a decoding target block can be predicted and generated when the decoding mode is a spatial direct mode. The addition operation unit 207 adds the prediction residual image data input from the prediction residual decoding unit 202 and the motion compensated image data input from the motion compensation decoding unit 204 to generate decoded image data. . The picture memory 203 stores the generated decoded image data.

  Next, the operation of the moving picture decoding apparatus configured as described above will be described. Note that the order of pictures will be described with reference to FIG. Here, a P picture uses a neighboring I picture or P picture ahead in display time order as a reference picture, and a B picture has a neighboring I picture or P picture ahead in display time order and display time order. It is assumed that encoding is performed using one I picture or P picture in the back as a reference picture.

  The code string is input to the code string analyzer 201 in the picture order as shown in FIG. The code string analysis unit 201 extracts various data such as decoding mode information and motion vector information from the input code string. The code string analysis unit 201 outputs the extracted decoding mode information to the motion compensation decoding unit 204 and the motion vector information to the motion vector storage unit 205.

  Further, the code string analysis unit 201 outputs the extracted prediction residual encoded data to the prediction residual decoding unit 202. The prediction residual decoding unit 202 to which the prediction residual encoded data is input decodes the prediction residual encoded data, generates prediction residual image data, and outputs the prediction residual image data to the addition calculation unit 207.

  Regarding the subsequent operations, a case will be described in which the picture to be decoded is a B picture and the decoding mode extracted by the code stream analysis unit 201 is the direct mode.

  The motion compensation decoding unit 204 to which the decoding mode information is input from the code stream analysis unit 201 determines whether or not to decode the decoding target block in the direct mode. The determination unit 206 is notified.

  Next, the operation of determining the decoding mode by the direct mode availability determination unit 206 performed when the decoding mode is the direct mode will be described. The operation of determining the decoding mode can be performed by any one of methods 1 to 3 described below.

(Method 1)
FIG. 8 is a flowchart showing the operation of determining the decoding mode by the method 1. First, the direct mode availability determination unit 206 determines whether or not the temporal direct mode is designated (step S401). As a result, when it is determined that the temporal direct mode is selected, the direct mode availability determination unit 206 determines whether or not field coding is performed (step S402). As a result, if it is determined that field coding is not performed, the direct mode availability determination unit 206 instructs the motion compensation decoding unit 204 to perform decoding in the temporal direct mode (step S403). . On the other hand, when it is determined that the field coding is selected, the direct mode availability determination unit 206 determines whether or not the motion vector used in the decoding target block can be predicted and generated by the scaling process. (Step S404). That is, it is determined whether or not two pictures to be referenced belong to the same frame and are a top field and a bottom field having the same display order information. As a result, when the scaling process is possible (when the condition determination in step S404 is NO), the direct mode availability determination unit 206 instructs the motion compensation decoding unit 204 to perform decoding in the temporal direct mode. (Step S403). On the other hand, when the scaling process is not possible (when the condition determination in step S404 is YES), the direct mode availability determination unit 206 instructs the motion compensation decoding unit 204 to perform decoding in a mode other than the direct mode (step S405). ).

  Also, as a result of the determination (step S401), if it is determined that the mode is not the temporal direct mode (that is, the spatial direct mode), the direct mode propriety determination unit 206 determines whether or not field encoding is performed. Is determined (step S406). As a result, when it is determined that field coding is not selected, the direct mode availability determination unit 206 instructs the motion compensation decoding unit 204 to perform decoding in the spatial direct mode (step S407). .

  As a result of the determination (step S406), if it is determined that field coding is selected, the direct mode availability determination unit 206 performs decoding based on the display order information of the pictures in the spatial direct mode. It is determined whether or not the motion vector used in the block can be predicted and generated (step S408). That is, among the motion vectors of the three decoded blocks including the three pixels around the block that is the decoding target, the motion vectors that are closest to the decoding target picture (field) in terms of display time have already been decoded. It is determined whether or not there are a plurality of motion vectors referring to a picture, and the plurality of referenced pictures belong to the same frame and are a top field and a bottom field having the same display order information. At this time, when the above condition is satisfied, it is determined that a motion vector cannot be predicted and generated.

  As a result of the determination (step S408), when it is determined that a motion vector can be predicted and generated (when the condition determination in step S408 is NO), the direct mode availability determination unit 206 performs motion compensation decoding. The unit 204 is instructed to perform decoding in the spatial direct mode (step S407).

  On the other hand, when it is determined that the motion vector cannot be predicted and generated (when the condition determination in step S408 is YES), the direct mode availability determination unit 206 determines whether the top field and the bottom field have the same display order information. Of these, the motion compensation decoding unit 204 is instructed so that the field having the same attribute as the decoding target field is the field closest to the decoding target field in display time (step S409). Here, the field having the same attribute is a top field if the decoding target field is a top field, and a bottom field if the decoding target field is a bottom field. After that, the direct mode availability determination unit 206 instructs the motion compensation decoding unit 204 to perform decoding in the spatial direct mode (step S407).

(Method 2)
FIG. 9 is a flowchart showing the operation of determining the decoding mode by the method 2. Note that the processing other than the processing when it is determined that field coding is selected and further scaling processing is not possible (steps S501 to S504 and S506 to S509) is the same as method 1, and thus the description thereof is omitted. To do.

  When it is determined that the field coding is selected and it is further determined that the scaling process is not possible, the direct mode availability determination unit 206 sets the motion vector to “0” for the motion compensation decoding unit 204. Is instructed to be decrypted in a direct mode (step S505).

(Method 3)
FIG. 10 is a flowchart showing the operation of determining the decoding mode by the method 3. Since processes other than the process when it is determined that the field encoding is selected and the scaling process is not possible (steps S601 to S606 and S608) are the same as the method 1, the description thereof is omitted.

  If it is determined that field coding is selected and further scaling processing is not possible, the direct mode availability determination unit 206 can predict and generate a motion vector used in the decoding target block in the spatial direct mode. It is determined whether or not (step S607). Subsequent operations are the same as those in Method 1.

  In addition, in the above methods 1 to 3, when it is determined that a motion vector cannot be predicted and generated in the spatial direct mode (steps S409, S509, and S608), the following processing is performed. To obtain the methods 1 ′ to 3 ′. FIG. 11 is a flowchart showing the operation of determining the decoding mode by the method 1 '. Since the methods 2 'to 3' are the same as the method 1 ', description and illustration are omitted.

(Method 1 ')
The direct mode propriety determination unit 206 selects a field that is decoded later (that is, a field that is decoded closest in time from the decoding target field) among the top field and the bottom field that have the same display order information. The motion compensation decoding unit 204 is instructed to set the field closest to the display time from the field to be decoded (step S410 in FIG. 11).

  Next, the motion compensation decoding unit 204 generates motion compensation image data in the decoding mode determined by the direct mode availability determination unit 206 as described above. The operation will be described below for each determined decoding mode.

(Decryption in normal temporal direct mode)
In this case, the motion compensation decoding unit 204 performs motion compensation using the same method as in the temporal direct mode described with reference to FIG. 18 in the background art. That is, a motion vector of a block in the same position as the decoding target block in the decoded picture is used as a reference motion vector, and the motion compensation decoding unit 204 uses this reference motion vector as a motion vector storage unit 205. More specifically, a scaling process is performed on the basis of the reference motion vector and the display temporal positional relationship between pictures, and a motion vector used in the decoding target block is predicted and generated. Then, using this motion vector, the motion compensation decoding unit 204 performs bidirectional prediction from two reference pictures to generate motion compensated image data.

(Decoding in temporal direct mode with motion vector set to “0”)
The motion compensation decoding unit 204 generates prediction image data by performing bi-directional prediction from two reference pictures using “0” as a direct motion vector without performing motion vector prediction generation by scaling processing. .

  The value of the motion vector used at this time is not limited to “0”, and may be a predetermined value that can be determined without requiring scaling. In the above example, both motion vectors for two reference pictures are described as “0”. However, the present invention is not limited to this, and at least one of the motion vectors for two reference pictures is “0”. It is good also as.

(Decryption in non-direct mode)
The motion compensation decoding unit 204 reads out the motion vector used at the time of encoding from the motion vector storage unit 205, performs bi-directional prediction from two reference pictures using this motion vector, and generates motion compensated image data To do.

(Decryption in spatial direct mode)
In this case, the motion compensation decoding unit 204 performs motion compensation using the same method as that in the spatial direct mode described with reference to FIG. 19 in the background art. That is, refer to the already decoded picture that is closest to the decoding target picture in display time among the motion vectors of the three decoded blocks including the three pixels around the decoding target block. The motion vector used in the encoding target block is predicted and generated using the motion vector.

  At this time, among the motion vectors of the three blocks, there are a plurality of motion vectors that refer to already decoded pictures that are closest in display time to the decoding target picture (field), and the reference When a plurality of pictures belong to the same frame and are a top field and a bottom field having the same display order information, the motion compensation decoding unit 204 uses the same display order information based on an instruction from the direct mode availability determination unit 206 One of the top field and the bottom field having a field is the field closest to the decoding target field in terms of display time.

  That is, when the instruction from the direct mode availability determination unit 206 is the instruction described in the above methods 1 to 3, the field having the same attribute as the decoding target field among the top field and the bottom field having the same display order information. Is the field closest to the decoding target field in terms of display time. For example, in the example shown in FIG. 23, the field P2_T that is the same top field as the decoding target field B3_T among the fields P2_T and P2_B having the same display order information is closest to the decoding target field in terms of display time. A field is assumed. Therefore, the motion vector MVA1 referring to the field P2_T is determined as the first motion vector candidate of the decoding target block.

  In addition, when the instruction from the direct mode availability determination unit 206 is the instruction described in the methods 1 ′ to 3 ′, a field decoded later among the top field and the bottom field having the same display order information is displayed. The field closest to the decoding target field in terms of display time. For example, in the example shown in FIG. 23, if the field P2_B of the fields P2_T and P2_B is decoded later, the decoded field P2_B of the fields P2_T and P2_B having the same display order information is decoded. The field closest to the target field in terms of display time. Therefore, the motion vector MVC1 referring to the field P2_B is determined as a candidate for the first motion vector MV_F of the decoding target block. The same applies to obtaining MV_B, which is the second motion vector.

  When there are three motion vectors determined in this way, the median value is selected as the motion vector of the decoding target block. If there are two, the average value of them is obtained and used as the motion vector of the decoding target block. If there is only one (example shown in FIG. 23), the motion vector is set as the motion vector of the decoding target block. Using the motion vector obtained in this way, the motion compensation decoding unit 204 performs motion compensation from the reference picture to generate motion compensated image data.

  Next, the motion compensation decoding unit 204 outputs the motion compensated image data (block) generated as described above to the addition operation unit 207. The addition operation unit 207 adds the motion compensated image data and the prediction residual image data input from the prediction residual decoding unit 202, generates decoded image data, and stores the decoded image data in the picture memory 203.

  Thereafter, the decoding process is performed for the remaining macroblocks of the picture B11 by the same process. In the example shown in FIG. 2B, when the processing is completed for all the macroblocks of the picture B11, the decoding processing of the picture B12 is performed next. The pictures decoded as described above are sequentially output from the picture memory 203 as output images as shown in FIG.

  As described above, when field coding is selected and the extracted decoding mode is the temporal direct mode, it is determined whether or not the scaling process is possible. When it is determined that the scaling process is not possible, the process such as changing the decoding mode is performed, so that the scaling process cannot be performed and the decoding cannot be performed.

  In addition, when field coding is selected and the extracted decoding mode is the spatial direct mode, a motion vector used in the block to be encoded is predicted and generated based on the display order information of the picture. It is determined whether or not If it is determined that the motion vector cannot be predicted and generated, which of the top field and the bottom field having the same display order information is closest to the decoding target field in terms of display time. Therefore, there is no possibility that a motion vector cannot be predicted and generated and decoding cannot be performed.

  In this embodiment, at the time of encoding in the spatial direct mode, the motion compensation encoding unit 107 performs the motion of each of the three encoded blocks including the three pixels around the block to be encoded. Among the vectors, when determining motion vector candidates for the encoding target block, a motion vector that refers to an already encoded picture that is closest in display time to the encoding target picture is determined as a candidate. However, it is not limited to this. For example, at the time of field encoding, a motion vector that refers to a field that is closest to the encoding target field in terms of display time from among the fields having the same attributes as the encoding target field may be determined as a candidate. In this case, the present embodiment first determines candidates based on the display order information, but first determines the candidates by giving priority to fields having the same attributes. The same applies to the operation of the motion compensation decoding unit 204 at the time of decoding.

  In the present embodiment, each picture has been described as being adaptively encoded and decoded using either a frame structure or a field structure. Even if encoding or decoding processing is adaptively performed using any of the structures, it can be performed by processing similar to the present invention, and the same effect can be obtained.

  In the present embodiment, the P picture has been described as a picture that is processed with reference to a picture in one front direction, and the B picture has been described as a picture that is processed with reference to two pictures in the front and rear directions. Even if the P picture is processed with reference to a picture in one backward direction and the B picture is processed with reference to a picture in two forward or two backward directions, the same effect can be obtained.

  Note that the display order information in the embodiment of the present invention is not limited to the display order, but the actual display time or a predetermined picture whose value increases as the display time value increases. The relative order of each picture as a reference may be used.

(Embodiment 2)
Further, the program for realizing the configuration of the image encoding method or the image decoding method shown in the first embodiment is recorded on a storage medium such as a flexible disk, so that the first embodiment can be used. The illustrated processing can be easily performed in an independent computer system.

  FIG. 12 is an explanatory diagram in the case of implementing by a computer system using a flexible disk storing the image encoding method or image decoding method of the first embodiment.

  FIG. 12B shows an appearance, a cross-sectional structure, and a flexible disk as seen from the front of the flexible disk, and FIG. 12A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, an image encoding method as the program is recorded in an area allocated on the flexible disk FD.

  FIG. 12C shows a configuration for recording and reproducing the program on the flexible disk FD. When recording the program on the flexible disk FD, the image encoding method or the image decoding method as the program is written from the computer system Cs via the flexible disk drive. When the image encoding method is built in a computer system by a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

  Furthermore, application examples of the moving picture coding method and the moving picture decoding method shown in the above embodiment and a system using the same will be described.

  FIG. 13 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

  The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

  However, the content supply system ex100 is not limited to the combination as shown in FIG. 13, and any of the combinations may be connected. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the encoded data transmitted by the user using the camera ex113 can be performed. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. Further, the moving image data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images, such as a digital camera. In this case, the encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that moving image encoding / decoding software may be incorporated in any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile telephone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and also realize personal broadcasting by receiving, decoding, and reproducing in real time at the client. It is a system that becomes possible.

  For the encoding and decoding of each device constituting this system, the moving image encoding device or the moving image decoding device described in the above embodiments may be used.

A mobile phone will be described as an example.
FIG. 14 is a diagram illustrating the mobile phone ex115 that uses the moving picture coding method and the moving picture decoding method described in the above embodiment. The cellular phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera such as a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording medium ex207, and slot portion ex20 for enabling recording medium ex207 to be attached to mobile phone ex115 The has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the image coding for the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

  When the end call and power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital mobile phone ex115 by supplying power from the battery pack to each unit. .

  The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After the data is converted, it is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 is configured to include the moving image encoding device described in the present invention, and the image data supplied from the camera unit ex203 is a code used in the moving image encoding device described in the above embodiment. The encoded image data is converted into encoded image data by compression encoding using the encoding method, and is sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit A spectrum spread process is performed at ex306, a digital-analog conversion process and a frequency conversion process are performed at the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.

  When data of a moving image file linked to a homepage or the like is received in the data communication mode, the received data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Is sent to the demultiplexing unit ex308.

  In addition, in order to decode multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and synchronizes them. The encoded image data is supplied to the image decoding unit ex309 via the bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 is configured to include the moving image decoding apparatus described in the present invention, and a bit stream of image data is decoded by a decoding method corresponding to the encoding method described in the above embodiment. Reproduction moving image data is generated by decoding, and this is supplied to the display unit ex202 via the LCD control unit ex302, thereby displaying, for example, moving image data included in the moving image file linked to the homepage. . At the same time, the audio processing unit ex305 converts the audio data into analog audio data, and then supplies the analog audio data to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  It should be noted that the present invention is not limited to the above system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic. As shown in FIG. Any of the video decoding devices can be incorporated. Specifically, in the broadcasting station ex409, a bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the bitstream and reproduces it. In addition, the moving picture decoding apparatus described in the above embodiment can also be implemented in a playback apparatus ex403 that reads and decodes a bitstream recorded on a storage medium ex402 such as a CD or DVD as a recording medium. . In this case, the reproduced video signal is displayed on the monitor ex404. A configuration is also possible in which a moving picture decoding apparatus is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 on the television. At this time, the moving picture decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce the moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Furthermore, the image signal can be encoded by the moving image encoding apparatus shown in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder that records an image signal on a DVD disk ex421 or a disk recorder that records on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the moving picture decoding apparatus shown in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 in the configuration shown in FIG. 15 excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 can be considered, and the same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

  In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

  As described above, the moving picture encoding method or the moving picture decoding method described in the above embodiment can be used in any of the above-described devices and systems, and as a result, the above-described embodiment has been described. An effect can be obtained.

  In addition, the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the present invention.

  As described above, the moving image encoding method and the moving image decoding method according to the present invention generate a code string by encoding each picture constituting a moving image by, for example, a mobile phone, a DVD device, and a personal computer. Or a method for decoding the generated code string.

It is a block diagram which shows the structure of one Embodiment of the moving image encoder which concerns on this invention. It is explanatory drawing which shows the order of the picture in a picture memory, (a) The input order, (b) It is explanatory drawing which shows the rearranged order. It is a flowchart which shows the operation | movement of the determination of the encoding mode by the method 1 in a direct mode availability determination part. It is a flowchart which shows the operation | movement of the determination of the encoding mode by the method 2 in the direct mode availability determination part. It is a flowchart which shows the operation | movement of the determination of the encoding mode by the method 3 in the direct mode availability determination part. It is a flowchart which shows the operation | movement of determination of the encoding mode by the method 1 'in a direct mode availability determination part. It is a block diagram which shows the structure of one Embodiment of the moving image decoding apparatus which concerns on this invention. It is a flowchart which shows the operation | movement of determination of the decoding mode by the method 1 in a direct mode availability determination part. It is a flowchart which shows the operation | movement of determination of the decoding mode by the method 2 in the direct mode availability determination part. It is a flowchart which shows the operation | movement of determination of the decoding mode by the method 3 in the direct mode availability determination part. It is a flowchart which shows the operation | movement of determination of the decoding mode by the method 1 'in a direct mode availability determination part. It is explanatory drawing about the recording medium for storing the program for implement | achieving the moving image encoding method and moving image decoding method of Embodiment 1 with a computer system, (a) Of the flexible disk which is a recording medium main body An explanatory diagram showing an example of a physical format, (b) an external view from the front of the flexible disk, a sectional structure, and an explanatory diagram showing the flexible disk, and (c) a configuration for recording and reproducing the program on the flexible disk FD It is explanatory drawing which showed. It is a block diagram which shows the whole structure of the content supply system which implement | achieves a content delivery service. It is a figure which shows an example of a mobile telephone. It is a block diagram which shows the internal structure of a mobile telephone. It is a block diagram which shows the whole structure of the system for digital broadcasting. It is a schematic diagram which shows the prediction relationship of each picture in the conventional moving image encoding system. It is a schematic diagram which shows the prediction production | generation method of the motion vector in temporal direct mode. It is a schematic diagram which shows the prediction production | generation method of the motion vector in spatial direct mode. It is a schematic diagram which shows the display order information which the field in an interlace image and a progressive image has. It is a schematic diagram which shows the prediction production | generation method of the motion vector in the temporal direct mode in an interlaced image. It is a schematic diagram which shows the prediction production | generation method of the motion vector in the temporal direct mode in a progressive image. It is a schematic diagram which shows the prediction production | generation method of the motion vector in the spatial direct mode in a progressive image.

Explanation of symbols

101, 105, 203 Picture memory 102 Prediction residual encoding unit 103 Code sequence generation unit 104 Prediction residual decoding unit 106 Motion vector detection unit 107 Motion compensation encoding unit 108 Motion vector storage unit 109 Direct mode availability determination unit 110 Difference Calculation unit 111 Addition calculation unit 112, 113, 208 Switch 201 Code sequence analysis unit 202 Prediction residual decoding unit 204 Motion compensation decoding unit 205 Motion vector storage unit 206 Direct mode availability determination unit 207 Addition calculation unit

Claims (6)

An image encoding method for encoding a picture constituting an image,
Obtaining a motion vector of a block at the same position as the processing target block, which is a block included in an already encoded picture;
A scaling determination step for determining that the motion vector cannot be scaled when the display order information of the picture including the same position block and the display order information of the reference picture referred to by the same position block are the same;
If it is determined by the scaling determination step that the scaling process cannot be performed, a predetermined value set in advance is generated as the motion vector of the processing target block without performing the scaling process, and it is determined that the scaling process can be performed. A motion vector generating step for generating a motion vector of the processing target block by performing the scaling process;
A motion compensation step of performing motion compensation of the processing target block using the motion vector generated by the motion vector generation step. The image coding method according to claim 1, wherein in the motion vector generation step, the motion vector of the processing target block is generated by setting the predetermined value to 0. An image decoding method for decoding pictures constituting an image,
Obtaining a motion vector of the same position block which is a block included in a picture that has already been decoded and is located at the same position as the processing target block;
A scaling determination step for determining that the motion vector cannot be scaled when the display order information of the picture including the same position block and the display order information of the reference picture referred to by the same position block are the same;
If it is determined by the scaling determination step that the scaling process cannot be performed, a predetermined value set in advance is generated as the motion vector of the processing target block without performing the scaling process, and it is determined that the scaling process can be performed. A motion vector generating step for generating a motion vector of the processing target block by performing the scaling process;
A motion compensation step of performing motion compensation of the processing target block using the motion vector generated by the motion vector generation step. The image decoding method according to claim 3, wherein, in the motion vector generation step, the predetermined vector is set to 0 to generate a motion vector of the processing target block. An image encoding device for encoding a picture constituting an image,
Means for obtaining a motion vector of a block at the same position as the processing target block, which is a block included in an already encoded picture;
Scaling determining means for determining that the motion vector cannot be scaled when the display order information of the picture including the same position block is the same as the display order information of the reference picture referred to by the same position block;
When the scaling determination unit determines that the scaling process cannot be performed, a predetermined value set in advance is generated as the motion vector of the processing target block without performing the scaling process, and it is determined that the scaling process can be performed. If so, a motion vector generation means for generating a motion vector of the processing target block by performing the scaling process;
An image encoding apparatus comprising: a motion compensation unit that performs motion compensation of the processing target block using the motion vector generated by the motion vector generation unit. An image decoding apparatus for decoding pictures constituting an image,
Means for obtaining a motion vector of a block at the same position as the processing target block, which is a block included in an already decoded picture;
Scaling determining means for determining that the motion vector cannot be scaled when the display order information of the picture including the same position block is the same as the display order information of the reference picture referred to by the same position block;
When the scaling determination unit determines that the scaling process cannot be performed, a predetermined value set in advance is generated as the motion vector of the processing target block without performing the scaling process, and it is determined that the scaling process can be performed. If so, a motion vector generation means for generating a motion vector of the processing target block by performing the scaling process;
An image decoding apparatus comprising: motion compensation means for performing motion compensation of the block to be processed using the motion vector generated by the motion vector generation means.

Images