JP2010011075A - Method and apparatus for encoding and decoding moving image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image encoding method by which a background region and a moving region are separated in a moving image and motion compensation/prediction processing is performed upon the separated moving region. <P>SOLUTION: A binary moving region separation mask indicating a moving region and a background region is generated for each reference image signal, one background image signal is generated or updated by comparing two or more reference image signals or in accordance with a value of the binary moving region separation mask for each reference image signal, the moving region separation mask is used to perform motion compensation processing upon a first portion of the image to be predicted corresponding to the moving region, and a signal obtained by interpolating the background image signal is complemented for a second portion of the image to be predicted corresponding to the background region, thereby generating a prediction image signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving image coding and moving image decoding method and apparatus for separating a background region and a moving region from a moving image and performing motion compensation prediction processing on the separated moving region.

  In recent years, a moving picture coding method with greatly improved coding efficiency has been jointly developed by ITU-T and ISO / IEC, and ITU-T Rec. H. 264 and ISO / IEC 14496-10 (hereinafter referred to as H. 264). Recommended). In H.264, prediction processing, conversion processing, and entropy encoding processing are performed in units of rectangular blocks (16 × 16, 8 × 8, etc.). For this reason, when predicting an object that cannot be represented by a rectangular block in H.264, the prediction efficiency is increased by selecting a smaller predicted block shape (4 × 4, etc.). In order to predict such objects effectively, a method of preparing multiple prediction patterns in a rectangular block, a method of dividing a block with an arbitrary line segment, and applying motion compensation for each divided shape are proposed. Has been.

  As a prediction method that separates the background image and the foreground image, focusing on a coded slice (B-slice) sandwiched between two reference images, a method that separates the foreground and the foreground and performs motion compensation separately is proposed. [Patent Document 1]. In addition, a method has been proposed in which a background image mask and a background reference image corresponding to each reference image are created from the differences between a plurality of reference images that have already been encoded, and combined in motion compensated prediction [ Non-patent document 1].

  In the method of Patent Document 1, since it is necessary to encode motion vector information and block division information corresponding to the foreground and the background, there is a problem that the encoding efficiency is lowered at a low bit rate. In addition, in the encoder, it is necessary to repeatedly encode in order to select an optimal prediction mode, which increases the amount of calculation.

In the method of Non-Patent Document 1, since an area is separated for each pixel on the basis of an absolute difference value between images, noise included in a video to be encoded, quantization error that occurs when encoding is performed with high compression, and the like. For example, it may be difficult to separate the object and the background area, and the prediction efficiency may be reduced. Further, it is necessary to generate a background image mask and a background reference image memory for each reference image, and there is a problem that the memory of the decoder increases.
JP 2002-359854 A R. Ding, F. Wang, Q. Dai, W. Xu and D. Zhu, “Composite-Block Model And Joint-Prediction Algorithm For Inter-frame Video Coding,” ICASSP-2006 May 2006

  An object of the present invention is to generate a binary moving region separation mask and a single background image signal corresponding to each reference image signal from a plurality of decoded reference images, and generate a moving region from the moving region separation mask. The motion compensation prediction is performed on the area determined as the background area, and the area determined as the background area is supplemented with the value supplemented with the background image signal, thereby increasing the code amount due to excessive block segmentation. Prevent and improve prediction efficiency.

  One embodiment of the present invention divides an input image signal into a plurality of pixel blocks, performs a prediction process on each pixel block using a reference image signal, and encodes a difference signal between the input image signal and the predicted image signal In the moving image encoding method, a mask generating step for generating a binary moving region separation mask indicating a moving region and a background region for each reference image signal, and comparing the signals of two or more reference images or the A background image generating / updating step for generating or updating a signal of one background image according to a binary moving region separation mask value for each signal of a reference image, and using the moving region separation mask, (1) Motion compensation processing is performed on the first part of the prediction target image corresponding to the moving area, and (2) the background image signal is interpolated in the second part of the prediction target image corresponding to the background area Complement the signal , To provide a moving picture coding method configured to have a predictive image generation step of generating a predictive image signal by.

  By using the method of the present invention, in order to predict a moving object that is not suitable for a rectangular block, excessive block division is prevented and block division information is prevented from increasing. In other words, the effect of improving coding efficiency and subjective image quality by separating the motion area and background area in a block without applying additional information and applying an optimal prediction method to each of them. Play.

  Hereinafter, first to sixth embodiments of the present invention will be described with reference to the drawings.

<Moving picture encoding apparatus>
A configuration of a moving picture coding apparatus 100 for realizing moving area separation prediction coding according to the present invention is shown in FIG. A detailed block diagram of the prediction unit 106 of the moving picture coding apparatus 100 is shown in FIG. FIG. 3 shows a block diagram of an inter prediction unit related to the motion region separation predictive coding that performs the motion region separate prediction coding method. First, an embodiment will be described with reference to FIGS. 1, 2, and 3 regarding a moving region separation predictive coding method related to moving image coding.

(First embodiment)
With reference to FIG. 1, a moving picture encoding apparatus according to the first embodiment will be described. The image coding apparatus is configured to divide each frame constituting an input image signal into a plurality of pixel blocks, perform coding processing on the divided pixel blocks, perform compression coding, and output a code string Has been. Specifically, the image encoding device 100 calculates a difference between the input image signal 110 and the predicted image signal 117, converts and quantizes the subtraction value 101 that outputs the prediction error signal 111, and the prediction error signal 111, A transform / quantization unit 102 that outputs a transform coefficient 112 and an inverse quantization / inverse transform unit 103 that inversely quantizes the transform coefficient 112 and performs inverse transform to generate a restored prediction error signal 113 are included. Furthermore, the image coding apparatus 100 adds the restored prediction error signal 113 and the predicted image signal 117 to generate a decoded image signal 114, and a reference image memory 105 that stores the decoded image signal 114 as a reference image signal. And a prediction unit 106 that generates a predicted image signal 117 using the reference image signal 116 and the input image signal 110. Furthermore, the moving image encoding apparatus 100 includes a code sequence encoding unit 108 that encodes the transform coefficient 112 into an encoded sequence and outputs the code sequence to the output buffer 109. The moving image encoding apparatus 100 is controlled by the encoding control unit 107.

  In the moving image encoding apparatus having the above configuration, the input image signal 110 of a moving image or a still image is divided into small pixel blocks, for example, macroblocks, and input to the moving image encoding apparatus 100. Here, the input image signal 110 means one encoding processing unit (picture) including both a frame and a field. Here, the macro block is assumed to be the basic processing block size of the encoding process. The macroblock is typically a 16 × 16 pixel block as shown in FIG. 4A, for example, but may be a 32 × 32 pixel block unit or an 8 × 8 pixel block unit, and the shape of the macroblock Need not be a square lattice. Hereinafter, the encoding target macroblock of the input image signal 110 is simply referred to as a target block. In the present embodiment, in order to simplify the description, it is assumed that the encoding process is performed from the upper left to the lower right as illustrated in FIG. 4A.

  The moving image encoding apparatus 100 is provided with a plurality of prediction modes having different block sizes and generation methods of the predicted image signal 117. Specifically, the method of generating the predicted image signal 117 is roughly divided into a plurality of reference frames that are temporally different from intra prediction (intraframe prediction) in which a predicted image is generated only within a frame (field) to be encoded. There is inter prediction (interframe prediction) in which prediction is performed using (reference field).

  Next, the flow of encoding by the moving image encoding apparatus 100 will be described. First, the input image signal 110 is first input to the subtractor 101. The subtracter 101 further receives a prediction image signal 117 corresponding to each prediction mode output from the prediction unit 106 described later. The subtractor 101 calculates a prediction error signal 111 obtained by subtracting the prediction image signal 117 from the input image signal 110. The prediction error signal 111 generated and output by the subtractor 101 is input to the transform / quantization unit 102. In the transform / quantization unit 102, a transform coefficient is generated by performing orthogonal transform such as discrete cosine transform (DCT) on the prediction error signal 111, for example.

  The transform / quantization unit 102 quantizes the transform coefficient in accordance with quantization information represented by a quantization parameter, a quantization matrix, and the like given by the encoding control unit 107. The quantized transform coefficient 112 is output from the transform / quantization unit 102, input to the code string encoding unit 108, and also output to the inverse quantization / inverse transform unit 103. Here, although the discrete cosine transform as used in H.264 has been described for the transform in the transform / quantization unit 102, techniques such as discrete sine transform, wavelet transform, and independent component analysis may be used. .

  In the code string encoding unit 108, various encoding parameters used when the target block is encoded, including the prediction information 119 output from the encoding control unit 107, together with the transform coefficient 112 after quantization, are set. Entropy encoding, for example, Huffman encoding or arithmetic encoding, is performed on the encoded data. Here, the encoding parameter refers to all parameters necessary for decoding such as information regarding transform coefficients and information regarding quantization as well as prediction information 119.

  The encoded data 118 generated by the code string encoding unit 108 is output from the video encoding apparatus 100, multiplexed with parameters necessary for decoding by a multiplexer (not shown), and temporarily stored in the output buffer 109. Is done. The encoded data 118 in the output buffer 109 is output to the outside of the moving image encoding apparatus 100 according to the output timing managed by the encoding control unit 107. The encoded data 118 is sent to a storage system (storage medium) or a transmission system (communication line) (not shown).

  On the other hand, the quantized transform coefficient 112 output from the transform / quantization unit 102 is input to the inverse quantization / inverse transform unit 103. In the inverse quantization / inverse transform unit 103, the transform coefficient 112 is first subjected to inverse quantization processing. Here, the quantization information represented by the same quantization parameter and quantization matrix as used in the transform / quantization unit 102 is loaded from the encoding control unit 107, and the transform coefficient 112 is dequantized. Processing is performed.

  The inverse-quantized transform coefficient is subjected to inverse orthogonal transform such as inverse discrete cosine transform (IDCT), thereby reproducing the decoded prediction error signal 113. The decoded prediction error signal 113 is input to the adder 104. The adder 104 adds the decoded prediction error signal 113 and the predicted image signal 117 output from the prediction unit 106 to generate a decoded image signal 114 (local decoded image signal). The decoded image signal 114 is stored as a reference image signal 116 in the reference image memory 105. The reference image signal 116 stored in the reference image memory 105 is output to the prediction unit 106 and is referred to at the time of prediction. The moving region separation mask 115 output from the prediction unit 106 is input to the reference image memory 105 and stored in the reference image memory 105 together with the decoded image signal 114 at the same time. Hereinafter, the reference image signal 116 refers to a set of a decoded image signal 114 and a moving region separation mask 115 that have been encoded or locally decoded at the same time.

  The prediction unit 106 performs inter prediction or intra prediction using pixels of the reference image signal 116 accumulated in the reference image memory 105 (decoded reference pixels and generated moving region separation mask pixels), A predictive image signal 117 that can be selected for the target block is generated. However, H.264 intra prediction, for example, intra prediction corresponding to the 4 × 4 pixel block shown in FIG. 4C or intra prediction for the 8 × 8 pixel block shown in FIG. With respect to a prediction mode in which the next prediction cannot be performed without creating an image, even if conversion / quantization and inverse quantization / inverse conversion or a decoding process for each corresponding pixel block is performed inside the prediction unit 106 Good.

  FIG. 2 shows a block diagram of the prediction unit 106. The prediction unit 106 includes an intra prediction unit 201, an inter prediction unit 202, a motion vector estimation unit 203, a mode determination switch 204, and a mode determination unit 205. When the reference image signal 116 is input to the prediction unit 106, the intra prediction unit 201 and the inter prediction unit 202 generate a prediction image signal 117 of an available prediction mode in the pixel block. Each prediction method will be described later. The prediction image signal generated by the intra prediction unit 201 and the prediction image signal generated by the inter prediction unit 202 are output to the mode determination switch 204. The mode determination switch 204 has a function of switching which of the input predicted image signals is used. The information for switching the switch is based on the prediction information 206 provided from the mode determination unit 205. The operation of the mode determination unit 205 will be described later.

  As an example of the prediction mode in the intra prediction unit 201, H.264 intra prediction will be described. In H.264 intra prediction, 4 × 4 pixel intra prediction (see FIG. 4C), 8 × 8 pixel intra prediction (see FIG. 4D), and 16 × 16 pixel intra prediction (see FIG. 4B) are defined. In this intra prediction, an interpolated pixel is created from the reference image signal 116 stored in the reference image memory 105, and a predicted value is generated by copying in the spatial direction.

  Next, the configuration and operation of the inter prediction unit 202 will be described with reference to FIG. According to FIG. 3, the inter prediction unit 202 includes a motion compensation unit 301 to which a reference image signal 116 is input, a motion region separation prediction unit 302, and a background image generation unit 303. The motion region separation prediction unit 302 receives a motion region separation mask 115, a reference image signal 116, a motion vector 207, and a background image signal 306 for motion region separation prediction. The motion compensation unit 301 and the motion region separation prediction unit 302 are switched by a prediction separation switch 305. The prediction separation switch 305 is switched by the prediction switching unit 304.

  The inter prediction unit 202 having the above configuration generates a predicted image signal 117 by performing an interpolation process based on the motion vector 207 and the reference image signal 116 of the prediction target block calculated by the motion vector estimation unit 203 of FIG. FIG. 5 shows an example of motion compensation prediction of inter prediction. In inter prediction, interpolation processing is performed using a plurality of reference image signals 116 stored in the reference image memory 105, and prediction is performed based on the amount of deviation from the pixel block at the same position between the created interpolated image and the original image signal. An image signal 117 is generated. As the interpolation processing, interpolation processing with 1/2 pixel accuracy, interpolation processing with 1/4 pixel accuracy, or the like is used, and the value of the interpolation pixel is generated by performing filtering processing on the reference image signal 116. For example, H.P. capable of performing interpolation processing up to 1/4 pixel accuracy on a luminance signal. In H.264, the shift amount is expressed by four times the integer pixel accuracy. This amount of deviation is called a motion vector.

  In inter prediction, a block size suitable for the current prediction target block can be selected from among a plurality of prediction blocks. FIG. 6A shows the size of the motion compensation block in units of macroblocks, and FIG. 6B shows the size of the motion compensation block in units of sub-blocks (8 × 8 pixel blocks or less). Since a motion vector can be obtained for each size of these prediction blocks, an optimal prediction block shape and motion vector can be used in accordance with the local nature of the input image signal 110. Further, information on which reference image signal is calculated for the motion vector can be changed as a minimum for each 8 × 8 pixel block as Ref_idx.

  Next, the motion vector estimation unit 203 will be described. The motion vector estimation unit 203 has a function of calculating a motion vector 207 suitable for the prediction target block using the input image signal 110 and the reference image signal 116. In the calculation of the motion vector 207, block matching is performed between the prediction target block of the input image signal 110 and the interpolated image of the reference image signal 116. As an evaluation criterion for matching, a value obtained by accumulating the difference between the input image signal 110 and the interpolated image after matching for each pixel is used. In the determination of the optimal motion vector 207, a value obtained by converting the difference between the predicted image and the original image may be used in addition to the above-described method. The determination may be made by taking the amount into consideration. Moreover, you may utilize Formula (1) (2) etc. which are mentioned later. Further, the matching method may be a full search within the matching range based on search range information provided from the outside of the encoding device, or may be performed hierarchically for each pixel accuracy.

  The motion vector 207 calculated for a plurality of reference image signals (pointing to locally decoded image signals that are temporally different) in this way is input to the inter prediction unit 202 and used to generate the predicted image signal 117. The The calculated motion vector 207 is stored in the encoding control unit 107 as prediction information 119 together with information related to prediction such as a corresponding pixel block shape, and is passed to the code string encoding unit 108 as prediction information 119 to be used as an entropy code. And then multiplexed into encoded data.

  Next, an outline of the mode determination unit 205 will be described. The mode determination unit 205 outputs switch switching information 206 to the mode determination switch 204 in accordance with information on the slice currently being encoded. The switch switching information 206 describes information about which of the output terminal of the intra prediction unit 201 and the output terminal of the inter prediction unit 202 is connected to the switch.

  Next, the function of the mode determination unit 205 will be described. When the currently encoded slice is an intra encoded slice, the mode determination unit 205 connects the output terminal of the mode determination switch 204 to the intra prediction unit 201. On the other hand, when the currently encoded slice is an inter-coded slice, the mode determination unit 205 determines whether to connect the mode determination switch 204 to the output end of the intra prediction unit 201 or to the output end of the inter prediction unit 202. To do.

More specifically, in the above case, the mode determination unit 205 performs mode determination using a cost such as the following equation (1). A code amount (for example, a motion vector code amount or a block shape code amount) related to the prediction information 119 required when the prediction mode is selected is set to OH, and an absolute difference (prediction error) between the input image signal 110 and the predicted image signal 117. If the SAD is an absolute cumulative sum of the signal 111, the following mode determination formula is used.

  Here, K represents a cost, and λ represents a constant. λ is a Lagrangian undetermined multiplier determined based on the quantization scale and the value of the quantization parameter. The mode determination is performed based on the cost K obtained in this way. That is, the mode that gives the smallest value of cost K is selected as the optimal prediction mode.

  The mode determination unit 205 may perform the mode determination using (a) only the prediction information 119 and (b) only the SAD instead of the equation (1), or (a) only the prediction information 119 ( b) A value obtained by performing Hadamard transformation only on SAD or a value approximate thereto may be used. Further, the mode determination unit 205 may create a cost using the activity (variation of signal values) of the input image signal 110, or may create a cost function using a quantization scale or a quantization parameter. .

As yet another example, a provisional encoding unit is prepared, and the amount of codes when the prediction error signal 111 generated by the provisional encoding unit in a certain prediction mode is actually encoded, the input image signal 110 and the decoded image signal. The mode determination may be performed using a square error with respect to 114. The mode determination formula in this case is as follows.

Here, J is an encoding cost, and D is an encoding distortion representing a square error between the input image signal 110 and the decoded image signal 114. On the other hand, R represents a code amount estimated by provisional encoding.

  When the encoding cost J of Expression (2) is used, provisional encoding and local decoding processing are required for each prediction mode, so that the circuit scale or the amount of calculation increases. On the other hand, since a more accurate code amount and encoding distortion are used, high encoding efficiency can be maintained. The cost may be calculated using only R or only D instead of Equation (2), or the cost function may be created using a value approximating R or D.

  As described above, it is determined whether the prediction image signal generated by the intra prediction unit 201 or the prediction image signal generated by the inter prediction unit 202 is selected, and the output terminal of the mode determination switch 204 is switched. The prediction image signal 117 of the prediction mode selected here is output from the prediction unit 106, input to the subtractor 101, and output to the adder 104.

  Next, the inter prediction unit 202 will be described in more detail. FIG. 3 shows a block diagram of the inter prediction unit 202. As described above, the inter prediction unit 202 includes the motion compensation unit 301, the moving region separation prediction unit 302, the background image generation unit 303, the prediction switching unit 304, and the prediction separation switch 305.

The reference image signal 116 output from the reference image memory 105 is input to the prediction unit 106 and input to the inter prediction unit 202. At the same time, the motion vector 207 estimated by the motion vector estimation unit 203 is input. In the motion compensation unit 301, first, the position referred to by the motion vector 207 is determined from the position of the predicted pixel block according to the information of the motion vector 207 using the following equation (3). Here, as described above, H.P. A description will be given by taking an example of H.264 1/4 pixel precision interpolation. That is, when each component of the motion vector is a multiple of 4, it means that it is an integer pixel position. In other cases, it is understood that the predicted position corresponds to the interpolation position with fractional accuracy.

Here, (x, y) is an index in the vertical and horizontal directions representing the start position of the prediction target block, and (x_pos, y_pos) represents the corresponding predicted position of the reference image signal. (mv_x, mv_y) represents a motion vector having a 1/4 pixel accuracy. Next, predicted pixels are generated by interpolation or interpolation processing of corresponding pixel positions of the reference image signal 116 with respect to the determined pixel positions.

FIG. An example of H.264 prediction pixel generation is shown. In the figure, alphabets indicated by capital letters (squares indicated by diagonal lines) indicate pixels at integer positions, and squares indicated by hatching indicate interpolation pixels at 1/2 pixel positions. A square displayed in white indicates an interpolation pixel corresponding to a 1/4 pixel position. For example, the interpolation processing of 1/2 pixel corresponding to the positions of alphabets b and h in the figure is calculated by the following equation (4).

In the figure, the interpolation processing of 1/4 pixels corresponding to the positions of alphabets a and d is calculated by the following equation (5).

Thus, the interpolation pixel at the 1/2 pixel position is generated using a 6-tap FIR filter (tap coefficient: (1, -5, 20, 20, -5, 1) / 32), and the 1/4 pixel position is obtained. These interpolation pixels are calculated using a 2-tap average value filter (tap coefficients: (1/2, 1/2)). The interpolation process of 1/2 pixel corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated by performing both directions of 6 taps in the vertical direction and 6 taps in the horizontal direction. Interpolated values can be generated by the same rule for pixel positions other than those described. The above is an example of predictive image signal generation in the motion compensation unit 301.

Next, the background image generation unit 303 will be described. The background image generation unit 303 has a function of generating the background image signal 306 and the moving region separation mask 115 using the input reference image signal 116 and a function of a memory for holding the generated background image signal 306. First, generation of the moving region separation mask 115 will be described. One moving region separation mask 115 exists for each decoded image signal 114 decoded at each time provided by the reference image signal 116. The moving region separation mask 115 is a rule in which a temporal luminance change (difference value) between the decoded image signal 114 decoded at the same time and the decoded image signal 114 decoded before is determined in advance. It is a binary mask map that is recognized as a background pixel when it is smaller than the value TH and determines that a luminance change exceeds a specified value TH as a moving pixel.

When there are a plurality of reference image signals 116 that can be used, a difference value is calculated for all pixels at the same position in the time direction, a representative value to be described later is determined, and a specified value is set for the determined difference value. It is used to determine whether it is a background pixel or a moving pixel.

Here, LD represents a locally decoded image signal. s is an index representing the displacement in the time direction, and s = 0 indicates the prediction target image itself. For example, it corresponds to the index of the reference image signal. FIG. 8 shows the correspondence between reference pixels and target pixels when obtaining difference values for a plurality of reference image signals. w is a variable that performs weighting according to a temporal distance. For example, it is possible to consider temporal correlation by giving a large weight to a decoded image signal close in time and giving a small weight to a decoded image signal far in time. FIG. 9 shows an example in which the weight w is changed according to the temporal distance from the prediction target pixel block.

  In addition, although the example which performs pixel area | region determination only with only a difference value was shown above, the absolute sum of the difference value of the pixel between several decoding image signals which can be used (time direction) is used as an index for determining a representative value. The maximum difference value, the average value of the difference values, the median value of the difference values, and the variance of the difference values may be used for the determination, or the pixels adjacent to the pixel that performs the region determination of the decoded image signal (in the spatial direction) Determination may be made using indices such as the absolute sum of the difference values, the maximum value of the difference values, the average value of the difference values, the median value of the difference values, and the variance of the difference values.

Further, correction may be performed on the once generated moving region separation mask. For example, in the correction target pixel of the generated moving region separation mask, an area that becomes an isolated point is obtained by using the mask values of four points on the top, bottom, left, and right corresponding to the adjacent positions, or nine points including the diagonal direction. It may be corrected or the mask value at the block boundary may be corrected in accordance with the predicted block shape. An example of this case is shown by the following equation (8).

Here, (i, j) indicates an index of a pixel adjacent to the target pixel, and (i, j) = (0,0) indicates a correction target pixel. FIG. 10 shows the relationship between the target pixel and adjacent pixels. It means that the distance from the target pixel increases as the density of the circles increases. Further, v is a variable that performs weighting according to the positional relationship between adjacent pixels. For example, when the spatial distance is close (i, j) = (0,1), (1,0), (0, -1), (-1,0), the weight is increased and the spatial distance is Consider spatial correlation such as reducing the weight of pixels such as large (i, j) = (1,1), (1, -1), (-1,1), (-1, -1) To be used.

  FIG. 11 shows an example in which the weight v is changed according to the city area distance in the spatial direction. When the calculated Diff is larger than the predetermined value TV, the value of the moving region separation mask of the adjacent pixel is different and it can be determined that the correlation is low, so the mask value of the target pixel is changed. On the other hand, when the value is smaller than the specified value TV, processing such as not changing the value because the spatial correlation is high is performed. By appropriately setting the weight v in the spatial direction in this way, the generated moving region separation mask can be corrected, and isolated points are removed, discontinuous points are connected, and the region is expanded to a rectangular block. Reduction, edge correction, pixel compensation, pixel matching, and the like are possible. In this embodiment, an example of changing the weight based on the city distance is shown, but the definition of the distance can be calculated using one of the Minkowski distances including the city distance, the Manhattan distance, and the like.

Next, generation of the background image signal 306 will be described. The background image signal 306 is a signal obtained by collecting only background regions with little change in luminance in the time direction, and is derived for each pixel based on the moving region separation mask 115 and the decoded image signal 114 closest in time. The A background image signal 306 is generated from the moving region separation mask 115 using the following equation (9).

Here, BG represents the background image signal 306, and LD represents the decoded image signal 114 closest in time to the frame to be updated.

  As shown in the above equation, when the background image signal 306 at that time is updated, the decoded image signal 114 and the moving region separation mask 115 that are temporally closest in the reference image signal 116 are used, and the mask value is 0 ( Only in the case of a background pixel, the update is performed with the weighted sum of the nearest decoded image signal 114 and the background image signal 306 before the update. The weighted sum becomes an average value filter by setting, for example, wt = 1/2. On the other hand, when the mask value is 1 (moving pixel), the update is not performed. The initial value of the background image signal 306 is a predetermined luminance value (for example, 0 or maximum luminance value (256 for 8 bits) for luminance signals, and intermediate luminance value (128 for 8 bits) for color difference signals). It is also possible to use I-slice luminance values that are encoded only by intra prediction. The background image signal 306 is refreshed when a scene change is performed in the input image signal or when an IDR picture is inserted. In this embodiment, an example is shown in which the background image signal 306 is always refreshed at the timing of I-slice. In the above process, the background image signal 306 is updated at an appropriate timing.

  The background image signal 306 is held in the internal memory of the background image generation unit 303, and the updated signal is output to the moving region separation prediction unit 302. Further, the generated moving region separation mask 115 is output from the inter prediction unit 202, and is stored in the reference image memory 105 as the reference image signal 105 together with the decoded image signal 114 at the same time via the prediction unit 106.

  In this example, the example using the decoded image signal closest to the time has been described. (1) A method of compensating for the pixel value of the closest reference image that can be used for the display time of the display. (3) The pixel value of the nearest reference image that can be temporally used for the next encoded image is calculated. A method of compensating, (4) a method of compensating for a pixel generated by a linear sum of a pixel stored in the background image memory and a pixel of the nearest reference image usable in display time, and (5) the above A method of compensating for a pixel generated by a linear sum of a pixel stored in a background image memory and a pixel of the nearest reference image usable in the encoding time; (6) stored in the background image memory; Of the nearest reference image that is temporally available for the current pixel and the next encoded image. How to compensate for pixels generated by the linear sum of elementary may utilize any one of the methods from the.

Next, the moving region separation prediction unit 302 will be described. The motion region separation prediction unit 302 includes a motion vector 207 output from the motion vector estimation unit 203, a reference image signal 116 output from the reference image memory 105, and a background image signal 306 output from the background image signal generation unit 303. Entered. The motion region separation prediction unit 302 uses the input motion region separation mask 115 to perform motion compensation processing for the motion region and background image signal for the background region, and use different prediction methods. It has a function of synthesizing the predicted signal. Note that matching is also performed on the moving region separation mask 115 using the input motion vector 207. That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 115. In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the 1/4 pixel precision motion compensation process is expressed by the following equation (10).

Here, (mv_x, mv_y) represents the horizontal component and vertical component of the motion vector with 1/4 pixel accuracy, respectively, and (imv_x, imv_y) represents the horizontal component and vertical component of the motion vector with integer pixel accuracy, respectively. ing. Using the derived integer precision motion vector, motion region separation prediction is performed as shown in the following equation (11).

Here, P represents a predicted image signal generated by moving region separation prediction. MC is a predicted image signal generated by the motion compensation prediction performed by the motion compensation prediction unit 301, and since the details have already been described in the description of the motion compensation unit 301, description thereof is omitted here. For example, values such as the interpolated pixels a, b, and j and integer pixels G, H, and M generated in FIG. 7 enter the predicted image signal MC. The motion vector 207 is applied to the decoded image signal 114 and the motion region separation mask 115 at the same time, and normal motion compensation prediction is performed for the motion region and the background image signal 306 is compensated for the background region. Thus, the prediction accuracy can be increased regardless of the shape of the moving object. FIG. 12 shows an example of the decoded image signal 114 and the moving region separation mask 115 and an example of the background image signal 306 when four reference image signals are available in the time direction. The prediction image signal generated in this way is output from the moving region separation prediction unit 302, and prediction information 119 such as a block shape and a motion vector used at this time is recorded in the encoding control unit 107.

  Next, the prediction switching unit 304 and the prediction separation switch 305 will be described. The prediction switching unit 304 outputs prediction switching information 307 for controlling the prediction separation switch 305 based on the input information of the moving region separation mask 115. The prediction separation switch 305 has a function of switching between connecting the output end of the switch to the motion compensation unit 301 side or connecting to the motion region separation prediction unit 302 side according to the prediction switching information 307. More specifically, the ratio of the moving region separation mask included in the prediction target pixel block is calculated, and the prediction switching information 307 is updated depending on whether the moving region is larger or smaller than a preset specified value TP. For example, if only 4 pixels take 0 and the remaining 60 pixels take 1 out of 64 mask values included in the 8 × 8 pixel block to be predicted, the target pixel block Since 90% or more is the moving region, the output end of the switch is connected to the motion compensation unit 301. As described above, the ratio of the moving region separation mask in the prediction target pixel block is calculated, and which prediction unit is connected can be dynamically switched depending on the magnitude of the ratio value. FIG. 13 shows an example of switching when TP = 90%. In this way, the inter prediction prediction method (motion compensation prediction and motion region separation prediction) of the prediction target pixel block is switched, and the prediction image signal 117 is output from the inter prediction unit 202.

  Next, a processing flow of the background image generation unit 303 in the inter prediction unit 202 will be described with reference to FIG. First, the generation of the moving region separation mask 115 and the update of the background image signal 306 performed by the background image generation unit 303 are performed after the encoding process or local decoding process of one frame or slice is completed, or after the next frame or This is performed immediately before the slice encoding process is performed (S501). First, the background image generation unit 303 checks the slice type of the current coding slice (the coding slice to be predicted next). When the coded slice is an intra-coded slice (I-slice) (YES in S502), the background image signal 306 is initialized (S503). When the encoded slice is other than I-slice (NO in S502), the moving region separation mask 115 is generated using the reference image signal 116 (S504). Further, the background image signal 306 is updated by using the reference image signal 116 and the generated moving region separation mask 115 (S505). The background image signal 306 is held in an internal memory that exists in the background image generation unit 303. The generated moving region separation mask 115 is output (S506), and the background image signal 306 is output to the moving region separation prediction unit 302 (S507). Next, it is determined whether or not the encoded slice is the final encoded frame (S508). If the determination is NO, the process returns to S502 after waiting for the encoded slice to be encoded. . On the other hand, if the determination is YES, the process ends (S509).

  Next, an overall processing flow of the background image generation unit 303 in the inter prediction unit 202, excluding the detailed functions described above, will be described with reference to FIG. When the motion vector 207, the reference image signal 116, and the background image signal 306 are input to the motion region separation prediction unit 302 (S601), the corresponding decoded image signal 114 in the reference image signal 116 is predicted using the input vector 207. The position is derived (S602). Next, a motion vector with integer precision is derived using the motion vector 207, and the corresponding position of the motion region separation mask is derived (S603). Further, the ratio of moving pixels included in the prediction target block in the moving region separation mask 115 is calculated (S604). It is checked whether or not the calculated ratio of moving pixels is larger than a preset specified value TP (S605). If this determination is YES, the pixel idx is initialized to 0 (S613), the motion compensation prediction process is performed on the pixel corresponding to the pixel idx (S614), and the pixel idx is incremented (S616). It is determined whether or not the incremented pixel idx is a value corresponding to the last pixel of the predetermined target prediction block (S616). If this determination is NO, the incremented pixel idx again corresponds to the pixel idx Motion compensation prediction is performed on the pixel to be performed (S614). On the other hand, if this determination is YES, the predicted image signal 117 is output (S617) and the process is terminated (S618).

  If the determination in S605 is NO, the pixel idx is first initialized to 0. The value of the corresponding position of the moving region separation mask is checked for the pixel idx (S607), and if the mask value of the pixel is a moving pixel (YES in S607), motion compensation prediction processing is performed for the pixel (S612). On the other hand, when the mask value of the pixel is a background pixel (NO in S607), a predicted position of the background image signal is derived (S608), and the background image signal at the predicted position is compensated (S609). Next, the value of the pixel idx is incremented (S610), and it is determined whether or not the pixel idx is a value corresponding to the last pixel of the target prediction block determined in advance (S611). The value of the corresponding position of the moving region separation mask is checked again with the incremented pixel idx (S607). If this determination is YES, the predicted image signal 117 is output (S617) and the process is terminated (S618). Among the steps of the flowchart, S604 and S605 are functions that the prediction switching unit 304 has, and steps S613 to S616 are functions that the motion compensation unit 301 has. Steps S602, S603, and S607-S611 are functions that the moving region separation prediction unit 302 mainly has.

  Next, a syntax structure in the moving image encoding apparatus 100 will be described. As shown in FIG. 23, the syntax is mainly composed of three parts, and the high-level syntax 1601 is packed with syntax information of higher layers above the slice. The slice level syntax 1602 specifies information necessary for each slice, and the macro block level syntax 1603 specifies data required for each macro block.

  Each has a more detailed syntax. The high-level syntax 1601 includes a sequence parameter sequence syntax 1604, a picture parameter set syntax 1605, and the like, and a picture level syntax. The slice level syntax 1602 includes a slice header syntax 1605, a slice data syntax 1606, and the like. Further, the macroblock level syntax 1603 includes a macroblock layer syntax 1607, a macroblock prediction syntax 1608, and the like.

  FIG. 24 shows an example of slice header syntax. The slice_motion_region_separation_flag shown in the figure is used for the prediction switching information 307 output from the prediction switching unit 304 in the inter prediction unit 202. When slice_motion_region_separation_flag is 0, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output terminal of the motion compensation prediction unit 301 is always output in the slice. That is, it means that motion compensation prediction is always performed. On the other hand, when slice_motion_region_separation_flag is 1, the motion compensation prediction and the motion region separation prediction are dynamically switched based on the signal of the motion region separation mask 115 output from the background image generation unit 303 in the slice as described above.

  FIG. 25 shows an example of macroblock layer syntax as an example of encoding parameters. Mb_type shown in the table indicates macroblock type information. That is, it includes information such as whether the current macroblock is intra-coded, inter-coded, or in what block shape is predicted. “Coded_block_pattern” shown in the table indicates whether or not a transform coefficient exists for each 8 × 8 pixel block. For example, when this value is 0, it means that there is no transform coefficient in the target block. Mb_qp_delta in the table indicates information on the quantization parameter. This information represents a difference value from the quantization parameter of the block encoded immediately before the target block. Intra_pred_mode in the table indicates a prediction mode indicating a prediction method of intra prediction. Ref_idx_l0 and ref_idx_l1 in the table indicate the index of a reference image that indicates which reference image was used to predict the target block when inter prediction is selected. Mv_l0 and mv_l1 in the table indicate motion vector information. In the table, transform_8 × 8_flag represents conversion information indicating whether or not the target block is 8 × 8 conversion.

  A syntax element not defined in the present invention can be inserted between the rows in the table, and other conditional branch descriptions may be included. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Furthermore, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.

  The above is the description of the moving picture coding apparatus 100 according to the present invention.

(First Embodiment: Modification Example 1: Switching Information Signaling)
In the present embodiment, as an example of a prediction method in the inter prediction unit 202, an example in which the motion compensation unit 301 and the motion region separation prediction unit 302 are dynamically switched by the prediction switching unit 304 has been described. An embodiment in which the dynamic region separation prediction is not dynamically switched is also possible. In this case, it is necessary to encode an index indicating which prediction method is used. This index is described in the prediction switching information 307. The index for the selected prediction image signal 117 is described in the prediction switching information 307, and this information is held in the encoding control unit 107. At the same time as the prediction image signal 117 generated using the prediction method is encoded, the prediction switching information 307 held as the prediction information 119 is loaded from the encoding control unit 107 and the code string encoding unit 108 is loaded. And the encoding process is performed.

  FIG. 14 shows an example of encoding an index indicating a prediction method used for each macroblock. When 90% or more of the pixels are moving pixels, motion compensation prediction is selected, and the macroblock occupied by the background pixels performs moving region separation prediction. Further, when the ratio of the moving pixel and the background pixel is included between the specified values THMAX and THMIN, an index indicating which prediction is used is encoded.

  FIG. 26 shows an example of macroblock layer syntax in the present embodiment. Mb_motion_region_separation_flag shown in the figure is used for the prediction switching information 307 output from the prediction switching unit 304 in the inter prediction unit 202. When mb_motion_region_separation_flag is 0, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output terminal of the motion compensation prediction unit 301 is always output in the macroblock. That is, it means that motion compensation prediction is always performed. On the other hand, when mb_motion_region_separation_flag is 1, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output end of the moving region separation prediction unit 302 is always output in the macroblock. That is, it means that the motion region separation prediction is always performed. SignalingFlag is an internal parameter for determining whether to encode mb_motion_region_separation_flag. When SignalingFlag is 1, it means that the ratio of moving pixels is included between the specified values THMAX and THMIN. On the other hand, when SignalingFlag is 0, it means that the ratio of moving pixels is not included between the prescribed values THMAX and THMIN.

(First Embodiment: Modification Example 2: Reuse of Predictive Image Signal)
In this embodiment, the motion compensation unit 301 and the motion region separation prediction unit 302 are described as separate prediction methods, but as shown in the flowchart of FIG. 16, the motion compensation unit is included in the motion region separation prediction unit 302. A prediction method similar to 301 is also used. In order to avoid an increase in the amount of calculation due to performing the same process a plurality of times as described above, a structure in which the predicted image signal 117 calculated by the motion compensation unit 301 is input to the moving region separation prediction unit 302 as shown in FIG. Also good. Alternatively, the function of the motion compensation unit 301 may be integrated with the motion region separation prediction unit 302.

(First embodiment: Modification 3: Deletion of switching structure)
In this embodiment, the motion compensation unit 301 and the motion region separation prediction unit 302 are described as separate prediction methods, but the prediction method is unified into the motion region separation prediction 302 and the prediction switching unit 304 is deleted. It is good also as a structure to do. FIG. 18 shows an embodiment in which the motion compensation unit 301, the prediction switching unit 304, and the prediction separation switch 305 are deleted. Since the prediction structure is simplified, it is possible to prevent an increase in hardware scale and the like.

(Second embodiment: Global MC)
In the present embodiment, the structure of moving picture coding apparatus 100 is the same as that in FIG. However, since the function of the prediction unit 106 is different, a prediction 701 is provided. FIG. 19 shows the structure of the prediction unit 701 in the second embodiment. In addition, the same index is given to those having the same functions as those already described, and the description thereof is omitted. Note that an inter prediction unit 801 is provided as a different index from the inter prediction unit 202 in FIG.

  The prediction unit 701 includes a global vector estimation unit 802 in addition to the inter prediction unit 801. The global vector estimation unit 802 calculates a vector (global MV (motion vector) 803) representing a change amount of the entire screen caused by a change in an imaging system such as a camera for each encoded frame, each encoded slice, or each macroblock. It has the function to do. In this embodiment, a parallel movement model is described as a framework for obtaining the motion of the entire screen. However, a model using affine transformation, a model based on similarity transformation, projective transformation, or the like may be used as a motion model. good. The parallel movement model can handle panning and tilting of the captured video, but it can also handle enlargement and reduction by using an affine transformation model. Further, the case where the accuracy of the global MV is an integer pixel accuracy will be described, but as described above, the expansion to the fractional accuracy is easy.

  The basic vector estimation function of the global vector estimation unit 802 is the same as that of the motion vector estimation unit 203 already described, but the local motion vector (local motion vector) calculated for each region such as a block is integrated. A function for calculating the global MV 803 has been added. For example, a motion vector in the screen for each 4 × 4 pixel block is calculated, and a histogram of the calculated motion vector is created. A local motion vector calculated by a local block may not be able to follow the movement of the camera due to the influence of a moving object in the screen. Therefore, in order to obtain a global motion vector, a motion vector having the highest appearance frequency is set as the global motion vector 702. The global MV 803 calculated by the global vector estimation unit 802 is input to the inter prediction unit 801.

  Next, the inter prediction unit 801 will be described. FIG. 20 is a block diagram of the inter prediction unit 801. The background image signal generation unit 901 and the motion region separation prediction unit 902 are the same as those in FIG. 1 except that the global MV 803 is input to the background image generation unit 303 and the motion region separation prediction unit 302 of the first embodiment. The processing of is different.

First, the background image generation unit 901 will be described. The background image generation unit 901 receives the reference image signal 116 and the global MV 803 output from the reference image memory 105. By using the global MV 803, the background image generation unit 901 can generate the background image signal 306 even for an image in which the camera is moving. First, a method for generating the moving region separation mask 115 will be described. The moving region separation mask 115 is calculated by the following equation (12) using the reference image signal 116 and the global MV 803.

Here, (gmv_x, gmv_y) represents the horizontal / vertical component of the global MV 803. The MCLD represents a decoded image signal subjected to motion compensation processing. When the global MV 803 has fractional accuracy, the motion compensation processing described in the motion compensation unit 301 is applied. For example, in the case of 1/4 pixel accuracy, (gmv_x, gmv_y) in the equation is replaced with (gmv_x / 4, gmv_y / 4), respectively. When the global MV 803 has integer precision, the MCLD in Expression (12) is replaced with LD.

  Here, as the index for determining the representative value of the difference value, the method described in the first embodiment can be applied. Moreover, you may correct | amend with respect to the dynamic region separation mask once produced | generated similarly to 1st Embodiment.

Next, generation of the background image signal 306 will be described. The background image signal 306 is derived by the following equation (13) after using the moving region separation mask 115, the decoded image signal 114, and the global MV 803 described above.

Here, MCBG indicates a value obtained by performing motion compensation processing on the background image signal 306 using the global MV 803. As shown in the above equation, when the background image signal 306 at that time is updated, the decoded image signal 114 and the moving region separation mask 115 that are temporally closest in the reference image signal 116 are used, and the mask value is 0 ( In the case of (background pixel), the update is performed with the weighted sum of the nearest decoded image signal 114 and the background image signal 306 before update in consideration of the global MV 803.

Next, the moving region separation prediction unit 902 will be described. The motion region separation prediction unit 902 includes a motion vector 207 output from the motion vector estimation unit 203, a reference image signal 116 output from the reference image memory 105, and a background image signal 306 output from the background image signal generation unit 901. And the global MV 803 is input. The motion region separation prediction unit 902 performs motion compensation processing on the motion region using the input motion region separation mask 115 and performs motion compensation processing on the background region using the global MV 803, and performs separate processing. It has a function of synthesizing signals predicted by the prediction method. Note that matching is also performed on the moving region separation mask 115 using the input motion vector 207. That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 115. In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the ¼ pixel precision motion compensation process is expressed by Expression (11). Using the derived integer precision motion vector, motion region separation prediction is performed as shown in the following equation (14).

Here, P represents a predicted image signal generated by moving region separation prediction. By using the global MV 803 to perform motion compensation for normal motion compensation for a moving region and the background image signal 306 for a background region, the prediction accuracy can be improved regardless of the shape of the moving object. Is possible. The prediction image signal generated in this way is output from the motion region separation prediction unit 302, and the prediction information 119 such as the block shape, the motion vector 207, and the global MV 803 used at this time is sent to the encoding control unit 107. Recorded, entropy encoded, and finally multiplexed into encoded data.

  FIG. 27 shows an example of slice header syntax in the present embodiment. The slice_global_motion_flag shown in the figure is a flag indicating whether or not to perform motion region separation prediction using the global MV 803. When slice_global_motion_flag is 0, the background image generation unit 901 and the motion region separation prediction unit 902 perform the same prediction as the background image generation unit 303 and the motion region separation prediction unit 302 described in the first embodiment. That is, the global MV 803 is not sent and is not used. On the other hand, when slice_global_motion_flag is 1, gmv_param is encoded by the number of NumOfGMP indicating the number of parameters of global MV 803 determined in advance. Using these pieces of information, a corresponding predicted image signal is generated by the background image generation unit 901 and the moving region separation prediction 902. In the present embodiment, an example of NumOfGMP = 2 is shown, where gmv_param [0] represents a horizontal motion vector and gmv_param [1] represents a vertical motion vector. These pieces of information are calculated by the global vector estimation unit 802 and encoded by the code string encoding unit 108 as the prediction information 119 given by the encoding control unit 107.

  Here, although an example in which gmv_param is directly given as a parameter of global MV 803 has been shown in the present embodiment, a difference value from global MV 803 of the most recently encoded slice may be encoded, or a predetermined prediction A global MV 803 may be calculated by a method, and a difference value therefrom may be encoded.

  The above is the description of the inter prediction unit 801 of the video encoding device 100 according to the present invention.

(Third embodiment: adaptive interpolation filter)
In the present embodiment, the structure of moving picture coding apparatus 100 is the same as that in FIG. However, since the function of the prediction unit 106 is different, the prediction unit 1001 is provided. FIG. 21 shows a prediction unit 1001 in the third embodiment. In addition, the same index is given to those having the same functions as those already described, and the description thereof is omitted. Note that the inter prediction unit 202 is provided with an inter prediction unit 1101 for functional differences.

  The prediction unit 1001 includes a motion compensation filter coefficient estimation unit 1102 in addition to the inter prediction unit 1101. The motion compensation filter coefficient estimation unit 1102 has a function of calculating the filter coefficient 1103 used in the inter prediction motion compensation process for each encoded frame, each encoded slice, or each macroblock. In this embodiment, a two-dimensional 6-tap FIR filter will be described as an example of motion compensation processing. However, the number of taps can be assumed to be N taps, and can be freely set according to restrictions on the hardware to be used. You can choose. In addition, a one-dimensional filter, a two-dimensional filter, a three-dimensional filter, and the like are applicable.

The motion compensation filter coefficient estimation unit 1102 designs filter coefficients according to the properties of the input image signal 110 and the predicted image signal 117. For example, as described in the motion compensation unit 301 in the first and second embodiments, the correspondence between the prediction error and the motion vector when predicted by the motion compensation filter with a fixed filter coefficient is accumulated, and the motion vector The filter coefficient is calculated using the least square method so that the prediction error for each fractional position indicated by is minimized. The following formula (15) is used as an evaluation criterion at this time.

Here, O indicates the input image signal 110, and MC is a predicted image signal calculated using a fixed filter. h indicates the filter coefficient 1103 to be derived, and (i, j) indicates the fractional position where the filtering process is performed. (A, b) is a fixed value indicating the filter offset. The filter coefficient h is designed so that the square cost of Expression (15) is minimized. The designed filter coefficient 1103 is input to the inter prediction unit 1101.

  In this embodiment, the method of designing a filter using a normal fixed motion compensation filter has been described. However, the filter may be designed using the feature amount of the input image signal 110. For example, a filter coefficient set for high-frequency components, a filter coefficient set for medium-frequency components, and a filter coefficient set for low-frequency components are prepared, and filter coefficients are selectively selected according to the frequency characteristics of the input image signal. You may enter.

  Next, the inter prediction unit 1101 will be described. FIG. 22 is a block diagram of the inter prediction unit 1101. Since it is the same as that of FIG. 1 except that the filter coefficient 1103 is input to the moving region separation prediction unit 302 of the first embodiment, the other description is omitted.

First, the moving region separation prediction unit 1201 will be described. The motion region separation prediction unit 1201 includes a motion vector 207 output from the motion vector estimation unit 203, a reference image signal 116 output from the reference image memory 105, and a background image signal 306 output from the background image signal generation unit 901. The filter coefficient 1103 is input. The motion region separation prediction unit 1201 uses the input motion region separation mask 115 to perform adaptive motion compensation processing for the motion region and perform background image signal 306 compensation for the background region to perform separate prediction. A function of synthesizing a signal predicted by the method; Note that matching is also performed on the moving region separation mask 115 using the input motion vector 207. That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 115. In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the ¼ pixel precision motion compensation process is expressed by Expression (10). A predicted image signal is generated by the following equation (16) using the derived integer precision motion vector.

Here, AMC indicates a prediction value derived by adaptive motion compensation prediction. The adaptive motion compensation prediction will be described more specifically with reference to FIG.

First, predicted values of pixel positions a, b, c, d, h, and n corresponding to 1/2 pixel positions are generated by a 6-tap one-dimensional filter. For example, predicted values corresponding to the pixel positions a and d are generated by the following equation (17).

Next, predicted values of pixel positions e, f, g, i, j, k, p, q, and r corresponding to the remaining fractional precision positions are generated by a 6-tap two-dimensional filter. For example, the prediction corresponding to the pixel position of e is generated by the following equation (18).

When a predicted image is created by the above generation method, about 360 filter coefficients are generated at the maximum. Therefore, the filter coefficients are integrated taking into account the spatial contrast. For example, the filter coefficients are integrated by the following equation (19) using the objectivity of the pixels a, c, d, and l.

By using a coefficient using such contrast, it is possible to reduce the filter coefficient used in adaptive motion compensation prediction.

  In this way, the AMC prediction image signal of Expression (16) is generated using the filter coefficient 1103 calculated and input by the motion compensation filter coefficient estimation unit 1102.

  For the moving area, adaptive motion compensation is performed using the calculated filter coefficient 1103, and for the background area, the background image signal 306 is compensated to optimize the moving object and the background area. Since a predictive image signal can be generated, the prediction accuracy can be increased. The prediction image signal 117 generated in this way is output from the motion region separation prediction unit 1201 and the prediction information 119 such as the block shape, the motion vector 207, and the filter coefficient 1103 used at this time is included in the encoding control unit. It is recorded in 107, entropy-encoded, and finally multiplexed into encoded data.

  FIG. 27 shows an example of slice header syntax in the present embodiment. The slice_adaptive_filter_flag shown in the figure is a flag indicating whether or not to perform motion region separation prediction using adaptive motion compensation prediction. When slice_adaptive_filter_flag is 0, the moving region separation prediction unit 1201 performs the same prediction as the moving region separation prediction unit 302 described in the first embodiment. That is, adaptive motion compensation prediction for moving pixels is not performed, and filter coefficients are not used. On the other hand, when slice_adaptive_filter_flag is 1, filter_coeff is encoded by the number of NumOfPosX and NumOfPosY indicating the number of predetermined two-dimensional filter coefficients 1103. Using these pieces of information, adaptive motion compensation prediction is performed on the moving pixels in the moving region separation prediction 1201, and a predicted image signal is generated. These pieces of information are calculated by the motion compensation filter coefficient estimation unit 1102 and encoded by the code string encoding unit 108 as the prediction information 119 given by the encoding control unit 107.

  Here, although an example in which filter_coeff is directly given as a parameter of the filter coefficient 1103 has been described in the present embodiment, a difference value from the filter coefficient 1103 of the most recently encoded slice may be encoded or predetermined. Alternatively, the filter coefficient 1103 may be calculated by the prediction method, and the difference value therefrom may be encoded.

  The above is the description of the inter prediction unit 1101 of the video encoding device 100 according to the present invention.

  As described above, in this embodiment, in order to predict a moving object that is not suitable for a rectangular block, excessive block division is prevented and block division information is prevented from increasing. In other words, the effect of improving coding efficiency and subjective image quality by separating the motion area and background area in a block without applying additional information and applying an optimal prediction method to each of them. Play.

<Video decoding device>
Next, fourth to sixth embodiments relating to moving picture decoding will be described.
(Fourth embodiment)
FIG. 29 illustrates a video decoding device according to the fourth embodiment corresponding to the video encoding devices according to the first to third embodiments described with reference to FIGS. 1 to 28. The video decoding apparatus 400 includes a code string decoding unit 402 that decodes encoded data 409 input from the input buffer 401, and inverse quantization and inverse conversion that inversely quantizes transform coefficients from the code string decoding unit 402 and performs inverse transform. An adder 404 that adds the prediction error signal 411 and the predicted image signal 415 from the transform unit 403, the inverse quantization / inverse transform unit 403, and a reference image memory 405 that stores the decoded image signal from the adder 404 as a reference image; A prediction unit 406 that receives the reference image signal 413, the moving area mask 414, the prediction information, and the motion vector 417 and generates a predicted image signal 415 is provided. The moving image encoding apparatus 400 is controlled by the encoding control unit 408 and outputs the decoded image signal to the output buffer 407.

  In the above configuration, encoded data 409 transmitted from the moving image encoding apparatus 100 shown in FIG. 1 and transmitted via the storage system or transmission system is once stored in the input buffer 401 and multiplexed. Data is input to the video decoding device 400.

  In the video decoding device 400, the encoded data is input to the code string decoding unit 402, and decoding is performed by syntax analysis based on the syntax for each frame or field. That is, in the code string decoding unit 402, the code string of each syntax is sequentially entropy decoded, and the prediction information 416, the transform coefficient 410, the encoding parameter of the target block, and the like are reproduced. In the present embodiment, the encoding parameter refers to all parameters necessary for decoding such as information on transform coefficients and information on quantization as well as prediction information 416.

  The transform coefficient 410 decoded by the code string decoding unit 402 is input to the inverse quantization / inverse transform unit 403. Various information relating to the quantization decoded by the code string decoding unit 402, that is, the quantization parameter and the quantization matrix are set in the decoding control unit 408 and loaded when used as an inverse quantization process. The inverse quantization / inverse transform unit 403 first performs inverse quantization processing using the loaded information regarding quantization. The inversely quantized transform coefficient 410 is subsequently subjected to inverse transform processing (for example, inverse discrete cosine transform). Here, the inverse orthogonal transform has been described. However, when wavelet transform or the like is performed in the encoding apparatus, the inverse quantization / inverse transform unit 403 may execute the corresponding inverse quantization and inverse wavelet transform. good.

  Through the inverse quantization / inverse transform unit 403, the restored prediction error signal 411 is input to the adder 404, where it is added to the predicted image signal 415 generated by the prediction unit 406, which will be described later, and the decoded image signal 412 is generated. The generated decoded image signal 412 is output from the video decoding device 400, temporarily stored in the output buffer 407, and then output according to the output timing managed by the decoding control unit 408. The decoded image signal 412 is stored in the reference image memory 405 and becomes a reference image signal 413. The reference image signal 413 is sequentially read from the reference image memory 405 for each frame or each field and input to the prediction unit 406.

  Next, the prediction unit 406 will be described. Prediction information 416 indicating the prediction method decoded by the code string decoding unit 402 is input to the prediction unit 406, and an already encoded decoded image signal 412 stored in the reference image memory 405 is predicted as a reference image 413. Input to the unit 406. In this figure, for simplification of explanation, the motion vector 417 in the prediction information 416 used in motion compensation prediction and motion region separation prediction is input separately.

  FIG. 30 shows a block diagram of the prediction unit 406. The prediction unit 406 includes a prediction changeover switch 503, an intra prediction unit 501, and an inter prediction unit 502. The prediction changeover switch 503 has a function of switching which prediction method is used according to the prediction mode included in the prediction information 416 input to the prediction unit 406. When the prediction mode is intra prediction, the prediction changeover switch 503 is connected to the intra prediction unit 501. On the other hand, when the prediction mode is inter prediction, the prediction changeover switch is connected to the inter prediction unit 502.

  The intra prediction unit 501 generates the predicted image signal 417 by performing the processing described in the first embodiment. In the present embodiment, 4 × 4 pixel intra prediction (see FIG. 4C), 8 × 8 pixel intra prediction (see FIG. 4D), and 16 × 16 pixel intra prediction (see FIG. 4B) are defined. In this intra prediction, an interpolated pixel is created from a reference image signal 413 stored in the reference image memory 405, and a predicted value is generated by copying in the spatial direction.

  Next, the inter prediction unit 502 will be described. The structure of the inter prediction unit 502 is exactly the same as that of the inter prediction unit 202 in the video encoding apparatus described with reference to FIG. However, the prediction image signal 415 generated in the prediction unit 406 only needs to perform a prediction image signal generation process only in the prediction mode given by the prediction information 416. That is, it is not necessary to generate a predicted image signal 415 other than the given prediction mode. For example, when the prediction mode given by the prediction information 416 is inter prediction, the motion vector 417 decoded and generated by the code string decoding unit 402, the block shape information included in the prediction information 416, and the reference image signal to be used It is sufficient to generate only one prediction image signal 415 from the given information for the target block.

  The motion compensation unit 301 in the inter prediction unit 502 (202) will be described in more detail. In the motion compensation unit 301, first, according to the information of the motion vector 417 (207), the position referred to by the motion vector 417 (207) is calculated from the position of the predicted pixel block using Expression (3). Here, H. A description will be given by taking an example of H.264 1/4 pixel precision interpolation. When each component of the motion vector is a multiple of 4, it means that it is an integer pixel position. In other cases, it is understood that the predicted position corresponds to the interpolation position with fractional accuracy. Next, with respect to the determined pixel position, a predicted pixel is generated by interpolation or interpolation of the corresponding pixel position of the reference image signal 413 (116). FIG. An example of H.264 prediction pixel generation is shown. For example, the interpolation process of 1/2 pixel corresponding to the positions of alphabets b and h in the figure is calculated by Expression (4). In the figure, the interpolation processing of 1/4 pixels corresponding to the positions of the alphabets a and d is calculated by Expression (5). Thus, the interpolation pixel at the 1/2 pixel position is generated using a 6-tap FIR filter (tap coefficient: (1, -5, 20, 20, -5, 1) / 32), and the 1/4 pixel position is obtained. These interpolation pixels are calculated using a 2-tap average value filter (tap coefficients: (1/2, 1/2)). The interpolation process of 1/2 pixel corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated by performing both directions of 6 taps in the vertical direction and 6 taps in the horizontal direction. Interpolated values can be generated by the same rule for pixel positions other than those described. The above is an example of predictive image signal generation in the motion compensation unit 301.

  Next, the background image generation unit 303 will be described. The background image generation unit 303 uses the input reference image signal 413 (116) to generate a background image signal 306 and a moving region separation mask 414 (115), and a memory that holds the generated background image signal 306 As a function. First, generation of the moving region separation mask 414 (115) will be described. One moving region separation mask 414 (115) exists for each decoded image signal 114 decoded at each time provided by the reference image signal 413 (116). The moving region separation mask 414 (115) has a temporal luminance change (difference value) with respect to each pixel of the decoded image signal 412 decoded at the same time in advance with the decoded image signal 412 previously decoded. This is a binary mask map that is recognized as a background pixel when it is smaller than the prescribed value TH and is determined to be a moving pixel when the luminance change exceeds the prescribed value TH, and is represented by Expression (6).

  When there are a plurality of reference image signals 413 (116) that can be used, it is determined whether the pixel is a background pixel or a moving pixel using Equation (7). FIG. 9 shows a correspondence relationship when obtaining difference values for a plurality of reference image signals. FIG. 11 shows an example in which the weight w is changed according to the temporal distance from the prediction target pixel block.

  In addition, although the example which performs the area | region determination of a pixel only by only a difference value was shown above, as an index for determining a representative value, a pixel difference value or a difference value between a plurality of available decoded image signals (time direction) May be determined using the maximum value, the average value of the difference values, the median value of the difference values, and the variance of the difference values, or the difference value of the pixels (space direction) adjacent to the pixel that performs the region determination of the decoded image signal It may be determined using indices such as the maximum value, the average value of the difference values, the median value of the difference values, and the variance of the difference values.

  Further, correction may be performed on the once generated moving region separation mask. For example, in the correction target pixel of the generated moving region separation mask, an area that becomes an isolated point is obtained by using the mask values of four points on the top, bottom, left, and right corresponding to the adjacent positions, or nine points including the diagonal direction. It may be corrected or the mask value at the block boundary may be corrected in accordance with the predicted block shape. An example of this case is shown in equation (8). FIG. 11 shows the relationship between the target pixel and adjacent pixels. As the density of the circles in FIG. 10 increases, the distance from the target pixel increases. Further, FIG. 12 shows an example in which the weight v is changed according to the city area distance in the spatial direction. By appropriately setting the weight v in the spatial direction in this way, the generated moving region separation mask can be corrected, and isolated points are removed, discontinuous points are connected, and the region is expanded to a rectangular block. Reduction, edge correction, pixel compensation, pixel matching, and the like are possible. In the present embodiment of the present invention, an example of changing the weight based on the city distance is shown, but the definition of the distance can be calculated using one of the Minkowski distances including the city distance, the Manhattan distance, and the like. It is.

  Next, generation of the background image signal 306 will be described. The background image signal 306 is a signal obtained by collecting only background regions with little change in luminance in the time direction, and each pixel is based on the moving region separation mask 414 (115) and the decoded image signal 412 closest in time. To be derived. A background image signal 306 is generated from the moving region separation mask 414 (115) using Equation (9). When the background image signal 306 at the time is updated, the decoded image signal 412 and the moving region separation mask 414 (115) that are closest in time in the reference image signal 413 (116) are used, and the mask value is 0. Only in the case of the background pixel, the update is performed with the weighted sum of the nearest decoded image signal 114 and the background image signal 306 before the update. The weighted sum becomes an average value filter by setting, for example, wt = 1/2. On the other hand, when the mask value is 1 (in the case of moving pixels), no update is performed. The initial value of the background image signal 306 is a predetermined luminance value (for example, 0 or maximum luminance value (256 for 8 bits) for luminance signals, and intermediate luminance value (128 for 8 bits) for color difference signals). It is also possible to use I-slice luminance values that are encoded only by intra prediction. The background image signal 306 is refreshed when an I-slice is inserted or when an IDR picture is inserted. In this embodiment, an example is shown in which the background image signal 306 is always refreshed at the timing of I-slice. In the above process, the background image signal 306 is updated at an appropriate timing.

  The background image signal 306 is held in the internal memory of the background image generation unit 303, and the updated signal is output to the moving region separation prediction unit 302. The generated moving region separation mask 414 (115) is output from the inter prediction unit 202, and is stored in the reference image memory 405 as a reference image signal 413 together with the decoded image signal 412 at the same time via the prediction unit 106. The

  Next, the moving region separation prediction unit 302 will be described. The motion region separation prediction unit 302 outputs the motion vector 417 (207) decoded by the code string decoding unit 402, the reference image signal 413 (116) output from the reference image memory 405, and the background image signal generation unit 303. The background image signal 306 is input. The motion region separation prediction unit 302 uses the input motion region separation mask 414 (115) to perform motion compensation processing for the motion region and background image signal for the background region, and perform separate processing. It has a function of synthesizing signals predicted by the prediction method. Note that matching is also performed on the moving region separation mask 414 (115) using the input motion vector 417 (207). That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 115. In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the ¼ pixel precision motion compensation process is expressed by Expression (10). Using the derived integer precision motion vector, the motion region separation prediction is performed as shown in Equation (11).

  For example, values such as interpolation pixels a, b, and j and integer pixels G, H, and M generated in FIG. 8 enter MC. FIG. 13 shows an example of the decoded image signal 412 and the moving region separation mask 414 (115) and an example of the background image signal 306 when four reference image signals are available in the time direction. The predicted image signal created in this way is output from the moving region separation prediction unit 302.

  Next, the prediction switching unit 304 and the prediction separation switch 305 will be described. The prediction switching unit 304 outputs prediction switching information 307 for controlling the prediction separation switch 305 based on the input information of the moving region separation mask 414 (115). The prediction separation switch 305 has a function of switching between connecting the output end of the switch to the motion compensation unit 301 side or connecting to the motion region separation prediction unit 302 side according to the prediction switching information 307. More specifically, the ratio of the moving region separation mask included in the prediction target pixel block is calculated, and the prediction switching information 307 is updated depending on whether the moving region is larger or smaller than a preset specified value TP. FIG. 14 shows an example of switching when TP = 90%. In this manner, the prediction method (motion compensation prediction and motion region separation prediction) of the inter prediction of the prediction target pixel block is dynamically switched, and the predicted image signal 415 (117) is output from the inter prediction unit 202.

  Next, a syntax structure in main video decoding apparatus 400 will be described. As shown in FIG. 24, the syntax is mainly composed of three parts, and the high-level syntax 1601 is packed with syntax information of an upper layer higher than a slice. The slice level syntax 1602 specifies information necessary for each slice, and the macro block level syntax 1603 specifies data required for each macro block.

  Each has a more detailed syntax. The high-level syntax 1601 includes a sequence parameter sequence syntax 1604, a picture parameter set syntax 1605, and the like, and a picture level syntax. The slice level syntax 1602 includes a slice header syntax 1605, a slice data syntax 1606, and the like. Further, the macroblock level syntax 1603 includes a macroblock layer syntax 1607, a macroblock prediction syntax 1608, and the like.

  FIG. 25 shows an example of slice header syntax. The slice_motion_region_separation_flag shown in the figure is used for the prediction switching information 307 output from the prediction switching unit 304 in the inter prediction unit 502 (202). When slice_motion_region_separation_flag is 0, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output terminal of the motion compensation prediction unit 301 is always output in the slice. That is, it means that motion compensation prediction is always performed. On the other hand, when slice_motion_region_separation_flag is 1, motion compensation prediction and motion region separation prediction are dynamically switched based on the signal of the motion region separation mask 414 (115) output from the background image generation unit 303 in the slice as described above. .

  FIG. 26 shows an example of macroblock layer syntax as an example of encoding parameters. Mb_type shown in the table indicates macroblock type information. That is, it includes information such as whether the current macroblock is intra-coded, inter-coded, or in what block shape is predicted. “Coded_block_pattern” shown in the table indicates whether or not a transform coefficient exists for each 8 × 8 pixel block. For example, when this value is 0, it means that there is no transform coefficient in the target block. Mb_qp_delta in the table indicates information on the quantization parameter. The difference value from the quantization parameter of the block encoded immediately before the target block is represented. Intra_pred_mode in the table indicates a prediction mode indicating a prediction method of intra prediction. Ref_idx_l0 and ref_idx_l1 in the table indicate the index of a reference image that indicates which reference image was used to predict the target block when inter prediction is selected. Mv_l0 and mv_l1 in the table indicate motion vector information. In the table, transform_8 × 8_flag represents conversion information indicating whether or not the target block is 8 × 8 conversion.

  A syntax element not defined in the present invention can be inserted between the rows in the table, and other conditional branch descriptions may be included. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Furthermore, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.

  The above is the description of the moving picture decoding apparatus 400 according to the present invention.

(Fourth embodiment: modification example 1: switching information signaling)
In the present embodiment, as an example of a prediction method in the inter prediction unit 502 (202), an example in which the motion compensation unit 301 and the motion region separation prediction unit 302 are dynamically switched by the prediction switching unit 304 has been described. An embodiment that does not dynamically switch between compensation prediction and dynamic region separation prediction is also possible. In this case, it is necessary to decode an index indicating which prediction method is used. This index is described in the prediction switching information 307, and the index for the selected prediction image signal 117 is described in the prediction switching information 307.

  FIG. 15 shows an example of decoding an index indicating a prediction method used for each macroblock. FIG. 27 shows an example of macroblock layer syntax in the present embodiment. The mb_motion_region_separation_flag shown in the figure is used for the prediction switching information 307 output from the prediction switching unit 304 in the inter prediction unit 502 (202). When mb_motion_region_separation_flag is 0, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output terminal of the motion compensation prediction unit 301 is always output in the macroblock. That is, it means that motion compensation prediction is always performed. On the other hand, when mb_motion_region_separation_flag is 1, the prediction switching unit 304 switches the prediction separation switch 305 by setting the prediction switching information 307 so that the output end of the moving region separation prediction unit 302 is always output in the macroblock. That is, it means that the motion region separation prediction is always performed. SignalingFlag is an internal parameter for determining whether to encode mb_motion_region_separation_flag. When SignalingFlag is 1, it means that the ratio of moving pixels is included between the specified values THMAX and THMIN. On the other hand, when SignalingFlag is 0, it means that the ratio of moving pixels is not included between the prescribed values THMAX and THMIN.

(Fourth embodiment: Modification 2: Reuse of predicted image signal)
In this embodiment, the motion compensation unit 301 and the motion region separation prediction unit 302 are described as separate prediction methods, but the same prediction method as the motion compensation unit 301 is also used in the motion region separation prediction unit 302. Yes. In order to avoid an increase in the amount of calculation due to performing the same process a plurality of times as described above, the prediction image signal 415 (117) calculated by the motion compensation unit 301 is input to the motion region separation prediction unit 302 as shown in FIG. It is good also as a structure to do. Alternatively, the function of the motion compensation unit 301 may be integrated with the motion region separation prediction unit 302.

(Fourth embodiment: modification example 3: deletion of switching structure)
In this embodiment, the motion compensation unit 301 and the motion region separation prediction unit 302 are described as separate prediction methods, but the prediction method is unified into the motion region separation prediction 302 and the prediction switching unit 304 is deleted. It is good also as a structure to do. FIG. 19 shows an embodiment in which the motion compensation unit 301, the prediction switching unit 304, and the prediction separation switch 305 are deleted. Since the prediction structure is simplified, it is possible to prevent an increase in hardware scale and the like.

(Fifth embodiment: Global MC)
In the present embodiment, in the video decoding device 400, the prediction information 416 includes information on the global MV1401. Note that the structure of the moving picture decoding apparatus 400 is the same as that shown in FIG. However, since the function of the prediction unit 406 is different, a prediction unit 1400 is newly provided as shown in FIG. The prediction unit 1400 is the same in structure as the prediction unit 406, except that the global MV 1401 included in the prediction information 416 is input to the inter prediction unit 801.

  Functions in the inter prediction unit 801 will be described with reference to FIG. First, the background image generation unit 901 will be described. The background image generation unit 901 receives the reference image signal 413 (116) and the global MV 1401 (803) output from the reference image memory 405 (105). By using the global MV 1401 (803), the background image generation unit 901 can generate the background image signal 306 even for a video in which the camera is moving. First, a method for generating the moving region separation mask 414 (115) will be described. The moving region separation mask 414 (115) is calculated by Expression (12) using the reference image signal 413 (116) and the global MV 1401 (803). Here, as the index for determining the representative value of the difference value, the method described in the fourth embodiment can be applied. Moreover, you may correct | amend with respect to the dynamic region separation mask once produced | generated similarly to 4th Embodiment.

  Next, generation of the background image signal 306 will be described. The background image signal 306 is derived by Expression (13) after using the above-described moving region separation mask 414 (115), the decoded image signal 412, and the global MV 1401 (803).

  Next, the moving region separation prediction unit 902 will be described. The motion region separation prediction unit 902 receives the motion vector 417 (207), the reference image signal 413, the background image signal 306 output from the background image signal generation unit 901, and the global MV 1401 (803). The motion region separation prediction unit 902 uses the input motion region separation mask 414 (115) to perform motion compensation processing on the motion region and motion compensation using the global MV1401 (803) on the background region. It has a function of performing processing and synthesizing signals predicted by different prediction methods. Note that matching is also performed on the moving region separation mask 414 (115) using the input motion vector 417 (207). That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 414 (115). In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the ¼ pixel precision motion compensation process is expressed by Expression (10). Using the derived integer precision motion vector, the motion region separation prediction is performed as shown in Equation (14).

  By performing motion compensation using the global MV1401 (803) for the motion region using normal motion compensation prediction and for the background region using the background image signal 306, the prediction accuracy can be achieved regardless of the shape of the moving object. Can be mentioned.

  FIG. 27 shows an example of slice header syntax in the present embodiment. The slice_global_motion_flag shown in the figure is a flag indicating whether or not to perform motion region separation prediction using the global MV 1401 (803). When slice_global_motion_flag is 0, the background image generation unit 901 and the motion region separation prediction unit 902 perform the same prediction as the background image generation unit 303 and the motion region separation prediction unit 302 described in the fourth embodiment. That is, the global MV 1401 (803) is not decrypted and cannot be used.

  On the other hand, when slice_global_motion_flag is 1, gmv_param is decoded by the number of NumOfGMP indicating the number of parameters of global MV1401 (803) determined in advance. Using these pieces of information, a corresponding predicted image signal is generated by the background image generation unit 901 and the moving region separation prediction 902. In the present embodiment, an example of NumOfGMP = 2 is shown, where gmv_param [0] represents a horizontal motion vector and gmv_param [1] represents a vertical motion vector.

  Here, although an example in which gmv_param is directly given as a parameter of global MV 1401 (803) has been described in the present embodiment, a difference value from the global MV 1401 (803) of the most recently decoded slice may be encoded. Alternatively, the global MV 1401 (803) may be calculated by a predetermined prediction method, and the difference value therefrom may be decoded.

  The above is the description of the moving picture decoding apparatus according to the present invention.

(Sixth embodiment: adaptive interpolation filter)
In the present embodiment of the present invention, in the video decoding device 400, the prediction information 416 includes information on the filter coefficient 1501. Note that the structure of the moving picture decoding apparatus 400 is the same as that shown in FIG. However, since the function of the prediction unit 406 is different, a new index of the prediction unit 1500 is given and will be described with reference to FIG. The prediction unit 1500 is the same as the prediction unit 406 in structure, except that the filter coefficient 1501 included in the prediction information 416 is input to the inter prediction unit 1101.

  Functions in the inter prediction unit 1101 will be described with reference to FIG. The motion region separation prediction unit 1201 receives the motion vector 417 (207), the reference image signal 413 (116), the background image signal 306 output from the background image signal generation unit 901, and the filter coefficient 1501 (1103). . The motion region separation prediction unit 1202 uses the input motion region separation mask 414 (115) to perform adaptive motion compensation processing for the motion region and perform background image signal 306 compensation for the background region, It has a function of synthesizing signals predicted by different prediction methods. Note that matching is also performed on the moving region separation mask 414 (115) using the input motion vector 417 (207). That is, the derivation of the interpolation position from the motion vector described in the description of the motion compensation unit 301 is also applied to the motion region separation mask 414 (115). In this case, since the moving region separation mask has only integer pixel accuracy, in the case of a fractional accuracy motion vector, mapping to integer pixel accuracy is performed. The mapping to the integer pixel position in the case of the ¼ pixel precision motion compensation process is expressed by Expression (10). A predicted image signal is generated by Expression (16) using the derived integer-precision motion vector.

  The adaptive motion compensation prediction will be described more specifically with reference to FIG. First, predicted values of pixel positions a, b, c, d, h, and n corresponding to 1/2 pixel positions are generated by a 6-tap one-dimensional filter. For example, predicted values corresponding to the pixel positions a and d are generated by Expression (17). Next, predicted values of pixel positions e, f, g, i, j, k, p, q, and r corresponding to the remaining fractional precision positions are generated by a 6-tap two-dimensional filter. For example, the prediction corresponding to the pixel position of e is generated by Expression (18). In consideration of the contrast of the filter, the filter coefficient 1501 (1103) is integrated using Expression (19). By using a coefficient using such contrast, the filter coefficient 1501 (1103) used in adaptive motion compensation prediction can be reduced.

  Adaptive motion compensation is performed using the decoded filter coefficient 1501 (1103) for the moving region, and the background image signal 306 is supplemented for the background region, so that the moving object and each background region are compensated. Therefore, it is possible to improve the prediction accuracy.

  FIG. 28 shows an example of slice header syntax in the present embodiment. The slice_adaptive_filter_flag shown in the figure is a flag indicating whether or not to perform motion region separation prediction using adaptive motion compensation prediction. When slice_adaptive_filter_flag is 0, the motion region separation prediction unit 1201 performs the same prediction as the motion region separation prediction unit 302 described in the third embodiment. That is, adaptive motion compensation prediction for moving pixels is not performed, and filter coefficients are not used. On the other hand, when slice_adaptive_filter_flag is 1, filter_coeff is decoded by the number of NumOfPosX and NumOfPosY indicating the number of predetermined two-dimensional filter coefficients. Using these pieces of information, adaptive motion compensation prediction is performed on the moving pixels in the moving region separation prediction 1201, and a predicted image signal is generated.

  Here, although an example in which filter_coeff is directly given as a parameter of the filter coefficient 1501 (1103) is shown in the present embodiment, the difference value from the filter coefficient 1501 (1103) of the most recently decoded slice is decoded. Alternatively, the filter coefficient may be calculated by a predetermined prediction method, and the difference value therefrom may be decoded.

  The above is the description of the moving picture decoding apparatus according to the present invention.

(Modification of the first to sixth embodiments)
(1) In the first to sixth embodiments, the processing target frame is divided into short blocks of 16 × 16 pixel size or the like, and sequentially from the upper left block to the lower right side as shown in FIG. 4A. Although the case of encoding / decoding has been described, the encoding / decoding order is not limited to this. For example, encoding / decoding may be performed sequentially from the lower right to the upper left, or encoding / decoding may be performed sequentially from the center of the screen toward the spiral. Furthermore, encoding / decoding may be performed in order from the upper right to the lower left, or encoding / decoding may be performed in order from the periphery of the screen toward the center.

  (2) In the first to sixth embodiments, the block size is described as a 4 × 4 pixel block and an 8 × 8 pixel block. However, the target block does not need to have a uniform block shape, and 16 × 8. The block size may be a pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Also, it is not necessary to have a uniform block size within one macroblock, and blocks of different sizes may be mixed. In this case, the code amount for encoding the division information increases as the number of divisions increases, but the block size may be selected in consideration of the balance between the code amount of the transform coefficient and the locally decoded image.

  (3) In the first to sixth embodiments, the luminance signal and the color difference signal are not divided and described as an example limited to one color signal component. However, when the prediction processing is different between the luminance signal and the color difference signal, different prediction methods may be used, or the same prediction method may be used. When a different prediction method is used, the prediction method selected for the color difference signal is encoded / decoded in the same manner as the luminance signal.

  (4) In the first and fourth embodiments, as described with reference to FIG. 17, a modification example in which the prediction image signal generated by the motion compensation unit 301 is reused by the moving region separation prediction unit 302, or FIG. As described above, the modification example in which the motion compensation unit 301 is deleted and the moving region separation prediction unit 302 is always used has been described. These modification examples are the same as those in the second and third embodiments and the fifth and sixth embodiments. A similar framework can be applied in the form. In addition, the motion compensation prediction using the global MV 803 in the second and fifth embodiments may be applied to the motion compensation unit 301, or the adaptive motion compensation prediction using the filter coefficient 1103 in the third and sixth embodiments. It does not matter if it is adapted to the motion compensation unit 301.

  The above-described embodiment is not limited to the above-described embodiment, and the constituent elements can be modified and embodied without departing from the spirit of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

The block diagram which shows the moving image encoder according to 1st Embodiment. The block diagram which shows the prediction part according to 1st Embodiment The block diagram which shows the inter estimation part according to 1st Embodiment Diagram showing the flow of the encoding process A diagram showing a 16 × 16 pixel block A diagram showing a 4 × 4 pixel block Diagram showing an 8x8 pixel block The figure which shows the relationship between the positional relationship of a reference image signal and a prediction object image, and a motion vector The figure which shows the size of the motion compensation block per macroblock The figure which shows the size of the motion compensation block per subblock The figure which shows the positional relationship of the integer pixel and fractional pixel in the case of motion compensation prediction The figure which shows the relationship between the object pixel with respect to several reference image signals, and the pixel of the same position temporally The figure which shows the relationship between the time distance from a prediction object pixel block, and a weight. The figure which shows the spatial positional relationship and distance of an object pixel and an adjacent pixel The figure which shows the relationship between the spatial distance from a prediction object pixel block, and a weight. The figure which shows the outline | summary of prediction of several decoded image signals, a moving region separation mask, and a background image signal The figure which shows that a prediction method changes with the ratio of the moving pixel on a moving region separation mask, and a background pixel. The figure which shows switching prediction by the ratio of the moving pixel and background pixel on a moving region separation mask A flowchart showing the flow of processing of the background image signal generation unit The flowchart which shows the flow of a process of a motion area separation prediction part The block diagram of the inter prediction part shown as a modification of 1st Embodiment The block diagram of the inter prediction part shown as a modification of 1st Embodiment The block diagram of the prediction part provided in the moving image encoder according to the second embodiment. Block diagram of an inter prediction unit provided in the prediction unit of FIG. The block diagram of the prediction part provided in the moving image encoder according to the third embodiment. Block diagram of the inter prediction unit provided in the prediction unit of FIG. Diagram showing the syntax structure Diagram showing information contained in slice header The figure which shows the information contained in the macroblock layer in 1st Embodiment The figure which shows the information contained in the macroblock layer in the example of a change of 1st Embodiment The figure which shows the information contained in the slice header syntax in 2nd Embodiment The figure which shows the information contained in the slice header syntax in 3rd Embodiment Block diagram of moving picture decoding apparatus according to fourth, fifth and sixth embodiments The block diagram which shows the prediction part provided in the moving image decoding apparatus according to 4th Embodiment. The block diagram which shows the prediction part provided in the moving image decoding apparatus according to 5th Embodiment. The block diagram which shows the prediction part provided in the moving image decoding apparatus according to 6th Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Subtractor, 102 ... Transformation / quantization part, 103 ... Inverse transformation / inverse quantization part, 104 ... Adder, 105 ... Reference image memory, 106 ... Prediction part, 107 ... Coding control part, 108 ... Code sequence Encoding unit 109 ... output buffer 114 ... decoded image signal 115 ... moving region separation mask 116 ... reference image signal 117 ... prediction image signal 201 ... intra prediction unit 202 ... inter prediction unit 203 ... motion vector Estimating unit, 204 ... mode determination switch, 205 ... mode determination unit, 301 ... motion compensation unit, 302 ... moving region separation prediction unit, 303 ... background image generation unit, 304 ... prediction switching unit, 305 ... prediction separation switch, 306 ... Background image signal

Claims (36)

In the moving picture coding method for dividing an input image signal into a plurality of pixel blocks, performing a prediction process of each pixel block using a reference image signal, and coding a difference signal between the input image signal and the predicted image signal,
A mask generating step for generating a binary moving area separation mask indicating a moving area and a background area for each reference image signal;
A background image generating / updating step for generating or updating one background image signal based on a comparison of two or more reference image signals or a binary moving region separation mask value for each reference image signal;
Using the moving region separation mask, (1) motion compensation processing is performed on the first portion of the prediction target image corresponding to the moving region, and (2) the prediction target image corresponding to the background region is selected. In the second part, a predicted image generation step of generating a predicted image signal by supplementing a signal obtained by interpolating the signal of the background image, and
A moving picture encoding method configured to include:   In the mask generation step, a criterion for determining a pixel of the reference image as the moving region or the background region is determined according to a value derived from a difference value of pixels between two or more available reference images or in the reference image. The moving image encoding method according to claim 1, wherein the moving image encoding method is determined. The mask generation step and the background image generation / update step include
Estimating a global vector for correcting an amount of change between images due to a change in an imaging system between any of two or more available reference images and the prediction target image; and an estimated global Using the image interpolated based on the vector, generating the dynamic region separation mask and generating or updating the background image signal;
Encoding information on the global vector in units of any one of a sequence, an image, a slice, and a block;
The moving picture coding method according to claim 1 or 2, characterized by comprising: The predicted image generation step includes a step of changing a coefficient of a filter that generates an interpolated image of integer accuracy or fractional accuracy for each pixel position, for the pixel in which the moving region separation mask is determined to be a moving region, and the change Encoding the information regarding the filter coefficients in units of any one of each sequence, each image, each slice, and each block;
The moving picture encoding method according to claim 1, further comprising:   The mask generation step is derived from a weight based on a distance between a plurality of pixels of the moving region separation mask and spatially or temporally close to the generated moving region separation mask and a difference value of the pixels. The method includes a step of performing correction such as isolated point removal, discontinuous point connection, area enlargement / reduction to a rectangular block, edge correction, pixel compensation, pixel matching, and the like based on the value to be determined. 5. The moving image encoding method according to any one of 1 to 4. The predicted image generation step includes:
With respect to any one or more moving region division masks at the same position as the block of the first part of the prediction target image or a position derived based on a local motion vector mapped to the integer precision, Calculating a ratio or ratio of the background region;
Switching the prediction method according to whether either the ratio of the moving area or the ratio of the background area is larger or smaller than a predetermined value,
The moving picture encoding method according to claim 4, further comprising: The predicted image generation step includes:
A first prediction method that applies different prediction methods to the moving region and the background region based on the moving region separation mask, and all values of the moving region separation mask included in the block of the prediction target image The method further includes a step of encoding a piece of information indicating which one of the first and second prediction methods is used, the second prediction method having a second prediction method that is regarded as a moving region and is predicted by a single prediction method. 5. The moving picture encoding method according to claim 1, wherein The predicted image generation step includes:
Generating the predicted image signal by using a pixel value obtained by interpolating the signal of the background image based on the global vector for the pixel determined as the background region by the moving region separation mask;
Generating the predicted image signal by using a pixel value obtained by interpolating the reference image signal based on the local motion vector for the pixel determined as the moving region;
The moving picture encoding method according to claim 1, further comprising:   The predicted image generation step generates the predicted image signal by using a prediction method that can use the same moving region separation mask or different moving region separation masks for each of the luminance component and the color difference component or for each color component. The moving image encoding method according to claim 1, wherein the moving image encoding method is any one of claims 1 to 4. In a moving picture decoding method for decoding moving picture encoded data obtained by encoding each frame constituting an input image signal in units of pixel blocks, and performing decoding processing by a prescribed method,
A mask generation step for generating a binary moving region separation mask indicating a moving region and a background region for each signal of the reference image, and comparison of the signals of the two or more reference images or binary for each signal of the reference image A background image generating / updating step of generating or updating one background image signal according to the value of the moving region separation mask;
Using the moving region separation mask, (1) motion compensation processing is performed on the first portion of the prediction target image corresponding to the moving region, and (2) the second portion of the prediction target image corresponding to the background region is used. A predicted image signal generating step for generating a predicted image signal by compensating a signal obtained by interpolating the signal of the background image in the part;
A moving picture decoding method comprising:   In the mask generation step, a criterion for determining the moving area or the background area is determined according to a value derived from a difference value of pixels between two or more available reference images or in a reference image. The moving picture decoding method according to claim 10. (Moving image decoding: camera correction of moving region separation mask: middle concept)
The mask generation step and the background image generation / update step include
A global vector for correcting the amount of change between images caused by a change in the imaging system is estimated between one of two or more available reference image signals and the prediction target image. Using the image interpolated based on a global vector, generating the dynamic region separation mask and generating or updating the background image signal;
Encoding information on the global vector in units of any one of a sequence, an image, a slice, and a block;
The moving picture decoding method according to claim 10 or 11, further comprising: The predicted image generation step includes:
Changing a coefficient of a filter for generating an interpolated image of integer precision or fractional precision for each pixel position for a pixel in which the moving area division mask is determined to be a moving area;
Encoding the information regarding the changed filter coefficient in units of any one of each sequence, each image, each slice, and each block;
The moving picture decoding method according to any one of claims 10 to 12, further comprising:   The mask generation step is derived from a weight based on a distance between a plurality of pixels of the moving region separation mask and spatially or temporally close to the generated moving region separation mask and a difference value of the pixels. Including the steps of performing isolated point removal, discontinuous point connection, area expansion / reduction to a rectangular block, edge correction, pixel compensation, pixel matching, etc. The moving picture decoding method according to claim 10. The predicted image generation step is performed for any one or more moving region division masks at the same position as the partial block of the prediction target image or a position derived based on a local motion vector mapped with the integer precision. Calculating a ratio of the moving area or the ratio of the background area, and a prediction method according to whether either the ratio of the moving area or the ratio of the background area is larger or smaller than a predetermined value. Switching steps;
The moving picture decoding method according to claim 10, further comprising: The predicted image generation step includes:
A first prediction method that applies different prediction methods to the moving region and the background region based on the moving region separation mask, and the moving region separation included in the block of the first portion of the prediction target image It has a second prediction method in which all mask values are regarded as moving regions and is predicted by a single prediction method, and indicates which of the first prediction method and the second prediction method is used. 14. The moving picture decoding method according to claim 10, further comprising a step of decoding information. The predicted image generation step includes:
Generating the predicted image signal by using a pixel value obtained by interpolating the background image signal based on the global vector for the pixel determined as the background region by the moving region separation mask;
Generating the predicted image signal by using a pixel value obtained by interpolating the reference image signal based on the local motion vector for the pixel determined to be the moving region;
The moving picture decoding method according to any one of claims 10 to 12, further comprising:   The predicted image generation step generates the predicted image signal by using a prediction method that can use the same moving region separation mask or different moving region separation masks for each of the luminance component and the color difference component or for each color component. The moving picture decoding method according to any one of claims 10 to 13, wherein the moving picture decoding method is characterized. In a moving image encoding apparatus that divides an input image signal into a plurality of pixel blocks, performs a prediction process on each pixel block using a reference image signal, and encodes a difference signal between the input image signal and the predicted image signal.
Mask generating means for generating a binary moving region separation mask indicating a moving region and a background region for each reference image signal;
A background image generating / updating means for generating or updating one background image signal by comparing two or more reference image signals or by a binary moving region separation mask value for each reference image signal;
Using the moving region separation mask, (1) motion compensation processing is performed on the first portion of the prediction target image corresponding to the moving region, and (2) the prediction target image corresponding to the background region is selected. A predicted image generating means for generating a predicted image signal by supplementing a signal obtained by interpolating a background image signal in the second part;
A moving picture encoding apparatus configured to include:   The mask generating means determines a criterion for determining a pixel of the reference image as the moving region or the background region according to a value derived from a difference value of pixels between two or more available reference images or in the reference image. 20. The moving picture coding apparatus according to claim 19, further comprising means for determining. An estimation means for estimating a global vector for correcting an amount of change between images caused by a change in an imaging system between any one of two or more available reference images and the prediction target image; and the global vector And encoding means for encoding the information on the sequence, the image, the slice, or the block.
The mask generation unit and the background image generation / update unit generate a mask generation unit that generates the moving region separation mask and a signal of the background image using an image interpolated based on the estimated global vector, or Consists of background image generation / update means for updating,
21. The moving picture coding apparatus according to claim 19 or 20, wherein The prediction image means includes a changing means for changing a coefficient of a filter for generating an interpolation image with integer precision or fractional precision for each pixel position, for the pixel in which the moving area separation mask is determined to be a moving area, and the change Encoding means for encoding the information regarding the filter coefficient in units of any one of each sequence, each image, each slice, and each block;
The moving picture coding apparatus according to any one of claims 19 to 21, wherein the moving picture coding apparatus includes:   The mask generation means derives the weight based on the distance between the plurality of pixels of the moving region separation mask, which are spatially or temporally close, with respect to the generated moving region separation mask and the difference value of the pixels. And correcting means for performing correction such as isolated point removal, discontinuous point connection, area expansion / reduction to a rectangular block, edge correction, pixel compensation, pixel matching, and the like based on the value to be processed. Item 23. The moving picture encoding apparatus according to any one of Items 19 to 22. The predicted image means includes
With respect to any one or more moving region division masks at the same position as the block of the first part of the prediction target image or a position derived based on a local motion vector mapped to the integer precision, Calculating means for calculating a ratio or a ratio of the background region;
Switching means for switching a prediction method according to whether either the ratio of the moving area or the ratio of the background area is larger or smaller than a predetermined value,
23. The moving picture coding apparatus according to claim 22, further comprising: The predicted image generation means includes
A first prediction method that applies different prediction methods to the moving region and the background region based on the moving region separation mask, and all values of the moving region separation mask included in the block of the prediction target image Encoding means for encoding a piece of information indicating which one of the first and second prediction methods is used, having a second prediction method that is regarded as a moving region and predicting with a single prediction method 23. The moving picture coding apparatus according to claim 19, further comprising: The predicted image generation means includes
Generating means for generating the predicted image signal by using a pixel value obtained by interpolating the signal of the background image based on the global vector for the pixel determined as the background region by the moving region separation mask;
Generating means for generating the predicted image signal by using a pixel value obtained by interpolating the signal of the reference image based on the local motion vector for the pixel determined to be the moving region;
23. The moving picture coding apparatus according to claim 19, further comprising:   The predicted image generation means generates the predicted image signal using a prediction method that can use the same moving region separation mask or different moving region separation masks for each luminance component and color difference component or for each color component. 23. The moving picture coding apparatus according to claim 19, wherein the moving picture coding apparatus is any one of claims 19 to 22. In a moving image decoding apparatus that decodes moving image encoded data obtained by encoding each frame constituting an input image signal in units of pixel blocks, and performs decoding processing by a prescribed method,
A mask generating means for generating a binary moving region separation mask indicating a moving region and a background region for each reference image signal, and a comparison of the signals of the two or more reference images or a binary value for each signal of the reference image A background image generating / updating means for generating or updating one background image signal according to the value of the moving region separation mask;
Using the moving region separation mask, (1) motion compensation processing is performed on the first portion of the prediction target image corresponding to the moving region, and (2) the second portion of the prediction target image corresponding to the background region is used. A predicted image signal generating means for generating a predicted image signal by supplementing a signal obtained by interpolating the signal of the background image in the portion;
A moving picture decoding apparatus comprising:   The mask generation means includes means for determining a criterion for determining the moving area or the background area according to a value derived from a difference value of pixels between two or more available reference images or in a reference image. The moving picture decoding apparatus according to claim 28. An estimation means for estimating a global vector for correcting an amount of change between images caused by a change in an imaging system, between any of two or more available reference image signals and the prediction target image; Encoding means for encoding the information about the global vector in units of any one of a sequence, an image, a slice, and a block;
The mask generation means and the background image generation / update means generate or update the mask generation means for generating the moving region separation mask and the background image signal using an image interpolated based on the estimated global vector. Comprising background image generation / update means for
30. A moving picture decoding apparatus according to claim 28 or 29. The predicted image generation means includes
Change means for changing, for each pixel position, a coefficient of a filter that generates an integer-accurate or fraction-accurate interpolated image for pixels in which the moving area division mask is determined to be a moving area;
Encoding means for encoding the information regarding the changed filter coefficient in units of any one of a sequence, an image, a slice, and a block;
31. The moving picture decoding apparatus according to any one of claims 28 to 30, wherein the moving picture decoding apparatus includes:   The mask generation means derives the weight based on the distance between the plurality of pixels of the moving region separation mask, which are spatially or temporally close, with respect to the generated moving region separation mask and the difference value of the pixels. Including correction means for performing correction such as isolated point removal, discontinuous point connection, area enlargement / reduction to a rectangular block, edge correction, pixel compensation, pixel matching, etc. 32. The moving picture decoding apparatus according to any one of claims 28 to 31. The predicted image generation means applies to one or more moving region division masks at the same position as the partial block of the prediction target image or a position derived based on a local motion vector mapped with the integer precision. A prediction unit that calculates a ratio of the moving area or the ratio of the background area, and whether either the ratio of the moving area or the ratio of the background area is larger or smaller than a predetermined value. Switching means for switching between,
32. The moving picture decoding apparatus according to claim 28, further comprising: The predicted image generation means includes
A first prediction method that applies different prediction methods to the moving region and the background region based on the moving region separation mask, and the moving region separation included in the block of the first portion of the prediction target image It has a second prediction method in which all mask values are regarded as moving regions and is predicted by a single prediction method, and indicates which of the first prediction method and the second prediction method is used. 32. The moving picture decoding apparatus according to claim 28, further comprising decoding means for decoding information. The predicted image generation means includes
Generating means for generating the predicted image signal by using a pixel value obtained by interpolating the background image signal based on the global vector for the pixel determined as the background region by the moving region separation mask;
Generating means for generating the predicted image signal by using a pixel value obtained by interpolating the reference image signal based on the local motion vector for the pixel determined to be the moving region;
31. The moving picture decoding apparatus according to any one of claims 28 to 30, wherein the moving picture decoding apparatus includes:   The predicted image generation unit generates the predicted image signal using a prediction method that can use the same moving region separation mask or different moving region separation masks for each of the luminance component and the color difference component or for each color component. The moving picture decoding apparatus according to any one of claims 28 to 30.

Images