JP6363975B2 - Moving picture coding apparatus, moving picture coding method, and moving picture coding program - Google Patents

Description

  The present invention relates to a moving image encoding apparatus, a moving image encoding method, and a moving image encoding program.

  HEVC (High Efficiency Video Coding) It was formulated as a H.264 next-generation video coding standard system. HEVC is an H.264 standard. This is an encoding scheme that realizes encoding efficiency exceeding H.264. HEVC adopts a merge mode that applies a surrounding prediction mode as an option of a prediction mode even in inter coding. A technique for further improving the prediction efficiency compared to H.264 and the like has been introduced.

In HEVC, which is the latest video coding standard system, the following predictive image generation method is used in inter coding (see, for example, Non-Patent Document 1).
With respect to the encoding target block, a decoded image before that is set as a candidate for a predicted image, and a region where the error is minimized is searched for within a predetermined region. At this time, a fixed filter called a fractional pixel filter is applied to the entire area to be searched to express a position where the pixels in the search range are shifted by the fractional pixel, and a pixel that does not actually exist in the decoded image is generated. This can also be used for searching. When bi-directional prediction is performed, a weighted sum based on a distance in a time direction from a frame in which a block to be encoded exists is taken to generate a prediction image.

According to the above method, it is also possible to use pixels that are not directly present in the decoded image to generate a predicted image. However, since both the half-pixel generation filter and the bidirectional prediction apply a certain filter without considering the image properties, there is a possibility that an appropriate predicted image cannot be generated. In fact, in a block where a moving object and the background exist together, the position of the object in the reference image and the background shift, so the reference image that matches both the object and the background is extracted as one block. It is hard to be done. In addition, since the entire half-pixel filter shifts all at once, this problem cannot be solved in the same manner. Even when a weighted sum is used, it is difficult to solve this problem because a linear sum is obtained with the same weighting for the whole.
Thus, it is conceivable to use the following image composition technique as a solution.

As one of image synthesis methods, Poisson image synthesis (hereinafter referred to as “PIE”) is known. In PIE, the luminance gradient of one image is maintained and the image is embedded in the other background image. Thereby, a part of a region of a certain image is naturally synthesized with another image as a background (for example, see Non-Patent Document 2). In theory, when the background image is f ^* and the luminance gradient of the embedded image (hereinafter referred to as a gradient image) is v, the method obtains a solution f of the equation (1) shown below to obtain a composite image. Is.

The meaning of the equation (1) will be described. The first equation means that divv and Δf coincide with the luminance gradient of the solution f in (1), which is the synthesized image, on the region Ω. The second equation means that the luminance values of the background image f ^* and f match at the boundary δΩ of Ω. By finding the solution of this equation, it is possible to obtain the desired composite image.

  A technique of newly using an image generated by PIE as a predicted image can be easily inferred from the technique of Non-Patent Document 1. Thus, it is considered that a natural prediction image can be generated even when it is difficult to generate an appropriate prediction image only by weighted prediction as used in conventional inter prediction.

  The following are examples of predicted image generation techniques using this method. That is, this is a technique for improving the prediction efficiency by generating a new predicted image by combining the background image and the gradient image when generating the predicted image. In this technique, generation of a code amount of a background image is suppressed by template matching (see, for example, Non-Patent Document 3). Specifically, as a method for obtaining a more optimal prediction image, a composite image is generated for all combinations of the background image and the gradient image, and both the background image and the gradient image that generate the optimal composite image are used as motion vectors. Encode.

"Recommendation ITU-T H.265: High efficiency video coding", 2013 P. Perez, M. Gangnet, and A. Blake, "Poisson Image Editing", Proc. SIGGRAPH’03, 2003, p.313-318 Mayuko Watanabe, Masaki Kitahara, Satoshi Shimizu, “A Study on Application of Poisson Image Synthesis to Predictive Image Generation”, PCSJ / IMPS2014, 2014, P-1-02

  In generating a composite image, in order to generate an optimal composite image as a predicted image, it is necessary to search for an optimal one as a result of combining a gradient image and a background image. The following two search methods are conceivable.

  (First Method) When an optimal combination of a background image and a gradient image is obtained by a full search, the number of pixels of the encoding target block is X, a gradient image search candidate is A, and a background image search candidate is B , The maximum number of differential operations associated with the search is A × B × X (where X, A, and B are positive integers, respectively). Furthermore, it is necessary to generate a composite image for each search. Note that the number of difference computations associated with the search is counted as the number of occurrences of pixel value difference computations for determining a composite image for a certain encoding target block. For example, in the case of a full search, when determining an optimal combination in the full search, a composite image is generated from a combination of a gradient image and a background image that can be a predicted image candidate of the encoding target block, and the encoding target block is generated. The difference in pixel value from each of the synthesized images is calculated. This number of calculations is the number of difference calculations associated with the search.

(Second Method) On the other hand, even when the gradient image and the background image are searched independently so as to be optimized, the following problem arises.
(1) The number of difference calculations associated with the search needs to be (A + B) × X times.
(2) Since the combinational optimization is not performed, the prediction performance may be deteriorated.

  In view of the above circumstances, the present invention prevents deterioration in accuracy of a predicted image while suppressing the cost of searching for a combination of a gradient image and a background image used for generating a predicted image to be referred to during inter coding. An object of the present invention is to provide a moving image encoding device, a moving image encoding method, and a moving image encoding program.

One aspect of the present invention is a moving image encoding apparatus that encodes an image of a difference between an original image and a predicted image, for each motion vector, a pixel in an encoding target block in the original image, and a reference image A difference calculation unit that calculates a difference in pixel value from a pixel moved by the motion vector from the pixel, a sum calculation unit that calculates a sum of the differences for each motion vector, and a sum of the differences A background image determination unit that selects a small motion vector and determines a background image in the reference image that has been moved from the coding target block by the selected motion vector as the background image; and the variance of the difference for each motion vector A motion calculation unit that calculates a variance of the difference, and selects the motion vector selected from the encoding target block in the reference image. It includes a gradient image determination unit that determines only the moving region in the gradient image, and a predicted image generation unit for generating a predicted image by combining said background image and said gradient image.

One aspect of the present invention is a moving image encoding apparatus that encodes an image of a difference between an original image and a predicted image, for each motion vector, a pixel in an encoding target block in the original image, and a reference image A difference calculation unit that calculates a difference in pixel value from a pixel moved by the motion vector from the pixel, and a partial sum of the differences for each divided region obtained by dividing the encoding target block for each motion vector For each motion vector, a sum calculation unit for calculating the sum of the differences, and a motion vector having a small sum of the differences, and from the block to be encoded in the reference image, A background image determining unit that determines a region moved by the selected motion vector as a background image, and a gradient that selects the divided region having a small partial sum of the differences as a gradient image search region An image search region determination unit, and a variance calculation unit that calculates, for each motion vector, a variance of pixel value differences between the gradient image search region in the original image and the reference image region specified by the motion vector A gradient image determining unit that selects a motion vector having a small variance of the difference and determines, as a gradient image, a region moved by the selected motion vector from the encoding target block in the reference image; and the background image And a predicted image generating unit that generates a predicted image by combining the gradient image and the gradient image.

  One aspect of the present invention is a moving image encoding apparatus that encodes an image of a difference between an original image and a predicted image, for each motion vector, a pixel in an encoding target block in the original image, and a reference image A difference calculation unit that calculates a difference in pixel value from a pixel moved by the motion vector from the pixel, and a partial sum of the differences for each divided region obtained by dividing the encoding target block for each motion vector A partial sum calculation unit for calculating the difference, a sum calculation unit for calculating the sum of the differences for each of the motion vectors, and for each motion vector, the divided region having the smallest partial sum of the differences is selected and selected. A combination generation unit that selects a motion vector having a minimum variance of a partial sum of the divided regions and the difference, and for each motion vector, in the reference image, from the encoding target block, A synthesized image of a background image candidate block moved by a vector and a gradient image candidate block moved by the motion vector selected by the combination generation unit for the motion vector from the encoding target block, and the cost is A background and gradient image determination unit that determines the gradient image candidate block and the background image candidate block used to generate the minimum combined image as a gradient image and a background image, and combines the background image and the gradient image. And a predicted image generation unit that generates a predicted image.

One aspect of the present invention is a moving image encoding method executed by a moving image encoding apparatus that encodes an image of a difference between an original image and a predicted image, and an encoding target in the original image for each motion vector A difference calculating step of calculating a pixel value difference between a pixel in the block and a pixel at a position moved by the motion vector from the pixel in a reference image; and sum calculation for calculating a sum of the differences for each of the motion vectors A background image determination step of selecting a motion vector having a small sum of differences, and determining a region moved by the selected motion vector from the coding target block in the reference image as a background image; and the motion For each vector, a variance calculating step for calculating the variance of the difference and a motion vector having a small variance of the difference are selected and From the encoding target block, the gradient image determining step of determining a moved only the motion vector selected area to the gradient image, the predicted image generation step of generating a predicted image composition to the said background image and said gradient image Have.

  One aspect of the present invention is a moving image encoding program that causes a computer to function as any of the above-described moving image encoding devices.

  According to the present invention, it is possible to prevent deterioration in accuracy of a predicted image while suppressing the cost for searching for a combination of a gradient image and a background image used for generating a predicted image to be referred to during inter coding.

It is a functional block diagram which shows the structure of the moving image encoder by one Embodiment of this invention. It is a functional block diagram which shows the detailed structure of the motion vector estimation part by 1st Embodiment. It is a figure which shows the search range of the background image and gradient image by the embodiment. It is a processing flow which shows the encoding process of the motion vector estimation part by the embodiment. It is a figure which shows the division | segmentation of the encoding object block by 2nd Embodiment. It is a figure which shows the search range of the background image and gradient image by the embodiment. It is a functional block diagram which shows the detailed structure of the motion vector estimation part by the embodiment. It is a processing flow which shows the encoding process of the motion vector estimation part by the embodiment. It is a functional block diagram which shows the detailed structure of the motion vector estimation part by 3rd Embodiment. It is a processing flow which shows the encoding process of the motion vector estimation part by the embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a functional block diagram showing a configuration of a moving picture coding apparatus 100 according to an embodiment of the present invention, and only functional blocks related to the present embodiment are extracted and shown. The moving image encoding apparatus 100 handles a moving image composed of a plurality of images (frames) that are continuous on the time axis.

  The moving image encoding apparatus 100 includes an encoding control unit 101, a DCT (Discrete Cosine Transform) / quantization unit 102, an inverse quantization / IDCT (Inverse Discrete Cosine Transform) unit 103, a loop A filter 104, a motion vector prediction unit 105, a motion search unit 106 (predicted image generation unit), an intra prediction unit 107, a selection unit 108, and an entropy encoding unit 109 are provided. The encoding control unit 101, DCT / quantization unit 102, inverse quantization / IDCT unit 103, loop filter 104, motion search unit 106, intra prediction unit 107, selection unit 108, and entropy encoding unit 109 are respectively conventional. The functional unit that performs a part of the function of the image encoding process. Therefore, the following description describing these functions is merely an example, and other configurations may be adopted as long as moving image encoding can be realized.

  The encoding control unit 101 controls encoding processing for input moving image data (hereinafter referred to as “original image”). For example, the encoding control unit 101 controls the DCT / quantization unit 102 and the intra prediction unit 107. Note that the original image is an individual frame (image) of the moving image that is the target of the encoding process.

  The DCT / quantization unit 102 performs discrete cosine transform and quantization processing on a residual signal obtained as a difference between the original image and the predicted image output from the motion search unit 106 or the intra prediction unit 107. The inverse quantization / IDCT unit 103 performs an inverse quantization process and an IDCT process on the residual signal quantized by the DCT / quantization unit 102. The loop filter 104 outputs a result obtained by performing loop filtering on the output of the inverse quantization / IDCT unit 103 to the motion search unit 106 and the motion vector prediction unit 105 as a reference image. The motion vector prediction unit 105 performs motion vector prediction based on the original image and the reference image output from the loop filter 104. The motion vector prediction unit 105 determines a background image and a gradient image to be combined with the background image for each encoding target block by motion vector prediction, and outputs the determined result to the motion search unit 106.

  The motion search unit 106 generates a predicted image by synthesizing the background image and the gradient image determined by the motion vector predicting unit 105 by PIE for each block to be encoded. The intra prediction unit 107 generates a predicted image by executing an intra-screen prediction process. The selection unit 108 selects and outputs either the prediction image generated by the motion search unit 106 or the prediction image generated by the intra prediction unit 107. The difference between the original image and the predicted image selected by the selection unit 108 is input to the DCT / quantization unit 102. The entropy encoding unit 109 performs entropy encoding processing on the data quantized by the DCT / quantization unit 102 and outputs the result as a bit stream.

  The moving image encoding apparatus 100 performs encoding processing according to the following method 1 or method 2. In the encoding process described below, the order of processing can be reversed or parallel as long as no contradiction occurs.

<Method 1>
In Method 1, the moving image coding apparatus 100 first obtains a difference between an original image and a reference image, and determines a background image and a gradient image in the reference image using the obtained difference. As a result, the number of calculation of the difference is reduced. The encoding of method 1 is performed as follows.

(Processing 1) The moving image coding apparatus 100, for each possible motion vector, in the background image search, each pixel in the encoding target block of the original image, and a pixel of the reference image at a position moved by the motion vector from the pixel. The difference between the pixel values is obtained.
(Processing 2) The moving picture coding apparatus 100 obtains the sum (absolute value sum, etc.) of the differences obtained in Process 1 for each motion vector, and determines the background image based on the motion vector that minimizes the sum.
(Process 3) For each motion vector, the moving image encoding apparatus 100 determines the variance of the difference obtained in Process 1 and determines a gradient image based on the motion vector that minimizes the variance.

  According to the technique 1, when the gradient image and the background image search candidates overlap (A = B), the number of difference calculations can be suppressed to A × X times. In Method 1, the gradient image is calculated by using the dispersion of pixel values instead of the gradient (differential value) used in the prior art. As a result, it is possible to change a part of the calculation process of each cost of the gradient image and the background image into a commonly obtained form (difference between pixels at corresponding positions of the encoding target image and the reference image). It becomes. Therefore, it is possible to reduce the total number of difference calculations.

<Method 2>
In technique 2, in addition to technique 1, moving picture coding apparatus 100 obtains a partial sum of differences for each divided area obtained by dividing the coding area. This searches for a gradient image that improves a region in the background image where the difference from the original image is larger. The encoding of method 2 is as follows.

(Processing 1) The moving image coding apparatus 100, for each possible motion vector, in the background image search, each pixel in the encoding target block of the original image, and a pixel of the reference image at a position moved by the motion vector from the pixel. The difference between the pixel values is obtained.
(Processing 2) For each motion vector, the moving image encoding apparatus 100 calculates a sum of differences (such as an absolute value sum) obtained in Process 1 and a partial sum of differences for each divided region.
(Process 3) The moving image encoding apparatus 100 obtains a divided region in which the partial sum of the differences is maximized for each of the N motion vectors in order from the smallest sum of the differences, and distributes the divided regions and the differences. The motion vector of the reference image area in which is minimum is specified.
(Processing 4) The moving image encoding apparatus 100 generates a composite image from the background image candidates and gradient image candidate images specified by the motion vectors for the combination of the N pattern motion vectors obtained in Process 3. Generate and select the combination with the lowest cost. The moving image encoding apparatus 100 determines a combination of a background image and a gradient image used for generating a predicted image based on the selected combination.

  In Method 2, when determining a gradient image suitable for each background image in Method 1, instead of calculating the total gradient as in the conventional technique, the optimal gradient image for the region where the background image is deteriorated is distributed. Determined by Since only a composite image that can be a candidate can be generated and compared, a composite image with higher prediction performance can be generated while suppressing an increase in the amount of calculation from Method 1.

  Hereinafter, each embodiment for realizing the method 1 and the method 2 will be described.

[First Embodiment]
The moving image encoding apparatus 100 according to the present embodiment implements Method 1.
FIG. 2 is a functional block diagram showing a detailed configuration of the motion vector prediction unit 105 according to the present embodiment. The motion vector prediction unit 105 includes a difference calculation unit 151, a sum calculation unit 153, a background image determination unit 154, a variance calculation unit 155, and a gradient image determination unit 156. The difference calculation unit 151 calculates, for each motion vector that can be taken within the search range, a pixel value difference between each pixel in the encoding target block of the original image and a pixel in the reference image at a position moved by the motion vector from that pixel. calculate. The difference calculation unit 151 stores the calculation result in a memory 152 provided inside (may be provided outside). The sum calculation unit 153 obtains the sum of the differences calculated by the difference calculation unit 151 for each motion vector. The background image determination unit 154 determines a region in the reference image in which the block to be encoded is moved by the motion vector that minimizes the sum of differences as the background image. The variance calculation unit 155 obtains the variance of the difference calculated by the difference calculation unit 151 for each motion vector. The gradient image determination unit 156 determines a region in the reference image in which the encoding target block is moved by the motion vector having the minimum variance as a gradient image.

FIG. 3 is a diagram illustrating the search range of the background image and the gradient image. As shown in the figure, the encoding target block P is an area of [X, X + M] × [Y, Y + N], and the search range Q of the background image and the gradient image in the reference image is [X + X0, X + M + X1] × This is an area of [Y + Y0, Y + N + Y1]. The search range Q of the background image and the gradient image is, for example, a conventional motion vector used for a surrounding encoded image, such as a search center used when determining a motion vector in HEVC, or a background image set by this synthesis. A search center is determined from the gradient image, and an area of M × N (M and N are integers greater than 0) centered on the search center is considered.
In the following, the pixel value of the (x, y) coordinate of the original image is O (x, y), the pixel value of the (x, y) coordinate of the gradient image in the reference image is R (x, y), and the reference image The pixel value of the (x, y) coordinate of the background image in is described as R ′ (x, y).

Next, the encoding process of the motion vector prediction unit 105 will be described.
FIG. 4 is a processing flow showing the encoding process of the motion vector prediction unit 105.
First, the difference calculation unit 151 calculates the difference and stores the calculation result in the memory (step S110). The background image motion vector for determining the background image candidate block is (a, b) ε [X0, X1] × [Y0, Y1]. The difference calculation unit 151 calculates the difference D (x, y, a, b) shown in the following equation (2) for the combination of each pixel in the encoding target block P and each background image motion vector that can be taken in the search range Q. ) Is calculated and stored in the memory 152. The difference D (x, y, a, b) is moved by the pixel value O (x, y) of the pixel in the encoding target block P in the original image and the background image motion vector (a, b) from the pixel. This is the difference in pixel value R (x + a, y + b) of the pixel in the background image at the position.

  In equation (2), dis means a difference.

  The sum calculation unit 153 reads the difference D (x, y, a, b) from the memory 152. The sum calculation unit 153 calculates the difference sum S (a, b) for each background image motion vector (a, b) ε [X0, X1] × [Y0, Y1] (step S120). The difference sum S (a, b) is calculated as the sum S (a, b) of the absolute values of the difference D (x, y, a, b) as shown in the following equation (3). Note that the sum of squares of D (x, y, a, b) may be used as the sum of differences S (a, b) instead of the sum of absolute values of D (x, y, a, b). Good.

The background image determination unit 154 determines a background image using the difference sum S (a, b) obtained for each background image motion vector (a, b) (step S130). That is, the background image determination unit 154 calculates the sum in step S120 from the background image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1] as shown in the following equation (4). The background image motion vector (A ₁ , B ₁ ) indicating the region where the difference sum S (a, b) calculated by the unit 153 is minimum is determined and encoded. The background image determination unit 154 sets a background image candidate block determined by the background image motion vector (A ₁ , B ₁ ) as a background image. That is, the background image determination unit 154 sets a reference image area that has been moved from the encoding target block P by the background image motion vector (A ₁ , B ₁ ) as a background image. The background image determination unit 154 outputs information about the background image of the encoding target block P such as the encoded background image motion vector (A ₁ , B ₁ ) to the motion search unit 106.

The background image determination unit 154 uses the value obtained by adding the cost such as the code amount to the sum S (a, b) instead of the sum S (a, b), and similarly uses the background image motion vector (A _1). , B ₁ ) may be determined.

  Subsequently, the variance calculating unit 155 determines the difference D (x, y, a, b) for each gradient image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1] that determines the gradient image candidate block. ) Is calculated (step S140). That is, the variance calculation unit 155 reads the value D from the memory 152, and calculates the difference D (x, y, a, b) for each of the gradient image motion vectors (a, b) as shown in the following equation (5). ) Dispersion varV (a, b).

  In the formula (5), ave means an average value. According to equation (5), when the variance varV (a, b) ≧ 0 and the value of the difference D is constant regardless of (x, y) in the encoding target block P, varV (a, b) = 0.

The gradient image determination unit 156 determines a gradient image using the variance varV (a, b) calculated for each gradient image motion vector (a, b) (step S150). That is, the gradient image determination unit 156 determines a gradient image motion vector (A ₂ , B ₂ ) indicating a region where the gradient is minimum, as shown in the following formula (6). That is, the gradient image determination unit 156 calculates the variance variance varV (a of the difference calculated by the variance calculation unit 155 in step S140 from the gradient image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1]. , B) is determined as a gradient image motion vector (A ₂ , B ₂ ) and encoded. The gradient image determination unit 156 sets a gradient image candidate block determined by the gradient image motion vector (A ₂ , B ₂ ) as a gradient image. That is, the gradient image determination unit 156 sets the region of the reference image that is moved from the encoding target block P by the gradient image motion vector (A ₂ , B ₂ ) as the gradient image. The gradient image determination unit 156 outputs the gradient image information of the encoding target block P such as the encoded gradient image motion vector (A ₂ , B ₂ ) to the motion search unit 106.

The reason for using the variance of the difference to obtain the gradient image is as follows.
The gradient image, that is, R (x, y) of the reference image having a gradient close to the gradient of the original image is given by the following equation (7).

  In order to minimize these from the first and second terms in the last row, for all the differences D, D (x + 1, y, a, b) ≈D (x, y, a, b), D It is a sufficient condition that (x, y + 1, a, b) ≈D (x, y, a, b). When the gradient image motion vector (a, b) that minimizes the variance varV (a, b) of the difference calculated by the equation (5) is determined, D (x + 1, It is expected that y, a, b) ≈D (x, y, a, b) and D (x, y + 1, a, b) ≈D (x, y, a, b). At this time, it is expected that the first term and the second term in the last line of Expression (7) take a minimum value or a value close to the minimum value. Thus, by using the variance of the difference instead of using the first expression of Expression (7), it is possible to obtain an optimal or near-optimal image as the gradient image.

[Second Embodiment]
The moving image encoding apparatus 100 according to the present embodiment realizes Method 1, but differs in that the encoding target block P is divided into a plurality of blocks. In the following, the present embodiment will be described with a focus on differences from the first embodiment.

FIG. 5 is a diagram illustrating division of the encoding target block. In the present embodiment, one encoding target block P is divided into N areas. The divided areas are set as divided areas P _{1 to} P _N. The encoding target block P is divided so that the divided areas P _{1 to} P _N are rectangular. Hereinafter, a case where N = 4 will be described as an example.

  FIG. 6 is a diagram illustrating the search range of the background image and the gradient image. Similar to the first embodiment, the encoding target block P is an area of [X, X + M] × [Y, Y + N], and the search range Q of the background image and the gradient image in the reference image is [X + X0, X + M + X1]. ] × [Y + Y0, Y + N + Y1].

The moving picture encoding apparatus 100 according to the present embodiment is configured to include a motion vector prediction unit 205 illustrated in FIG. 7 instead of the motion vector prediction unit 105 according to the first embodiment illustrated in FIG. 2.
FIG. 7 is a diagram illustrating a detailed configuration of the motion vector prediction unit 205 according to the present embodiment. The motion vector prediction unit 205 includes a difference calculation unit 251, a partial sum calculation unit 253, a sum calculation unit 254, a background image determination unit 255, a gradient image search region determination unit 256, a variance calculation unit 257, and a gradient image determination unit 258. Prepare.

For each motion vector that can be taken within the search range, the difference calculation unit 251 calculates a difference in pixel value between each pixel in the encoding target block of the original image and a pixel in the reference image at a position moved by the motion vector from that pixel. calculate. The difference calculation unit 251 stores the calculation result in a memory 252 provided inside (may be provided outside). The partial sum calculation unit 253 obtains a partial sum of differences between pixel values of the encoding target block and the reference image for each divided region for each motion vector. The sum calculation unit 254 calculates the sum of differences between pixel values of the encoding target block and the reference image for each motion vector. The background image determination unit 255 determines a motion vector to an area where the sum of differences is minimum as a background image motion vector. The gradient image search region determination unit 256 sets the divided region having the maximum partial sum among the divided regions P _{1 to} P _N as the gradient image search region. The variance calculation unit 257 calculates the variance of the difference between the pixel values of the gradient image search region and the reference image for each motion vector. The gradient image determination unit 258 determines, in the gradient image, the region in the reference image to which the encoding target block has been moved, based on the motion vector that minimizes the variance of the difference calculated by the variance calculation unit 257.

Next, the encoding process of the motion vector prediction unit 205 will be described.
FIG. 8 is a processing flow showing the encoding process of the motion vector predicting unit 205 of the present embodiment.
First, the difference calculation unit 251 calculates the difference and stores the calculation result in the memory, similarly to step S110 of the first embodiment (step S210). The difference calculation unit 251 determines a combination of each pixel in the encoding target block P and each of the background image motion vectors (a, b) ∈ [X0, X1] × [Y0, Y1] that can be taken in the search range Q. The difference D (x, y, a, b) is calculated by equation (2) and stored in the memory 252.

Next, the partial sum calculation unit 253 reads the difference D (x, y, a, b) from the memory 252. The partial sum calculation unit 253 calculates a partial sum S _k (a, b) of the difference for each divided region for each background image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1]. The difference partial sum S _k (a, b) is calculated as a partial sum of absolute values of the difference D (x, y, a, b) as shown in the following equation (8). As a partial sum S _k (a, b) of the difference, a square portion of D (x, y, a, b) is used instead of a partial sum of absolute values of D (x, y, a, b). A sum or the like may be used. The sum calculation unit 254 uses the calculation result of the partial sum calculation unit 253, and shows the following equation (9) for each background image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1]. Thus, the sum S (a, b) of the differences is calculated (step S220).

As in the first embodiment, the background image determination unit 255 determines a background image using the difference sum S (a, b) obtained for each background image motion vector (a, b) (step S230). ). That is, the background image determination unit 255 calculates the sum in step S220 from the background image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1] as shown in the following equation (10). The background image motion vector (A ₁ , B ₁ ) indicating the region where the difference sum S (a, b) calculated by the unit 254 is minimum is determined and encoded. The background image determination unit 255 sets a background image candidate block (region) determined by the background image motion vector (A ₁ , B ₁ ) as a background image. The background image determination unit 255 outputs information on the background image of the encoding target block P such as the encoded background image motion vector (A ₁ , B ₁ ) to the motion search unit 106.

The background image determination unit 255 uses the value obtained by adding the cost such as the code amount to the sum S (a, b) instead of the sum S (a, b), and similarly uses the background image motion vector (A _1). , B ₁ ) may be determined.

The gradient image search area determination unit 256 determines a gradient image search area using the partial sum S _j (a, b) of the difference (step S240). First, the gradient image search region determination unit 256 sets a divided region P _{max in} which the difference between the pixel value of the background image and the pixel value of the original image is large among the divided regions P _{1 to} P _N of the following equation (11). Set as follows. The divided area P _max becomes the gradient image search area.

Subsequently, the variance calculation unit 257 calculates the variance of the difference D (x, y, a, b) of each gradient image motion vector (a, b) for the gradient image search region (step S250). In other words, the variance calculation unit 257 for each of the gradient image motion vectors (a, b) ∈ [X0, X1] × [Y0, Y1] from the memory 152, and the pixels in the divided area _Pmax of the original image The pixel value difference D (x, y, a, b) from the pixel of the reference image at the position moved by the gradient image motion vector is read out. The variance calculation unit 257 calculates the variance varV (a, b) of the read difference D (x, y, a, b) for each of the background image motion vectors (a, b) as in the following equation (12). Is calculated.

The gradient image determination unit 258 determines a gradient image using the difference variance varV (a, b) calculated in step S250 (step S260). That is, the gradient image determination unit 258 calculates the gradient image motion vector (A ₂ , B ₂ ) indicating the divided region P _max determined in step S240 and the region of the reference image where the gradient is minimum as the following formula (13 ). That is, the gradient image determination unit 258 has the smallest variance varV (a, b) calculated in step S250 from the gradient image motion vector (a, b) ∈ [X0, X1] × [Y0, Y1]. Are determined as gradient image motion vectors (A ₂ , B ₂ ) and encoded. The gradient image determination unit 258 sets, as a gradient image, a region in the reference image that has been moved from the encoding target block P by the gradient image motion vector (A ₂ , B ₂ ). The gradient image determination unit 258 outputs the gradient image information of the encoding target block P such as the encoded gradient image motion vector (A ₂ , B ₂ ) to the motion search unit 106.

[Third Embodiment]
This embodiment implements Method 2 described above. This embodiment will be described with a focus on differences from the second embodiment.
In the present embodiment, the encoding target block P is divided into a plurality of parts as in the second embodiment. That is, as shown in FIG. 5, one encoding target block P is divided into N divided regions P _{1 to} P _N each having a rectangular shape.
Further, the search range of the background image and the gradient image of this embodiment is the same as that of the second embodiment shown in FIG.

The moving picture coding apparatus 100 according to the present embodiment is configured to include a motion vector prediction unit 305 illustrated in FIG. 9 instead of the motion vector prediction unit 105 according to the first embodiment illustrated in FIG. 2.
FIG. 9 is a diagram illustrating a detailed configuration of the motion vector prediction unit 305 according to the present embodiment. The motion vector prediction unit 305 includes a difference calculation unit 351, a partial sum calculation unit 353, a sum calculation unit 354, a combination generation unit 355, and a background and gradient image determination unit 356.

  The difference calculation unit 351 calculates, for each motion vector that can be taken within the search range, a pixel value difference between each pixel in the encoding target block of the original image and a pixel of the reference image at a position moved by the motion vector from that pixel. calculate. The difference calculation unit 351 stores the calculation result in a memory 352 provided inside (may be provided outside). The partial sum calculation unit 353 obtains a partial sum of differences between pixel values of the encoding target block and the reference image for each divided region for each motion vector. The sum calculation unit 354 calculates the sum of differences between pixel values of the encoding target block and the reference image for each motion vector. For each motion vector, the combination generation unit 355 identifies a divided region that maximizes the partial sum of the differences, and obtains a motion vector that minimizes the variance of the divided regions and the difference. Thereby, a combination of motion vectors is generated. For each combination of motion vectors generated by the combination generation unit 365, the background and gradient image determination unit 356 generates a composite image from the background image candidates and the gradient image candidate images specified by the motion vectors. The background and gradient image determination unit 356 determines the background image and the gradient image based on the combination of motion vectors that minimizes the cost.

Next, the encoding process of the motion vector prediction unit 305 will be described.
FIG. 10 is a diagram illustrating a processing flow of the encoding process of the motion vector prediction unit 305 of the present embodiment.
First, the difference calculation unit 351 calculates the difference and stores the calculation result in the memory (step S310). The difference calculating unit 351 uses an expression (for the combination of each pixel in the encoding target block P and each motion vector (a, b) ∈ [X0, X1] × [Y0, Y1] that can be taken in the search range Q). The difference D (x, y, a, b) is calculated according to 2) and stored in the memory 352.

Next, the partial sum calculation unit 353 reads the difference D (x, y, a, b) from the memory 352. For each motion vector (a, b) ∈ [X0, X1] × [Y0, Y1], the partial sum calculation unit 353 calculates the partial sum S _k (for each divided region as shown in the following equation (14). a, b) are calculated. The sum calculation unit 354 uses the calculation result of the partial sum calculation unit 353 to calculate each motion vector (a, b) ∈ [X0, X1] × [Y0, Y1] as shown in the following equation (15). Then, the difference sum S (a, b) is calculated (step S320).

The combination generation unit 355 sets V ₁ , V ₂ , V ₃ ,..., V _{n in} order from the motion vector (a, b) that minimizes the difference sum S (a, b). At this time, the combination generation unit 355 determines a combination of each motion vector (a, b) and a divided region where the partial sum S _k (a, b) of the difference is maximum for the motion vector (a, b). The following is performed (step S330). Here, k ₁ ,..., K _N are each an integer from 1 to N.

Here, the combination generation unit 355 uses the difference D (x, y, a, b) for each divided region P _kl (l = 1,..., N; l is a subscript of k) to search for a difference in search range. Is obtained (step S340). Specifically, the combination generating unit 355, the variance of the difference of the search range for the divided area _{P kl,} instead of (x, y) ∈P _max of formula (12), (x, y ) ∈P kl Calculate using. The combination generation unit 355 selects a motion vector (a, b) that minimizes the variance of the calculated difference for the divided region P _{kl and} sets it as a vector U _kl (l is a subscript of k) (step S350). Here, for i (i ≠ j, i <j) where k _i = k _j , since the vector U _kj obtained at the j-th is the same as the vector U _ki obtained at the i-th, a new one is obtained. There is no need.
These combinations are summarized as follows.

The background and gradient image determination unit 356 generates a composite image from the background image candidate block and the gradient image candidate block specified by (V _l , U _kl ) obtained from the above combination (step S360). The background image candidate block is an area that is moved by the motion vector V ₁ from the encoding target block in the reference image. Further, the gradient image candidate block is an area that is moved by a motion vector V _kl from the encoding target block in the reference image. The background and gradient image determination unit 356 determines the combination of the background image candidate block and the gradient image candidate block with the lowest cost in the generated composite image as the combination of the background image and the gradient image (step S370). The cost is calculated from the code amount, for example. The background and gradient image determination unit 356 outputs information on the determined combination of the background image and the gradient image to the motion search unit 106.

  According to the first to third embodiments described above, the calculation amount can be further reduced as compared with the conventional second method for independently searching the gradient image and the background image, and the prediction performance is improved as compared with the second method. I can expect. According to the third embodiment, the difference between the background image and the original image is considered to be larger than that in the first and second embodiments in which the gradient image and the background image are determined independently. On the other hand, since a gradient image is set and a composite image is generated, an improvement in prediction performance can be expected while suppressing an increase in calculation amount.

  You may make it implement | achieve the function of a one part or all function part of the moving image encoder 100 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can be used for a moving image encoding apparatus that encodes a difference between a picture of an original image and a picture of a predicted image using temporal correlation of pictures.

DESCRIPTION OF SYMBOLS 100 Moving image encoder 101 Encoding control part 102 DCT / quantization part 103 Inverse quantization / IDCT part 104 Loop filter 105,205,305 Motion vector prediction part 106 Motion search part 107 Intra prediction part 108 Selection part 109 Entropy code Conversion unit 151,251,351 Difference calculation unit 152,252,352 Memory 153,254,354 Sum calculation unit 154,255 Background image determination unit 155,257 Dispersion calculation unit 156,258 Gradient image determination unit 253,353 Partial sum calculation Unit 256 gradient image search region determination unit 355 combination generation unit 356 background and gradient image determination unit

Claims (5)

A moving image encoding device that encodes an image of a difference between an original image and a predicted image,
For each motion vector, a difference calculation unit that calculates a pixel value difference between a pixel in the encoding target block in the original image and a pixel at a position moved by the motion vector from the pixel in the reference image;
A sum calculator for calculating the sum of the differences for each motion vector;
A background image determination unit that selects a motion vector having a small sum of differences and determines a region that has been moved by the selected motion vector from the encoding target block in the reference image as a background image;
A variance calculating unit that calculates the variance of the difference for each motion vector;
A gradient image determining unit that selects a motion vector having a small variance of the difference, and determines a region moved by the selected motion vector from the coding target block in the reference image as a gradient image;
A predicted image generating unit that generates a predicted image by combining the background image and the gradient image;
A moving picture encoding apparatus comprising: A moving image encoding device that encodes an image of a difference between an original image and a predicted image,
For each motion vector, a difference calculation unit that calculates a pixel value difference between a pixel in the encoding target block in the original image and a pixel at a position moved by the motion vector from the pixel in the reference image;
For each motion vector, a partial sum calculation unit that calculates a partial sum of the differences for each divided region obtained by dividing the encoding target block;
A sum calculator for calculating the sum of the differences for each motion vector;
A background image determination unit that selects a motion vector having a small sum of differences and determines a region that has been moved by the selected motion vector from the encoding target block in the reference image as a background image;
A gradient image search region determination unit that selects the divided region having a small partial sum of differences as a gradient image search region;
For each motion vector, a variance calculation unit that calculates a variance of pixel value differences between the gradient image search region in the original image and the region of the reference image specified by the motion vector;
A gradient image determining unit that selects a motion vector having a small variance of the difference, and determines a region moved by the selected motion vector from the coding target block in the reference image as a gradient image;
A predicted image generating unit that generates a predicted image by combining the background image and the gradient image;
A moving picture encoding apparatus comprising: A moving image encoding device that encodes an image of a difference between an original image and a predicted image,
For each motion vector, a difference calculation unit that calculates a pixel value difference between a pixel in the encoding target block in the original image and a pixel at a position moved by the motion vector from the pixel in the reference image;
For each motion vector, a partial sum calculation unit that calculates a partial sum of the differences for each divided region obtained by dividing the encoding target block;
A sum calculator for calculating the sum of the differences for each motion vector;
For each of the motion vectors, a combination generation unit that selects the divided region with the smallest partial sum of the differences, and selects the selected divided region and a motion vector with the smallest variance of the partial sum of the differences;
For each motion vector, in the reference image, a background image candidate block moved by the motion vector from the encoding target block, and the motion selected by the combination generation unit for the motion vector from the encoding target block A background for generating a composite image with a gradient image candidate block moved by a vector and determining the gradient image candidate block and the background image candidate block used for generating the composite image with the lowest cost as a gradient image and a background image And a gradient image determination unit;
A predicted image generating unit that generates a predicted image by combining the background image and the gradient image;
A moving picture encoding apparatus comprising: A moving image encoding method executed by a moving image encoding device that encodes a difference image between an original image and a predicted image,
For each motion vector, a difference calculation step of calculating a difference in pixel value between a pixel in the encoding target block in the original image and a pixel at a position moved by the motion vector from the pixel in the reference image;
A sum calculating step for calculating the sum of the differences for each motion vector;
A background image determining step of selecting a motion vector having a small sum of differences and determining, as a background image, a region that is moved by the selected motion vector from the encoding target block in the reference image;
A variance calculating step for calculating a variance of the difference for each motion vector;
A gradient image determination step of selecting a motion vector having a small variance of the difference, and determining a region moved by the selected motion vector from the encoding target block in the reference image as a gradient image;
A predicted image generation step of generating a predicted image by combining the background image and the gradient image;
A moving picture encoding method comprising: Computer
A moving picture coding program that functions as the moving picture coding apparatus according to any one of claims 1 to 3.

Images