JP2023053272A - Image encoding device, image decoding device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve coding efficiency.

SOLUTION: An image encoding device (1) encodes a target image in block units obtained by dividing a current image in frame units. The image encoding device (1) includes: a prediction unit (172) for predicting the target image using a plurality of reference images to generate a predicted image; an evaluation unit (180) for generating map information indicating a distribution of an error in the predicted image by evaluating similarity between the plurality of reference images in pixel units; a subtraction unit (110) for calculating a prediction residual indicating a difference between the target block and the prediction image in pixel units; a determination unit (190) for determining an orthogonal transformation to be applied to the prediction residual on the basis of the map information; and a transformation unit (121) for performing an orthogonal transformation process on the prediction residual by the determined orthogonal transformation.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to an image encoding device, an image decoding device, and a program.

Conventionally, in an image coding apparatus that encodes a target image in units of blocks obtained by dividing a current image in units of frames, a prediction image is generated by predicting a target image using a plurality of reference images, and a target image is generated. A known method is to calculate transform coefficients by performing orthogonal transform processing on a prediction residual that indicates the difference between an image and a predicted image, quantize and entropy-encode the transform coefficients, and output encoded data. .

Also, like the image encoding device, the image decoding device predicts a target image using a plurality of reference images to generate a predicted image. An image decoding device decodes encoded data to obtain transform coefficients, performs inverse quantization, performs inverse orthogonal transform processing on the transform coefficients after inverse quantization, calculates prediction residuals, and outputs predicted images and predictions. The target image is decoded by synthesizing the residuals.

HEVC defines two types of orthogonal transforms, DCT-2 and DST-7, as applicable to transform processing (orthogonal transform processing and inverse orthogonal transform processing) (see Non-Patent Document 1). Specifically, in HEVC, based on the block size of the target image and the mode of intra prediction to be applied to the target image, which of the two types of orthogonal transform is to be applied is determined.

Recommendation ITU-T H. 265, (12/2016), "High efficiency video coding", International Telecommunications Union

However, if the type of orthogonal transform is only determined based on the block size of the target image and the intra prediction mode to be applied to the target image, it is possible to apply the optimal type of orthogonal transform according to the energy distribution in the prediction residual. Can not. For example, even if DCT-2 is originally more efficient in concentrating the energy of prediction residuals, there are cases where DST-7 is determined as the orthogonal transform to be applied, which reduces coding efficiency. there is a problem

KLT is an orthogonal transform method that concentrates the energy of prediction residuals more efficiently. Therefore, there is a problem that the amount of information to be transmitted increases and the coding efficiency decreases.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an image encoding device, an image decoding device, and a program capable of improving encoding efficiency.

An image encoding device according to a first feature is an image encoding device that encodes a target image in units of blocks obtained by dividing a current image in units of frames, the target image using a plurality of reference images. a prediction unit that predicts an image to generate a predicted image; an evaluation unit that generates map information indicating the distribution of errors in the predicted image by evaluating the similarity between the plurality of reference images on a pixel-by-pixel basis; a subtraction unit that calculates a prediction residual indicating a difference between the target block and the predicted image on a pixel-by-pixel basis; a determination unit that determines an orthogonal transform to be applied to the prediction residual based on the map information; and a transform unit that performs orthogonal transform processing on the prediction residual by orthogonal transform. An image encoding device according to another feature is an image encoding device that encodes a target image in units of blocks obtained by dividing a current image in units of frames, wherein the target image is encoded using a plurality of reference images. and a prediction unit that generates a predicted image by predicting the reference images, and a sum of absolute differences indicating the similarity between the plurality of reference images is calculated for each region that is smaller than the block and is made up of a plurality of pixels. and an evaluation unit, wherein the evaluation unit controls encoding processing based on the sum of absolute differences calculated for each area.

An image decoding device according to a second feature is an image decoding device that decodes a target image in units of blocks obtained by dividing a current image in units of frames, and acquires transform coefficients by decoding encoded data. a decoding unit that predicts the target image using a plurality of reference images to generate a predicted image; and a similarity between the plurality of reference images is evaluated in units of pixels, so that an evaluation unit that generates map information indicating the distribution of errors; a determination unit that determines an inverse orthogonal transform to be applied to the transform coefficients based on the map information; and an inverse orthogonal transform to the transform coefficients by the determined inverse orthogonal transform. and an inverse transform unit that performs transform processing. An image decoding device according to another feature is an image decoding device that decodes a target image in units of blocks obtained by dividing a current image in units of frames, and predicts the target image using a plurality of reference images. a predicting unit for generating a predicted image by using a prediction unit; and an evaluating unit for calculating a difference absolute value sum indicating a degree of similarity between the plurality of reference images in units of regions smaller than the block and made up of a plurality of pixels. , wherein the evaluation unit controls decoding processing based on the sum of absolute differences calculated for each region.

A program according to a third feature is summarized to cause a computer to function as the image encoding device according to the first feature.

A gist of a program according to a fourth feature is to cause a computer to function as the image decoding device according to the second feature.

ADVANTAGE OF THE INVENTION According to this invention, the image encoding apparatus which can improve encoding efficiency, an image decoding apparatus, and a program can be provided.

It is a figure which shows the structure of the image coding apparatus which concerns on 1st Embodiment. FIG. 4 is a diagram showing an example of inter prediction according to the first to third embodiments; FIG. It is a figure which shows an example of a structure of the evaluation part which concerns on 1st thru|or 3rd embodiment. It is a figure which shows operation|movement of the adaptive conversion production|generation part which concerns on 1st Embodiment. It is a figure which shows the structure of the image decoding apparatus which concerns on 1st Embodiment. FIG. 11 is a diagram showing the configuration of an image encoding device according to a second embodiment; FIG. FIG. 10 is a diagram showing the configuration of an image decoding device according to a second embodiment; FIG. FIG. 11 is a diagram showing the configuration of an image encoding device according to a third embodiment; FIG. It is a figure which shows operation|movement of the feature-value evaluation part which concerns on 3rd Embodiment. FIG. 11 is a diagram showing the configuration of an image decoding device according to a third embodiment; FIG.

An image encoding device and an image decoding device according to embodiments will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

<First Embodiment>
An image encoding device and an image decoding device according to the first embodiment will be described. An image encoding device and an image decoding device according to the first embodiment encode and decode moving images represented by MPEG.

(Image encoding device)
FIG. 1 is a diagram showing the configuration of an image encoding device 1 according to the first embodiment. As shown in FIG. 1, the image coding apparatus 1 includes a block division unit 100, a subtraction unit 110, a transform/quantization unit 120, an entropy coding unit 130, an inverse quantization/inverse transform unit 140, It comprises a synthesis unit 150 , a memory 160 , a prediction unit 170 , an evaluation unit 180 and a determination unit 190 .

The block division unit 100 divides an input image in units of frames (or pictures) constituting a moving image into small block-shaped regions, and outputs the blocks obtained by the division to the subtraction unit 110 . The block size is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the block is not limited to square, and may be rectangular. A block is a unit for encoding by the image encoding device 1 and a unit for decoding by the image decoding device 2 .

The subtraction unit 110 calculates a prediction residual indicating a pixel-by-pixel difference between the block input from the block division unit 100 and a prediction image (prediction block) obtained by predicting the block by the prediction unit 170. . Specifically, the subtraction unit 110 calculates a prediction residual by subtracting each pixel value of the predicted image from each pixel value of the block, and outputs the calculated prediction residual to the transformation/quantization unit 120 .

The transform/quantization unit 120 performs orthogonal transform processing and quantization processing on a block-by-block basis. The transform/quantization unit 120 includes a transform unit 121 and a quantization unit 122 .

The transform unit 121 performs orthogonal transform processing on the prediction residual input from the subtraction unit 110 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122 . Orthogonal transformation refers to, for example, discrete cosine transform (DCT), discrete sine transform (DST), Karhunen Loeve transform (KLT), and the like. In the first embodiment, the transform unit 121 performs orthogonal transform processing using KLT.

The quantization unit 122 quantizes the transform coefficients input from the transform unit 121 using the quantization parameter (Qp) and the quantization matrix, and the quantized transform coefficients are subjected to the entropy coding unit 130 and inverse quantization/inverse transform. Output to unit 140 . Note that the quantization parameter (Qp) is a parameter commonly applied to each transform coefficient in a block, and is a parameter that determines coarseness of quantization. A quantization matrix is a matrix having, as elements, quantization values for quantizing each transform coefficient.

The entropy coding unit 130 performs entropy coding on the transform coefficients input from the quantization unit 122, performs data compression to generate coded data (bitstream), and transmits the coded data to the image coding device. Output to the outside of 1. For entropy coding, Huffman code and CABAC (Context-based Adaptive Binary
Arithmetic Coding (context adaptive binary arithmetic code) or the like can be used. The entropy coding unit 130 also receives control information about prediction from the prediction unit 170 and performs entropy coding on the input control information.

The inverse quantization/inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization/inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142 .

The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 . Specifically, the inverse quantization unit 141 restores the transform coefficients by inversely quantizing the transform coefficients input from the quantization unit 122 using the quantization parameter (Qp) and the quantization matrix, and restores the transform coefficients. The resulting transform coefficients are output to the inverse transform unit 142 .

The inverse transform unit 142 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 . For example, when the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 restores the prediction residual by performing inverse orthogonal transform processing on the transform coefficients input from the inverse quantization unit 141, and supplies the restored prediction residual, which is the restored prediction residual, to the synthesis unit 150. Output.

The synthesizing unit 150 synthesizes the restored prediction residual input from the inverse transform unit 142 with the predicted image input from the predicting unit 170 on a pixel-by-pixel basis. The synthesizing unit 150 adds each pixel value of the restored prediction residual and each pixel value of the predicted image to reconstruct (decode) the block, and outputs the decoded block-by-block decoded image to the memory 160 . Such decoded images are sometimes referred to as reconstructed images.

The memory 160 stores the decoded image input from the synthesizing section 150 . The memory 160 stores decoded images in units of frames. The memory 160 outputs the stored decoded image to the prediction section 170 . Note that a loop filter may be provided between the synthesizing section 150 and the memory 160 .

The prediction unit 170 performs prediction on a block-by-block basis. The prediction unit 170 includes an intra prediction unit 171 , an inter prediction unit 172 and a switching unit 173 .

The intra prediction unit 171 generates an intra prediction image by referring to the decoded pixel values around the block to be predicted among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. do. Also, the intra prediction unit 171 selects an optimal intra prediction mode to be applied to the target block from among a plurality of intra prediction modes, and performs intra prediction using the selected intra prediction mode. The intra prediction unit 171 outputs control information regarding the selected intra prediction mode to the entropy coding unit 130 . Note that intra prediction modes include planar prediction, DC prediction, and directional prediction.

The inter prediction unit 172 uses the decoded image stored in the memory 160 as a reference image, calculates a motion vector by a method such as block matching, predicts a block to be predicted, and generates an inter prediction image. The inter prediction image is output to switching section 173 . The inter prediction unit 172 selects an optimum inter prediction method from among inter prediction using a plurality of reference images (typically bi-prediction) and inter prediction using one reference image (unidirectional prediction), Perform inter prediction using the selected inter prediction method. The inter prediction unit 172 outputs control information related to inter prediction (inter prediction method, motion vector information, etc.) to the entropy encoding unit 130 . When using a plurality of reference images to generate an inter prediction image, the inter prediction unit 172 outputs the plurality of reference images to the evaluation unit 180 .

Prediction performed using a plurality of reference images is typically bi-prediction in inter-prediction, but is not limited to this. The prediction unit 170 may perform intra block copy prediction using a plurality of reference images. In intra-block copying, reference pictures in the same frame as the current frame are used to predict blocks in the current frame. When performing intra-block copy prediction using a plurality of reference images, the prediction unit 170 outputs the plurality of reference images to the evaluation unit 180 .

The switching unit 173 switches between the intra prediction image input from the intra prediction unit 171 and the inter prediction image input from the inter prediction unit 172 , and outputs either prediction image to the subtraction unit 110 and the synthesis unit 150 .

The evaluation unit 180 evaluates the degree of similarity between the plurality of reference images input from the prediction unit 170 on a pixel-by-pixel basis, thereby providing map information indicating the distribution of errors in the predicted image generated using the plurality of reference images. and outputs the generated map information to the determination unit 190 . Details of the evaluation unit 180 will be described later.

Based on the map information input from the evaluation unit 180, the determination unit 190 determines an orthogonal transform to be applied to the prediction residual corresponding to the predicted image whose prediction accuracy has been evaluated by the evaluation unit 180, and applies the determined orthogonal transform. It outputs to the transformation unit 121 and the inverse transformation unit 142 . In the first embodiment, the determination unit 190 includes an adaptive transform generation unit 191 that generates a vertical orthogonal transform and a horizontal orthogonal transform based on map information. The transformation unit 121 performs orthogonal transformation processing according to the orthogonal transformation input from the determination unit 190 . The inverse transformation unit 142 performs orthogonal transformation processing according to the orthogonal transformation input from the determination unit 190 . The details of the adaptive transformation generator 191 will be described later.

(Example of inter prediction)
FIG. 2 is a diagram illustrating an example of inter prediction. FIG. 2(a) shows bi-prediction as an example of inter-prediction, and FIG. 2(b) shows an example of a predicted image generated by bi-prediction.

As shown in FIG. 2(a), bi-prediction refers to frames temporally before and after the target frame (current frame). In the example of FIG. 2(a), block prediction in the tth frame image is performed by referring to the t−1th frame and the t+1th frame. In motion detection, portions (blocks) similar to the target image block are detected from within the search range set by the system from within the t−1 and t+1 reference frames.

The detected location is the reference image. Information indicating the relative position of the reference image with respect to the target image block is indicated by an arrow in the drawing, and is called a motion vector. The motion vector information is encoded together with the frame information of the reference image by entropy encoding in the image encoding device 1 . On the other hand, the image decoding device detects a reference image based on the motion vector information generated by the image encoding device 1 .

As shown in FIGS. 2(a) and 2(b), the reference images 1 and 2 detected by motion estimation are similar partial images aligned in the reference frame with respect to the target image block. Therefore, the image is similar to the target image block (encoding target image). In the example of FIG. 2B, the target image block includes a star pattern and a partial circle pattern. Reference image 1 includes a star pattern and a general circle pattern. Reference image 2 contains a pattern of stars, but does not contain a pattern of circles.

A predicted image is generated from the reference images 1 and 2 . It should be noted that the prediction process generally generates a predicted image with the features of each reference image by averaging reference images 1 and 2 that have different but partially similar features. However, a predicted image may be generated by using more advanced processing, for example, signal enhancement processing using a low-pass filter, high-pass filter, or the like. Here, since the reference image 1 includes a circle pattern and the reference image 2 does not include a circle pattern, when the predicted image is generated by averaging the reference images 1 and 2, the circle pattern in the predicted image is the same as the reference image The signal is halved compared to 1.

The difference between the predicted image obtained from the reference images 1 and 2 and the target image block (encoding target image) is the prediction residual. In the prediction residuals shown in FIG. 2(b), a large difference occurs only in the shifted portion of the edge of the star pattern and the shifted portion of the circle pattern (hatched portion). Prediction can be performed with good accuracy, and the difference becomes small (the difference does not occur in the example of FIG. 2(b)).

The portions where no difference occurs (the non-edge portion of the star pattern and the background portion) are portions where the similarity between the reference image 1 and the reference image 2 is high, and where highly accurate prediction has been performed. be. On the other hand, a portion where a large difference occurs is a portion unique to each reference image, that is, a portion where the degree of similarity between reference image 1 and reference image 2 is extremely low. Therefore, it can be seen that a portion where the degree of similarity between reference image 1 and reference image 2 is extremely low has low prediction accuracy and causes a large difference (residual error).

When a prediction residual in which a portion with a large difference and a portion with no difference are mixed is orthogonally transformed in this way, and deterioration of the transform coefficients occurs due to quantization, the deterioration of the transform coefficients is caused by inverse quantization and inverse orthogonal transformation. Propagate globally within the image (block). Then, when the prediction residual (restored prediction residual) restored by inverse quantization and inverse orthogonal transformation is combined with the predicted image to reconstruct the target image block, the non-edge of the star pattern shown in FIG. The deterioration of the image quality propagates to parts where highly accurate prediction has been performed, such as parts and background parts.

(Evaluation Department)
FIG. 3 is a diagram showing an example of the configuration of the evaluation unit 180. As shown in FIG. As shown in FIG. 3, the evaluation unit 180 includes a difference calculation unit (subtraction unit) 180a, a normalization unit 180b, and an adjustment unit 180c.

The difference calculation unit 180a calculates the difference value (absolute value of the difference) between the reference image 1 and the reference image 2 in units of pixels, and outputs the calculated difference value to the normalization unit 180b. Such a difference value is an example of a value indicating similarity. It can be said that the smaller the difference value, the higher the similarity, and the larger the difference value, the lower the similarity. The difference calculation unit 180a may calculate the difference value after performing filter processing on each reference image. The difference calculator 180a may calculate a statistic such as a squared error and use the statistic as the degree of similarity.

The normalization unit 180b normalizes the difference value input from the difference calculation unit 180a by the maximum difference value in the block (that is, the maximum value of the difference values in the block) and outputs the result. The smaller the difference value, the higher the degree of similarity and the higher the prediction accuracy. On the other hand, the larger the difference value, the lower the similarity and the lower the prediction accuracy (the larger the prediction error).

The normalization unit 180b normalizes the difference value of each pixel input from the difference calculation unit 180a by the difference value of the pixel having the maximum difference value in the block (that is, the maximum difference value in the block), Output a normalized difference value that is a normalized difference value. Such a normalized difference value can be used as an estimated value representing the magnitude of the prediction error.

The adjustment unit 180c adjusts the normalized difference value input from the normalization unit 180b based on a quantization parameter (Qp) that determines the coarseness of quantization, and outputs the adjusted normalized difference value. Since the degree of deterioration of the restored prediction residual increases as the coarseness of quantization increases, the adjustment unit 180c adjusts the normalized difference value (weight) based on the quantization parameter (Qp).

The estimated value Rij of the prediction error at each pixel position (ij) output by the evaluation unit 180 can be expressed, for example, by the following equation (1).

Rij = (abs(Xij-Yij)/maxD x Scale(Qp)) (1)

In equation (1), Xij is the pixel value of pixel ij in reference image 1, Yij is the pixel value of pixel ij in reference image 2, and abs is a function for obtaining an absolute value. The difference calculator 180a outputs abs(Xij-Yij).

Also, in equation (1), maxD is the maximum value of the difference values abs(Xij-Yij) in the block. In order to obtain maxD, it is necessary to obtain the difference values for all pixels in the block, but in order to omit this process, the maximum value of adjacent blocks that have already been encoded may be substituted. Alternatively, maxD may be obtained from the quantization parameter (Qp) using a table that defines the correspondence between the quantization parameter (Qp) and maxD. Alternatively, a fixed value defined in advance by specifications may be used as maxD. The normalization unit 180b outputs abs(Xij-Yij)/maxD.

Also, in Equation (1), Scale(Qp) is a coefficient that is multiplied according to the quantization parameter (Qp). Scale(Qp) is designed so that it approaches 1.0 when Qp is large and approaches 0 when Qp is small, and the degree is adjusted by the system. Alternatively, a fixed value defined in advance by specifications may be used as Scale (Qp). Furthermore, in order to simplify processing, Scale (Qp) may be a fixed value such as 1.0 designed according to the system.

The adjuster 180c outputs abs(Xij−Yij)/maxD×Scale(Qp) as the estimated error value Rij. This Rij may also output weightings adjusted by a sensitivity function designed for the system. For example, abs(Xij−Yij)/maxD×Scale(Qp)=Rij and Rij=Clip(Rij, 1.0, 0.0), or Rij=Clip(Rij+offset, 1.0, 0.0 ) and an offset may be added to adjust the sensitivity. Note that Clip(x, max, min) indicates a process of clipping at max when x exceeds max and at min when x is less than min.

The estimated error value Rij for each pixel position calculated in this manner is a value within the range from 0 to 1.0. Basically, the estimated error value Rij approaches 1.0 when the difference value of the pixel position ij between the reference images is large (that is, the prediction accuracy is low), and the difference value of the pixel position ij between the reference images is It approaches 0 when it is small (ie, the prediction accuracy is high). The evaluation unit 180 outputs two-dimensional map information (hereinafter referred to as "error map") consisting of the estimated error values Rij of each pixel position ij in the block.

(adaptive conversion generator)
FIG. 4 is a diagram showing the operation of the adaptive transformation generator 191 according to the first embodiment. The adaptive transform generation unit 191 uses the error map input from the evaluation unit 180 to perform a vertical adaptive orthogonal transform applied in the vertical direction and a horizontal adaptive orthogonal transform applied in the horizontal direction to the prediction residual by principal component analysis. Generate.

Specifically, the adaptive transform generator 191 regards the error map as a set of column vectors, generates a covariance matrix, and calculates the eigenvectors of the generated covariance matrix. The adaptive transform generator 191 outputs the obtained eigenvectors as vertical adaptive orthogonal transform. Also, the adaptive transformation generator 1
91 regards the matrix obtained by applying the generated vertical adaptive orthogonal transform in the vertical direction as a set of row vectors, generates a covariance matrix, and calculates its eigenvector. The adaptive transform generator 191 outputs the obtained eigenvectors as horizontal adaptive orthogonal transform.

As shown in FIG. 4(a), when the error map has width w and height h, the adaptive transform generation unit 191 treats the error map as w h×1 column vectors, and calculates the covariance matrix Λ _h Calculate The adaptive transform generator 191 diagonalizes the obtained covariance matrix to calculate eigenvectors. Here, the diagonalization of the covariance matrix is calculated by iterative calculation using, for example, the Jacobi method. The adaptive transform generator 191 outputs an h×h matrix as a vertically adaptive orthogonal transform by combining the obtained eigenvectors e ₀ to e _h transforms.

Furthermore, as shown in FIG. 4(b), the adaptive transform generator 191 regards the error map as h 1×w row vectors and calculates a covariance matrix Λ _h . The adaptive transform generator 191 diagonalizes the obtained covariance matrix to calculate eigenvectors. The adaptive transform generator 191 outputs a w×w matrix as a horizontal adaptive orthogonal transform by combining the obtained eigenvectors.

In video coding methods such as HEVC (see Non-Patent Document 1) and the latest video coding technology (JEM) under study by an international standardization organization, integer-precision transform coefficients are used for the purpose of fast and lightweight transform processing. and bit shift. In the adaptive transform generation unit 191 according to the present embodiment as well, the obtained eigenvector is approximated to an integer coefficient, and the bit shift amount that does not expand the dynamic range of the transform coefficient is defined in advance by the image coding device and the image decoding device. may

Note that, as shown in FIG. 1 , the transform unit 121 uses the vertical adaptive orthogonal transform and the horizontal adaptive orthogonal transform input from the adaptive transform generating unit 191 to apply vertical and horizontal transforms to the prediction residual generated by the subtracting unit 110 . Transform coefficients are calculated by performing orthogonal transform processing in the horizontal direction, and the calculated transform coefficients are output to the quantization unit 122 .

In addition, the inverse transform unit 142 uses the vertical adaptive orthogonal transform and the horizontal adaptive orthogonal transform input from the adaptive transform generating unit 191 to transform the transform coefficients input from the inverse quantizing unit 141 into the transform unit 121. Inverse orthogonal transform processing corresponding to orthogonal transform processing is performed.

(Image decoding device)
FIG. 5 is a diagram showing the configuration of the image decoding device 2 according to the first embodiment. As shown in FIG. 5, the image decoding device 2 includes an entropy code decoding unit 200, an inverse quantization/inverse transform unit 210, a synthesis unit 220, a memory 230, a prediction unit 240, an evaluation unit 250, a determination and a section 260 .

The entropy code decoding unit 200 decodes the encoded data generated by the image encoding device 1 and outputs the quantized transform coefficients to the inverse quantization/inverse transform unit 210 . The entropy code decoding unit 200 also acquires control information related to prediction (intra prediction and inter prediction) and outputs the acquired control information to the prediction unit 240 .

The inverse quantization/inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization/inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212 .

The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1 . The inverse quantization unit 211 restores and restores the transform coefficients by inversely quantizing the quantized transform coefficients input from the entropy code decoding unit 200 using the quantization parameter (Qp) and the quantization matrix. The transform coefficients are output to the inverse transform unit 212 .

The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1 . The inverse transform unit 212 performs inverse orthogonal transform processing on the transform coefficients input from the inverse quantization unit 211 to restore the prediction residuals, and sends the restored prediction residuals (restored prediction residuals) to the synthesizing unit 220. Output.

The synthesizing unit 220 reconstructs (decodes) the original block by synthesizing the prediction residual input from the inverse transform unit 212 and the predicted image input from the prediction unit 240 on a pixel-by-pixel basis. is output to the memory 230.

The memory 230 stores the decoded image input from the synthesizing section 220 . The memory 230 stores decoded images in units of frames. The memory 230 outputs the decoded image in units of frames to the outside of the image decoding device 2 . Note that a loop filter may be provided between the synthesizing section 220 and the memory 230 .

The prediction unit 240 performs prediction on a block-by-block basis. The prediction unit 240 includes an intra prediction unit 241 , an inter prediction unit 242 and a switching unit 243 .

The intra prediction unit 241 refers to the decoded image stored in the memory 230, performs intra prediction according to the control information input from the entropy code decoding unit 200, generates an intra prediction image, and switches the generated intra prediction image. Output to the unit 243 .

The inter prediction unit 242 performs inter prediction for predicting a prediction target block using the decoded image stored in the memory 230 as a reference image. The inter prediction unit 242 performs inter prediction according to the control information (inter prediction method, motion vector information, etc.) input from the entropy code decoding unit 200 to generate an inter prediction image, and switches the generated inter prediction image to the switching unit 243. output to When using a plurality of reference images to generate an inter prediction image, the inter prediction unit 242 outputs the plurality of reference images to the evaluation unit 250 .

Prediction performed using a plurality of reference images is typically bi-prediction in inter-prediction, but is not limited to this. The prediction unit 240 may perform intra block copy prediction using a plurality of reference images. When performing intra-block copy prediction using a plurality of reference images, the prediction unit 240 outputs the plurality of reference images to the evaluation unit 250 .

The switching unit 243 switches between the intra-predicted image input from the intra-prediction unit 241 and the inter-predicted image input from the inter-prediction unit 242 and outputs one of the predicted images to the synthesis unit 220 .

The evaluation section 250 operates in the same manner as the evaluation section 180 (see FIG. 3) of the image encoding device 1 . The evaluation unit 250 evaluates the degree of similarity between the plurality of reference images input from the prediction unit 240 on a pixel-by-pixel basis, thereby creating an error map showing the distribution of errors in the predicted image generated using the plurality of reference images. and outputs the generated error map to the determination unit 260 .

Based on the error map input from the evaluation unit 250, the determination unit 260 determines an inverse orthogonal transform to be applied to the prediction residual corresponding to the predicted image whose prediction accuracy has been evaluated by the evaluation unit 250, and determines the determined inverse orthogonal transform. The transform is output to the inverse transform unit 212 . The determination unit 260 includes an adaptive transform generation unit 261 that generates a vertical orthogonal transform and a horizontal orthogonal transform based on the error map. The adaptive transform generator 261 performs the same operation as the adaptive transform generator 191 of the image coding device 1 (see FIG. 4). The inverse transform section 212 performs inverse orthogonal transform processing according to the inverse orthogonal transform input from the determining section 260 .

(Summary of the first embodiment)
The image encoding apparatus 1 according to the first embodiment encodes a target image in units of blocks obtained by dividing a current image in units of frames. The image coding apparatus 1 predicts a target image using a plurality of reference images to generate a predicted image, and evaluates the similarity between the plurality of reference images on a pixel-by-pixel basis to generate a predicted image. An evaluation unit 180 that generates an error map that shows the distribution of errors in , a subtraction unit 110 that calculates a prediction residual that indicates the difference between the target block and the predicted image on a pixel-by-pixel basis, and an orthogonal transform that is applied to the prediction residual. It includes a determining unit 190 that determines based on the map, and a transforming unit 121 that performs orthogonal transform processing on the prediction residual using the determined orthogonal transform. The determination unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform based on map information.

Further, the image decoding device 2 according to the first embodiment decodes the target image in units of blocks obtained by dividing the current image in units of frames. The image decoding device 2 includes an entropy code decoding unit 200 that acquires transform coefficients by decoding encoded data, a prediction unit 240 that predicts a target image using a plurality of reference images and generates a predicted image, and An evaluation unit 250 that generates an error map showing the distribution of errors in a predicted image by evaluating similarities between a plurality of reference images on a pixel-by-pixel basis, and determining an inverse orthogonal transform to be applied to transform coefficients based on the error map. and an inverse transform unit 212 that performs inverse orthogonal transform processing on transform coefficients by the determined inverse orthogonal transform. The determination unit 260 includes an adaptive transform generation unit 261 that generates an inverse orthogonal transform based on map information.

Thus, according to the first embodiment, by generating an orthogonal transform based on an error map showing the distribution of errors in a predicted image, it is possible to apply the optimum orthogonal transform according to the energy distribution in the prediction residual. can be done.

Further, since each of the image encoding device 1 and the image decoding device 2 can generate an orthogonal transform based on the error map, the information for inverse processing of the orthogonal transform (KLT) performed by the image encoding device 1 is It is not required on the image decoding device 2 side. Therefore, an increase in the amount of information to be transmitted can be suppressed.

Therefore, according to the image coding device 1 and the image decoding device 2 according to the first embodiment, it is possible to apply the adaptive orthogonal transform that efficiently concentrates the energy of the prediction residual, thereby improving the coding efficiency.

<Second embodiment>
The image encoding device 1 and the image decoding device 2 according to the second embodiment will be mainly described with respect to differences from the first embodiment.

(Image encoding device)
FIG. 6 is a diagram showing the configuration of the image encoding device 1 according to the second embodiment. As shown in FIG. 6, the image coding apparatus 1 according to the second embodiment differs from the first embodiment in the configuration of the determination unit 190. FIG. The determination unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform by principal component analysis (see FIG. 4) of the error map, as in the first embodiment. In the second embodiment, the determination unit 190 further includes a candidate selection unit (first selection unit) 192 and an orthogonal transform selection unit (second selection unit) 193 .

The adaptive transform generating section 191 outputs the adaptive orthogonal transform (vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform) generated based on the error map to the candidate selecting section 192 .

Candidate selection section 192 selects one or more orthogonal transform candidates from among a plurality of types of predetermined orthogonal transforms in descending order of correlation with adaptive orthogonal transform input from adaptive transform generating section 191, and selects One or more orthogonal transform candidates are output to orthogonal transform selecting section 263 . A plurality of types of orthogonal transforms defined in advance are shared by the image encoding device 1 and the image decoding device 2 . In the second embodiment, it is assumed that DCT-2, DST-7, DCT-8, DST-1, and DCT-5 are defined in advance as multiple kinds of orthogonal transforms.

Specifically, the candidate selection unit 192 selects the horizontal adaptive orthogonal transform input from the adaptive transform generating unit 191 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5) and Evaluate the correlation of Based on the result of the correlation evaluation, candidate selection section 192 outputs one or more orthogonal transforms in descending order of correlation among the plurality of types of orthogonal transforms to orthogonal transform selection section 193 as adaptive orthogonal transform candidates.

Note that the number of adaptive orthogonal transform candidates output by the candidate selection unit 192 is defined in advance so that the number is the same between the image encoding device 1 and the image decoding device 2 . Also, the number of adaptive orthogonal transform candidates output by the candidate selection unit 192 may be variable according to the block size and color components (luminance component, color difference component) of the image block to be encoded.

Furthermore, candidate selection section 192 may select adaptive orthogonal transform candidates separately in the horizontal direction and in the vertical direction. In this case, the candidate selection unit 192 calculates the correlation between the vertical adaptive orthogonal transform input from the adaptive transform generating unit 191 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5). is further evaluated, and adaptive vertical orthogonal transform candidates are output to orthogonal transform selecting section 193 in addition to adaptive horizontal orthogonal transform candidates.

The orthogonal transform selection unit 193 selects an orthogonal transform to be applied to the prediction residual from among the one or more orthogonal transform candidates input from the candidate selection unit 192, and applies the selected orthogonal transform to the transform unit 121 and the inverse transform unit. 142. Orthogonal transform selecting section 193 also outputs an index indicating the type of the selected orthogonal transform to entropy coding section 130 .

For example, the orthogonal transform selection unit 193 calculates the coding efficiency when applying each orthogonal transform candidate by simulation, and selects the optimum orthogonal transform according to the result of the simulation. The orthogonal transform selection unit 193 may be configured to select the same type of orthogonal transform in the horizontal direction and the vertical direction, or may be configured to select different types of orthogonal transform in the horizontal direction and the vertical direction. good too.

The entropy coding section 130 entropy-codes the adaptive orthogonal transform index input from the orthogonal transform selecting section 193 . When selecting different types of orthogonal transforms in the horizontal and vertical directions, the entropy coding unit 130 entropy-codes different adaptive orthogonal transform indices in the horizontal and vertical directions. However, if the adaptive orthogonal transform candidate is composed of one type of orthogonal transform, the orthogonal transform can be uniquely specified in the image decoding device 2, so that the index does not need to be entropy coded.

(Image decoding device)
FIG. 7 is a diagram showing the configuration of the image decoding device 2 according to the second embodiment. As shown in FIG. 7, the image decoding device 2 according to the second embodiment differs from the first embodiment in the configuration of the determination unit 260. FIG. The determination unit 260 includes an adaptive transform generation unit 261 that generates an orthogonal transform by principal component analysis (see FIG. 4) of the error map, as in the first embodiment. In the second embodiment, the determination unit 260 further includes a candidate selection unit (first selection unit) 262 and an orthogonal transform selection unit (second selection unit) 263 .

The adaptive transform generating section 261 outputs the adaptive orthogonal transform (vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform) generated based on the error map to the candidate selecting section 262 .

Candidate selection section 262 selects one or more orthogonal transform candidates from among a plurality of types of predetermined orthogonal transforms in descending order of correlation with adaptive orthogonal transform input from adaptive transform generating section 261, and selects One or more orthogonal transform candidates are output to orthogonal transform selecting section 263 . As described above, it is assumed that DCT-2, DST-7, DCT-8, DST-1, and DCT-5 are defined in advance as multiple kinds of orthogonal transforms.

Specifically, the candidate selection unit 262 selects the horizontal adaptive orthogonal transform input from the adaptive transform generating unit 261 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5) and Evaluate the correlation of Based on the result of the correlation evaluation, candidate selection section 262 outputs one or more orthogonal transforms in descending order of correlation among the plurality of types of orthogonal transforms to orthogonal transform selection section 263 as adaptive orthogonal transform candidates.

As described above, the number of adaptive orthogonal transform candidates output by the candidate selection unit 262 is defined in advance such that the number is the same between the image encoding device 1 and the image decoding device 2 . Also, the number of adaptive orthogonal transform candidates output by the candidate selection unit 262 may be variable according to the block size and color components (luminance component, color difference component) of the image block to be decoded.

Further, the candidate selection section 262 may select adaptive orthogonal transform candidates separately in the horizontal direction and the vertical direction. In this case, the candidate selection unit 262 calculates the correlation between the vertical adaptive orthogonal transform input from the adaptive transform generating unit 261 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5). is further evaluated, and adaptive vertical orthogonal transform candidates are output to orthogonal transform selecting section 263 in addition to adaptive horizontal orthogonal transform candidates.

On the other hand, the entropy code decoding unit 200 decodes an index indicating an orthogonal transform selected by the image encoding device 1 from among one or more orthogonal transform candidates, and outputs the index to the orthogonal transform selecting unit 263 . When selecting different types of orthogonal transforms in the horizontal and vertical directions, the entropy code decoding unit 200 decodes different adaptive orthogonal transform indices in the horizontal and vertical directions.

Orthogonal transform selecting section 263 selects an inverse orthogonal transform to be applied to transform coefficients from among one or more orthogonal transform candidates input from candidate selecting section 262 based on the index input from entropy code decoding section 200. , outputs the selected inverse orthogonal transform to the inverse transform unit 212 . The orthogonal transform selection unit 263 may be configured to select the same type of orthogonal transform in the horizontal direction and the vertical direction, or may be configured to select different types of orthogonal transform in the horizontal direction and the vertical direction. good too.

(Summary of Second Embodiment)
In the image encoding device 1 according to the second embodiment, the determination unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform by principal component analysis of an error map, A candidate selection unit 192 that selects one or more orthogonal transform candidates in descending order of correlation with the generated orthogonal transform, and an orthogonal transform that selects an orthogonal transform to be applied to the prediction residual from among the one or more orthogonal transform candidates. and a selection unit 193 . The entropy encoding unit 130 encodes an index indicating an orthogonal transform selected by the orthogonal transform selecting unit 193 from one or more orthogonal transform candidates.

Further, in the image decoding device 2 according to the second embodiment, the determination unit 260 includes an adaptive transform generation unit 261 that generates an orthogonal transform by principal component analysis of the error map, and a plurality of types of predetermined orthogonal transforms , a candidate selection unit 262 that selects one or more orthogonal transform candidates in descending order of correlation with the generated orthogonal transform; and a conversion selection unit 263 . The entropy code decoding unit 200 decodes an index indicating an orthogonal transform selected by the image encoding device 1 from among one or more orthogonal transform candidates. The orthogonal transform selection unit 263 selects an inverse orthogonal transform to be applied to transform coefficients from one or more orthogonal transform candidates based on the index.

As described above, according to the second embodiment, an orthogonal transform candidate having a high correlation with the orthogonal transform (KLT) generated by principal component analysis of the error map is selected from among a plurality of types of orthogonal transform specified in advance. , an orthogonal transform to be applied to the transform process is selected from the orthogonal transform candidates. A plurality of types of predetermined orthogonal transforms (DCT-2, DST-7, DCT-8, DST-1, DCT-5) require less computational processing than KLT.

Therefore, according to the second embodiment, the orthogonal transform that efficiently concentrates the energy of the prediction residuals can be applied to improve the coding efficiency, while reducing the computational complexity of the transform process compared to the first embodiment. can be reduced.

<Third Embodiment>
The image encoding device 1 and the image decoding device 2 according to the third embodiment will be mainly described with respect to differences from the first and second embodiments.

In the above-described second embodiment, an orthogonal transform candidate having a high correlation with the orthogonal transform (KLT) generated by the principal component analysis of the error map is selected from among a plurality of types of orthogonal transform specified in advance. On the other hand, in the third embodiment, orthogonal transformation candidates are selected from a plurality of types of orthogonal transformations defined in advance by evaluating the feature amount of the error map.

(Image encoding device)
FIG. 8 is a diagram showing the configuration of the image encoding device 1 according to the third embodiment. As shown in FIG. 8, the image coding apparatus 1 according to the third embodiment differs from that of the second embodiment in that the determination unit 190 includes a feature amount evaluation unit 191a. The feature quantity evaluation unit 191 a evaluates the feature quantity of the error map input from the evaluation unit 180 and outputs the evaluation result to the candidate selection unit 192 .

FIG. 9 is a diagram showing the operation of the feature quantity evaluation unit 191a. As shown in FIG. 9, in order to evaluate the energy distribution of the error map, the feature quantity evaluation unit 191a divides the error map into 4 parts in the horizontal direction and 4 parts in the vertical direction. A total value Exy (E ₀₀ to E ₃₃ ) of the values is calculated. The feature quantity evaluation unit 191 a outputs the evaluated energy distributions E ₀₀ to E ₃₃ to the candidate selection unit 192 .

Candidate selection section 192 selects adaptive orthogonal transform candidates from a plurality of types of predefined orthogonal transforms (DCT-2, DST-7, DCT-8, DST-1, DCT-5) based on the following conditions: is selected, and the selected adaptive orthogonal transform candidate is output to orthogonal transform selecting section 193 .

(a) The candidate selection unit 192 performs horizontal direction:

select DCT-2 and DST-1 as horizontal adaptive orthogonal transform candidates when

select DCT-2 and DST-7 as horizontal adaptive orthogonal transform candidates when

select DCT-2 and DCT-5 as horizontal adaptive orthogonal transform candidates when
When neither is true, DCT-8 and DST-7 are selected as horizontal adaptive orthogonal transform candidates.

(b) The candidate selection unit 192 performs the vertical direction:

select DCT-2 and DST-1 as vertical adaptive orthogonal transform candidates when

select DCT-2 and DST-7 as vertical adaptive orthogonal transform candidates when

select DCT-2 and DCT-5 as vertical adaptive orthogonal transform candidates when
When neither is true, DCT-8 and DST-7 are selected as vertical adaptive orthogonal transform candidates.

Here, the number of adaptive orthogonal transform candidates output by candidate selecting section 192 is two in each of the horizontal direction and the vertical direction. may be variable according to the block size of the image block, color components (luminance component, color difference component), and the like.

As in the second embodiment, the orthogonal transform selection unit 193 selects an orthogonal transform to be applied to the prediction residual from among the orthogonal transform candidates input from the candidate selection unit 192, and applies the selected orthogonal transform to the transform unit 121. and output to the inverse transformation unit 142 . Orthogonal transform selecting section 193 also outputs an index indicating the type of the selected orthogonal transform to entropy coding section 130 . The entropy coding section 130 entropy-codes the adaptive orthogonal transform index input from the orthogonal transform selecting section 193 .

(Image decoding device)
FIG. 10 is a diagram showing the configuration of an image decoding device 2 according to the third embodiment. As shown in FIG. 10, the image decoding device 2 according to the third embodiment differs from that of the second embodiment in that the determination unit 260 includes a feature amount evaluation unit 261a. The feature amount evaluation unit 261a performs the same operation as the feature amount evaluation unit 191a of the image encoding device 1 (see FIG. 9).

In order to evaluate the energy distribution of the error map, the feature quantity evaluation unit 261a divides the error map into 4 parts in the horizontal direction and 4 parts in the vertical direction. ₀₀ to E ₃₃ ). The feature quantity evaluation unit 261 a outputs the evaluated energy distributions E ₀₀ to E ₃₃ to the candidate selection unit 262 .

The candidate selection unit 262 selects a plurality of types of predefined orthogonal transforms (DCT-2, DST-7, DCT-8, DST-1, It selects an adaptive orthogonal transform candidate from DCT-5) and outputs the selected adaptive orthogonal transform candidate to orthogonal transform selecting section 263 .

On the other hand, the entropy code decoding unit 200 decodes the index indicating the orthogonal transform selected by the image encoding device 1 from among the orthogonal transform candidates, and outputs the index to the orthogonal transform selecting unit 263 . Orthogonal transform selecting section 263 selects an inverse orthogonal transform to be applied to transform coefficients from the orthogonal transform candidates input from candidate selecting section 262 based on the index input from entropy code decoding section 200, and selects the inverse orthogonal transform to be applied to the transform coefficient. The orthogonal transform is output to the inverse transform unit 212 .

(Summary of the third embodiment)
In the image encoding device 1 according to the third embodiment, the determination unit 190 includes a feature amount evaluation unit 191a that evaluates the feature amount of the error map, and a plurality of types of predetermined orthogonal vectors based on the evaluated feature amount. It includes a candidate selection unit 192 that selects one or more orthogonal transform candidates from among the transforms, and an orthogonal transform selection unit 193 that selects an orthogonal transform to be applied to the prediction residual from among the one or more orthogonal transform candidates. The entropy encoding unit 130 encodes an index indicating an orthogonal transform selected by the orthogonal transform selecting unit 193 from one or more orthogonal transform candidates.

Further, in the image decoding device 2 according to the third embodiment, the determination unit 260 includes a feature amount evaluation unit 261a that evaluates the feature amount of the error map, and a plurality of types of predetermined types based on the evaluated feature amount. A candidate selection unit 262 that selects one or more orthogonal transform candidates from among the orthogonal transform candidates, and an orthogonal transform selection unit 263 that selects an inverse orthogonal transform to be applied to transform coefficients from among the one or more orthogonal transform candidates. . The entropy code decoding unit 200 decodes an index indicating an orthogonal transform selected by the image encoding device 1 from among one or more orthogonal transform candidates. The orthogonal transform selection unit 263 selects an inverse orthogonal transform to be applied to transform coefficients from one or more orthogonal transform candidates based on the index.

As described above, according to the third embodiment, since the feature quantity of the error map can be evaluated by simple arithmetic processing, the computation Processing volume can be reduced.

Therefore, according to the third embodiment, the orthogonal transform that efficiently concentrates the energy of the prediction residual can be applied to improve the coding efficiency, and the error map analysis is performed more efficiently than the second embodiment. Arithmetic processing can be reduced.

<Other embodiments>
In the first to third embodiments described above, an example of performing transform processing separately in the vertical direction and in the vertical direction using one-dimensional orthogonal transform has been described. However, instead of the one-dimensional orthogonal transform, two-dimensional orthogonal transform may be used to collectively perform transform processing in the vertical direction and the vertical direction.

Alternatively, the program may be provided by a program that causes a computer to execute each process performed by the image encoding device 1 and a program that causes a computer to execute each process performed by the image decoding device 2 . Also, the program may be recorded on a computer-readable medium. A computer readable medium allows the installation of the program on the computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as CD-ROM or DVD-ROM.

Alternatively, the image encoding device 1 may be configured as a semiconductor integrated circuit (chipset, SoC) by integrating a circuit that executes each process performed by the image encoding device 1 . Similarly, the image decoding device 2 may be configured as a semiconductor integrated circuit (chipset, SoC) by integrating a circuit that executes each process performed by the image decoding device 2 .

Although the embodiments have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the invention.

1: image coding device 2: image decoding device 100: block division unit 110: subtraction unit 120: transformation/quantization unit 121: transformation unit 122: quantization unit 130: entropy coding unit 140: inverse quantization/inverse transformation Unit 141 : Inverse quantization unit 142 : Inverse transform unit 150 : Synthesis unit 160 : Memory 170 : Prediction unit 171 : Intra prediction unit 172 : Inter prediction unit 173 : Switching unit 180 : Evaluation unit 180a : Difference calculation unit 180b : Normalization Unit 180c: Adjustment unit 190: Determination unit 191: Adaptive transformation generation unit 191a: Feature amount evaluation unit 192: Candidate selection unit 193: Orthogonal transformation selection unit 200: Entropy code decoding unit 210: Inverse quantization/inverse transformation unit 211: Inverse Quantization unit 212 : Inverse transform unit 220 : Synthesis unit 230 : Memory 240 : Prediction unit 241 : Intra prediction unit 242 : Inter prediction unit 243 : Switching unit 250 : Evaluation unit 260 : Determination unit 261 : Adaptive transformation generation unit 261a : Features Quantity evaluation unit 262: Candidate selection unit 263: Orthogonal transformation selection unit

Claims (12)

An image encoding device for encoding a target image in block units obtained by dividing a current image in frame units,
a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image;
an evaluation unit that generates map information indicating the distribution of errors in the predicted image by evaluating the degree of similarity between the plurality of reference images on a pixel-by-pixel basis;
a subtraction unit that calculates a prediction residual indicating a difference between the target image and the predicted image on a pixel-by-pixel basis;
a determination unit that determines an orthogonal transform to be applied to the prediction residual based on the map information;
and a transform unit that performs orthogonal transform processing on the prediction residual using the determined orthogonal transform. The decision unit
an adaptive transform generator that generates an orthogonal transform by principal component analysis of the map information;
a first selection unit that selects one or more orthogonal transform candidates in descending order of correlation with the generated orthogonal transform from among a plurality of types of predetermined orthogonal transforms;
2. The image coding apparatus according to claim 1, further comprising a second selection unit that selects an orthogonal transform to be applied to the prediction residual from among the one or more orthogonal transform candidates. The decision unit
a feature quantity evaluation unit that evaluates a feature quantity in the map information;
a first selection unit that selects one or more orthogonal transform candidates from a plurality of types of predetermined orthogonal transforms based on the evaluated feature amount;
2. The image coding apparatus according to claim 1, further comprising a second selection unit that selects an orthogonal transform to be applied to the prediction residual from among the one or more orthogonal transform candidates. 4. The image encoding according to claim 2, further comprising an encoding unit that encodes an index indicating the orthogonal transform selected by the second selection unit from among the one or more orthogonal transform candidates. Device. An image encoding device for encoding a target image in block units obtained by dividing a current image in frame units,
a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image;
an evaluation unit that calculates the sum of absolute differences indicating the degree of similarity between the plurality of reference images in units of regions smaller than the blocks and made up of a plurality of pixels;
An image encoding apparatus, wherein the evaluation unit controls encoding processing based on the sum of absolute differences calculated for each area. An image decoding device for decoding a target image in units of blocks obtained by dividing a current image in units of frames,
a decoding unit that obtains transform coefficients by decoding encoded data;
a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image;
an evaluation unit that generates map information indicating the distribution of errors in the predicted image by evaluating the degree of similarity between the plurality of reference images on a pixel-by-pixel basis;
a determination unit that determines an inverse orthogonal transform to be applied to the transform coefficients based on the map information;
an inverse transform unit that performs inverse orthogonal transform processing on the transform coefficients by the determined inverse orthogonal transform. The decision unit
an adaptive transform generator that generates an orthogonal transform by principal component analysis of the map information;
a first selection unit that selects one or more orthogonal transform candidates in descending order of correlation with the generated orthogonal transform from among a plurality of types of predetermined orthogonal transforms;
7. The image decoding device according to claim 6, further comprising a second selection unit that selects an inverse orthogonal transform to be applied to the transform coefficient from among the one or more orthogonal transform candidates. The decision unit
a feature quantity evaluation unit that evaluates a feature quantity in the map information;
a first selection unit that selects one or more orthogonal transform candidates from a plurality of types of predetermined orthogonal transforms based on the evaluated feature amount;
7. The image decoding device according to claim 6, further comprising a second selection unit that selects an inverse orthogonal transform to be applied to the transform coefficient from among the one or more orthogonal transform candidates. The decoding unit further decodes an index indicating an orthogonal transform selected by the image coding device from among the one or more orthogonal transform candidates,
9. The image according to claim 7, wherein the second selection unit selects an inverse orthogonal transform to be applied to the transform coefficient from the one or more orthogonal transform candidates based on the index. decryption device. An image decoding device for decoding a target image in units of blocks obtained by dividing a current image in units of frames,
a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image;
an evaluation unit that calculates the sum of absolute differences indicating the degree of similarity between the plurality of reference images in units of regions smaller than the blocks and made up of a plurality of pixels;
An image decoding apparatus, wherein the evaluation unit controls decoding processing based on the sum of absolute differences calculated for each region. A program that causes a computer to function as the image encoding device according to any one of claims 1 to 5. A program that causes a computer to function as the image decoding device according to any one of claims 6 to 10.

Images