TW201842781A - Encoding device, decoding device, encoding method, and decoding method - Google Patents

Abstract

An encoding device is equipped with a processor and memory. The processor uses the memory to determine asymmetrical filter characteristics on opposite sides of a block boundary on the basis of the position of a target pixel within the block and to perform deblocking filter processing having the determined filter characteristics on the target pixel. For example, in the determination of filter characteristics, the filter characteristics may be determined such that the effect of the filter processing on a pixel is larger the farther the pixel is from the reference pixel for intra prediction.

Description

Encoding device, decoding device, encoding method and decoding method

FIELD OF THE INVENTION The present disclosure relates to an encoding apparatus, a decoding apparatus, an encoding method, and a decoding method.

BACKGROUND OF THE INVENTION Image coding standard specifications called HEVC (High Efficiency Video Coding) have been standardized by JCT-VC (Joint Collaborative Team on Video Coding).

Advance Technical Literature Non-Patent Literature Non-Patent Document 1: H.265 (ISO/IEC 23008-2 HEVC (High Efficiency Video Coding))

SUMMARY OF THE INVENTION Problems to be Solved by the Invention In such encoding and decoding techniques, further improvements are sought.

Accordingly, it is an object of the present disclosure to provide an encoding apparatus, a decoding apparatus, an encoding method, or a decoding method that can achieve further improvement. Means for solving problems

An encoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine an asymmetric filter characteristic according to a position in a block of the target pixel, and The deblocking filtering process having the determined filter characteristics is performed on the target pixel.

The decoding device of the present disclosure includes a processor and a memory, and the processor uses the memory to determine an asymmetric filter characteristic according to a position in a block of the target pixel, and The deblocking filtering process having the determined filter characteristics is performed on the target pixel.

In addition, the generality or the specific aspect can be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM recording medium, or a system, a method, or a product. Any combination of body circuits, computer programs, and recording media is implemented. Effect of the invention

The present disclosure can provide an encoding apparatus, a decoding apparatus, an encoding method, or a decoding method that can achieve further improvement.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the deblocking filtering process of H.265/HEVC as an image coding method, a filter having a characteristic that the boundary of the block block is symmetrical is applied. Therefore, there is a case where, for example, when the error of the pixel located at one of the boundary of the clip block is small, and the error of the pixel located at the other side of the clip block is large, the error distribution is discontinuous because symmetrically Filtering is performed to reduce the efficiency of error reduction. Here, the error refers to the difference between the pixel values of the original image and the reconstructed image.

An encoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter characteristic in which a boundary of a clip block is asymmetric, and performs a solution having the determined filter characteristic. Block filtering processing.

Accordingly, the encoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical boundary of the clip block.

For example, in the determination of the filter characteristics, the pixel having a higher probability of being larger than the original image may be determined so that the influence of the deblocking filtering process becomes larger. Asymmetric filter characteristics.

Accordingly, the encoding apparatus can increase the influence of the filtering process on the pixel having a large error, and therefore there is a possibility that the error of the pixel can be more reduced. Further, since the encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the filter coefficients of the reference filter may be changed to be asymmetric with respect to the boundary of the block, thereby determining the asymmetric filter characteristics.

For example, in the determination of the filter characteristics, it may be determined that the weight of the block boundary is asymmetric, and in the above-described deblocking filtering process, a filtering operation using the filter coefficient is performed, by which the determined The asymmetry weights are used to weight the amount of change in pixel values before and after the filtering operation.

For example, in the determination of the filter characteristics, it may be determined that an offset value (offset) that is asymmetric with respect to the block boundary is determined, and in the above-described deblocking filter process, a filter operation using a filter coefficient is performed. The pixel value after the aforementioned filtering operation is added to the previously determined asymmetric offset value.

For example, in the determination of the filter characteristics, the reference value of the block boundary may be determined to be asymmetric, and in the deblocking filter process, a filter operation using the filter coefficient is performed, and the filtering is performed. When the amount of change in the pixel value before and after the calculation exceeds the reference value, the amount of change is cut into the reference value.

For example, in the determination of the filter characteristics, a condition for determining whether or not to perform the above-described deblocking filtering process may be set asymmetrically with respect to the block boundary.

A decoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter characteristic in which a boundary of a clip block is asymmetric, and performs a solution having the determined filter characteristic. Block filtering processing.

Accordingly, the decoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary.

For example, in the determination of the filter characteristics, the pixel having a higher probability of being larger than the original image may be determined so that the influence of the deblocking filtering process becomes larger. Asymmetric filter characteristics.

Accordingly, the decoding apparatus can increase the influence of the filtering process on the pixel having a large error, and therefore there is a possibility that the error of the pixel can be more reduced. Moreover, the decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the asymmetric filter characteristics may be determined such that the filter coefficients of the reference filter are changed so that the boundary of the block is asymmetric.

For example, in the determination of the filter characteristics, it may be determined that the weight of the block boundary is asymmetric, and in the above-described deblocking filtering process, a filtering operation using the filter coefficient is performed, by which the determined The asymmetry weights are used to weight the amount of change in pixel values before and after the filtering operation.

For example, in the determination of the filter characteristics, the offset value of the block boundary may be determined to be asymmetric, and in the deblocking filter process, a filter operation using the filter coefficient is performed, and the filtering is performed. The pixel value after the operation is added to the previously determined asymmetric offset value.

For example, in the determination of the filter characteristics, the reference value of the block boundary may be determined to be asymmetric, and in the deblocking filter process, a filter operation using the filter coefficient is performed, and the filtering is performed. When the amount of change in the pixel value before and after the calculation exceeds the aforementioned reference value, the amount of change is cut into the reference value.

For example, in the determination of the filter characteristics, a condition for determining whether or not to perform the above-described deblocking filtering process may be set asymmetrically with respect to the block boundary.

The coding method of one aspect of the present disclosure performs the following steps: determining the filter characteristics of the clip block boundary to be asymmetric, and performing deblocking filter processing with the determined filter characteristics.

Accordingly, the encoding method performs filtering processing, thereby reducing the possibility of error, and the filtering processing is a filter characteristic having an asymmetric boundary of the clip block.

The decoding method of one aspect of the present disclosure performs the following steps: determining the filter characteristics of the clip block boundary to be asymmetric, and performing deblocking filter processing with the determined filter characteristics.

Accordingly, the decoding method performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary.

An encoding device according to an aspect of the present disclosure includes a processor and a memory. The processor uses the memory to determine a filter characteristic that is asymmetric with respect to a boundary of the block according to a pixel value of a boundary of the block block. The deblocking filtering process of the aforementioned filter characteristics has been decided.

Accordingly, the encoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical boundary of the clip block. Further, the encoding device can determine an appropriate filter characteristic based on the pixel value of the boundary of the clip block.

For example, in the determination of the filter characteristics, the filter characteristics may be determined based on the difference in pixel values.

For example, in the determination of the filter characteristics, the difference in the pixel values may be larger, and the difference in the filter characteristics sandwiching the block boundaries may be further amplified.

According to this, for example, there is a possibility that it is possible to suppress unnecessary smoothing when the block boundary coincides with the edge of the object in the image.

For example, in the determination of the filter characteristics, the difference between the pixel values and the threshold based on the quantization parameter may be compared. When the difference between the pixel values is greater than the threshold, the difference from the pixel value is less than the threshold. At the same time, the difference of the aforementioned filter characteristics sandwiching the aforementioned block boundary is amplified.

Accordingly, the encoding device can determine filter characteristics that have an effect on the error of the quantization parameter.

For example, in the determination of the filter characteristics, the difference in the pixel values may be larger, and the difference in the filter characteristics sandwiching the block boundaries may be further reduced.

According to this, for example, when the subjective upper block boundary is easily noticeable, it is possible to suppress the occurrence of the smoothing due to the asymmetry, and it is possible to suppress the deterioration of the subjective function.

For example, in the determination of the filter characteristics, the filter characteristics may be determined based on the dispersion of the pixel values.

A decoding apparatus according to an aspect of the present disclosure includes: a processor and a memory, wherein the processor uses the memory to determine a filter characteristic that is asymmetric with respect to a boundary of the block according to a pixel value of a boundary of the block block, and A deblocking filtering process having the aforementioned filter characteristics determined is performed.

Accordingly, the decoding apparatus performs filtering processing, thereby having the possibility of reducing the error, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Further, the decoding apparatus has the possibility of determining an appropriate filter characteristic based on the difference in pixel values at the boundary of the slab block.

For example, in the determination of the filter characteristics, the filter characteristics may be determined based on the difference in pixel values.

For example, in the determination of the filter characteristics, the difference in the pixel values may be larger, and the difference in the filter characteristics sandwiching the block boundaries may be further amplified.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be more reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore it is possible to suppress the possibility of increasing the error of the pixel.

For example, in the determination of the filter characteristics, the difference between the pixel values and the threshold based on the quantization parameter may be compared. When the difference between the pixel values is greater than the threshold, the difference from the pixel value is less than the threshold. At the same time, the difference of the aforementioned filter characteristics sandwiching the aforementioned block boundary is amplified.

Accordingly, the decoding device can determine filter characteristics that have an effect on the error of the quantization parameter.

For example, in the determination of the filter characteristics, the larger the difference between the pixel values is, the smaller the difference in the filter characteristics sandwiching the block boundary may be.

According to this, for example, there is a possibility that it is possible to suppress unnecessary smoothing when the block boundary coincides with the edge of the object in the image.

For example, in the determination of the filter characteristics, the filter characteristics may be determined based on the dispersion of the pixel values.

An encoding method of the present disclosure is to perform the following steps: determining, according to the pixel value of the boundary of the block block, a filter characteristic that is asymmetric with respect to the boundary of the block, and performing a solution block having the determined filter characteristics. Filter processing.

Accordingly, the encoding method performs filtering processing, thereby reducing the possibility of error, and the filtering processing is a filter characteristic having an asymmetric boundary of the clip block. Moreover, the encoding method can determine an appropriate filter characteristic based on the difference in pixel values of the block boundary.

The decoding method of the present disclosure is to perform the following steps: determining, according to the pixel value of the boundary of the block block, a filter characteristic that is asymmetric with respect to the boundary of the block, and performing a solution block having the determined filter characteristics. Filter processing.

Accordingly, the decoding method performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Moreover, the decoding method can determine an appropriate filter characteristic based on the difference in pixel values at the boundary of the block.

An encoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter that is asymmetric with respect to a boundary of the block according to an angle of a prediction direction of the intra prediction and a boundary of the block boundary. Characteristic, a deblocking filtering process having the aforementioned filter characteristics determined is performed.

Accordingly, the encoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical boundary of the clip block. Further, the encoding apparatus can determine an appropriate filter characteristic based on the prediction direction of the intra prediction and the angle of the block boundary.

For example, in the determination of the filter characteristics, the closer the angle is to the vertical, the more the difference in the filter characteristics sandwiching the block boundary may be amplified.

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Further, since the encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the closer the angle is to the horizontal, the smaller the difference in the filter characteristics sandwiching the block boundary.

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Further, since the encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, there is a possibility that the error of the pixel can be suppressed from increasing.

A decoding apparatus according to an aspect of the present disclosure includes a processor and a memory. The processor uses the memory to determine a filter that is asymmetric with respect to a boundary of the block according to an angle of a prediction direction of the intra prediction and a boundary of the block boundary. Characteristic, a deblocking filtering process having the aforementioned filter characteristics determined is performed.

Accordingly, the decoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Further, the decoding apparatus has a possibility of determining an appropriate filter characteristic based on the prediction direction of the intra prediction and the angle of the block boundary.

For example, in the determination of the filter characteristics, the closer the angle is to the vertical, the more the difference in the filter characteristics sandwiching the block boundary may be amplified.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the closer the angle is to the horizontal, the smaller the difference in the filter characteristics sandwiching the block boundary.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore there is a possibility that the error of the pixel can be suppressed from increasing.

An encoding method of the present disclosure is to perform the following steps: determining, according to the prediction direction of the intra prediction and the angle of the block boundary, a filter characteristic that is asymmetric with respect to the boundary of the block, and performing the determined filter characteristic Solution block filtering.

Accordingly, the encoding method performs filtering processing, thereby reducing the possibility of error, and the filtering processing is a filter characteristic having an asymmetric boundary of the clip block. Moreover, the encoding method can determine an appropriate filter characteristic based on the prediction direction of the intra prediction and the angle of the block boundary.

The decoding method of the present disclosure is to perform the following steps: determining the filter characteristics of the clip boundary from being asymmetric according to the prediction direction of the intra prediction and the angle of the block boundary, and performing the aforementioned filter characteristics with the determined filter characteristics. Deblocking filter processing.

Accordingly, the decoding method performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Moreover, the decoding method can determine an appropriate filter characteristic based on the prediction direction of the intra prediction and the angle of the block boundary.

An encoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine an asymmetric filter characteristic according to a position in a block of the target pixel. The target pixel performs deblocking filtering processing having the aforementioned filter characteristics that have been determined.

Accordingly, the encoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing has filter characteristics in which the block boundary is asymmetric. Further, the encoding device can determine an appropriate filter characteristic based on the position in the block of the target pixel.

For example, in the determination of the filter characteristics, the filter characteristics may be determined in such a manner that the influence of the filter processing becomes larger as the pixel is farther from the intra-predicted reference pixel.

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Further, since the encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the aforementioned filter characteristics can be determined in such a manner that the influence of the above-described deblocking filtering process of the lower right pixel is greater than the influence of the above-described deblocking processing of the upper left pixel. .

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be more reduced. Moreover, this encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore it is possible to suppress the possibility of increasing the error of the pixel.

A decoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter characteristic that the boundary of the clip block is asymmetric according to a position in a block of the target pixel. The target pixel performs deblocking filtering processing having the aforementioned filter characteristics that have been determined.

Accordingly, the decoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Further, the decoding device can determine an appropriate filter characteristic based on the position in the block of the target pixel.

For example, in the determination of the filter characteristics, the filter characteristics may be determined such that the pixels which are farther from the reference pixels are more distant, and the influence of the filtering process becomes larger.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the aforementioned filter may be determined in such a manner that the influence of the deblocking filtering process of the lower right pixel is greater than the influence of the deblocking process of the upper left pixel. characteristic.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be more reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore it is possible to suppress the possibility of increasing the error of the pixel.

An encoding method according to an aspect of the present disclosure is to perform the following steps: determining, according to a position in a block of a target pixel, an asymmetric filter characteristic of the block boundary, and performing the determined filter characteristic on the target pixel. Deblocking filter processing.

Accordingly, the encoding method performs filtering processing, thereby reducing the possibility of error, and the filtering processing is a filter characteristic having an asymmetric boundary of the clip block. Moreover, the encoding method can determine an appropriate filter characteristic based on the pixel value of the boundary of the clip block.

The decoding method of the present disclosure is to perform the following steps: determining, according to the position in the block of the target pixel, a filter characteristic that the boundary of the clip block is asymmetric, and performing the foregoing filter characteristic on the target pixel. Deblocking filter processing.

Accordingly, the decoding method performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Moreover, the decoding method can determine an appropriate filter characteristic based on the position within the block of the target pixel.

An encoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter characteristic that is asymmetric of a boundary of a clip block according to a quantization parameter, and performs the determined filter. Feature deblocking filtering.

Accordingly, the encoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical boundary of the clip block. Further, the encoding device can determine an appropriate filter characteristic based on the quantization parameter.

For example, in the determination of the filter characteristics, the filter characteristics may be determined such that the larger the quantization parameter is, the larger the influence of the deblocking filtering process is.

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Further, since the encoding apparatus can reduce the influence of the filtering process on the pixel having a small error, there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the filter characteristics may be determined in such a manner that the change in the influence of the change in the quantization parameter of the pixel on the lower right side is The change in the aforementioned influence of the change of the aforementioned quantization parameter of the pixel at the upper left is large.

Accordingly, the encoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced.

A decoding apparatus according to an aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to determine a filter characteristic in which a boundary of a clip block is asymmetric according to a quantization parameter, and performs the filter having the determined filter. Feature deblocking filtering.

Accordingly, the decoding apparatus performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Further, the decoding device can determine an appropriate filter characteristic based on the quantization parameter.

For example, in the determination of the filter characteristics, the filter characteristics may be determined such that the larger the quantization parameter is, the larger the influence of the deblocking filtering process is.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced. Moreover, this decoding apparatus can reduce the influence of the filtering process on the pixel having a small error, and therefore there is a possibility that the error of the pixel can be suppressed from increasing.

For example, in the determination of the filter characteristics, the filter characteristics may be determined in such a manner that the change in the influence of the change in the quantization parameter of the pixel on the lower right side is The change in the aforementioned influence of the change of the aforementioned quantization parameter of the pixel at the upper left is large.

Accordingly, the decoding apparatus can amplify the influence of the filtering process on the pixel having a large error, and thus there is a possibility that the error of the pixel can be further reduced.

The encoding method of the present disclosure may also perform the following steps: determining, based on the quantization parameter, a filter characteristic in which the boundary of the clip block is asymmetric, and performing deblocking filtering processing having the determined filter characteristics.

Accordingly, the encoding method performs filtering processing, thereby reducing the possibility of error, and the filtering processing is a filter characteristic having an asymmetric boundary of the clip block. Moreover, the encoding method can determine an appropriate filter characteristic based on the quantization parameter.

The decoding method of the present disclosure may also perform the following steps: determining, based on the quantization parameter, a filter characteristic whose clip boundary is asymmetric, and performing deblocking filtering processing having the determined filter characteristics.

Accordingly, the decoding method performs filtering processing, thereby having the possibility of reducing errors, and the filtering processing is a filter characteristic having an asymmetrical block boundary. Moreover, the decoding method can determine an appropriate filter characteristic based on the quantization parameter.

In addition, the generality or the specific aspect can be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM recording medium, or a system, a method, or a product. Any combination of body circuits, computer programs, and recording media is implemented.

Hereinafter, embodiments will be specifically described with reference to the drawings.

In addition, each of the embodiments described below is an example showing generality or specificity. The numerical values, shapes, materials, constituent elements, arrangement positions, connection forms, steps, and order of steps shown in the following embodiments are merely examples, and the scope of the claims is not limited. Further, among the constituent elements in the following embodiments, the constituent elements of the independent request items that are not described in the uppermost concept are described as arbitrary constituent elements. (Embodiment 1)

First, an outline of the first embodiment will be described with respect to an example of an encoding device and a decoding device to which the processing and/or configuration described in each aspect of the present disclosure to be described later is applicable. However, the first embodiment is merely an example of an encoding device and a decoding device to which the processing and/or configuration described in the various aspects of the present disclosure can be applied. The processing and/or configuration described in the present disclosure may be applied to In the encoding device and the decoding device of the first embodiment.

In the first embodiment, when the processing and/or configuration described in each aspect of the present disclosure is applied, for example, any of the following aspects may be employed. (1) The coding device or the decoding device according to the first embodiment replaces the constituent elements corresponding to the constituent elements described in the respective aspects of the present disclosure among the plurality of constituent elements constituting the encoding device or the decoding device. The component described in each aspect of the cost disclosure; (2) The coding device or the decoding device according to the first embodiment applies the component of a part of the plurality of components constituting the coding device or the decoding device first. Any changes in the functions, or additions, substitutions, deletions, and the like of the processes to be performed, and the components corresponding to the components described in the aspects of the present disclosure are replaced by the descriptions of the various aspects of the cost disclosure. (3) The method to be implemented by the encoding device or the decoding device according to the first embodiment, the processing addition, and/or the processing of a part of the plurality of processing included in the method, first, replacement, deletion, etc. After any arbitrary change, the processing corresponding to the processing described in each aspect of the present disclosure is replaced by the various aspects disclosed in the cost disclosure. (4) The constituent elements of a part of the plurality of constituent elements constituting the encoding apparatus or the decoding apparatus of the first embodiment, and the constituent elements described in the respective aspects of the present disclosure, and the various aspects of the present disclosure The constituent elements of one of the functions of the constituent elements described in the example or the constituent elements of the processing to be performed by the constituent elements described in the various aspects of the present disclosure are combined; (5) Among the constituent elements of one of the plurality of constituent elements of the plurality of constituent elements of the encoding device or the decoding device of the first embodiment, or a plurality of constituent elements constituting the encoding device or the decoding device of the first embodiment The constituent elements of a part of the processing performed by the constituent elements, and the constituent elements described in the various aspects of the present disclosure, and the components having the functions of the constituent elements described in the various aspects of the present disclosure Element or part of the processing performed by the constituent elements described in the various aspects of the present disclosure (6) The method to be implemented by the encoding device or the decoding device according to the first embodiment, among the plurality of processes included in the method, will be described in the respective aspects of the present disclosure. The processing of the processing replaces the processing to be described in the various aspects of the cost disclosure; (7) processing a part of the plurality of processing included in the method to be implemented by the encoding apparatus or the decoding apparatus of the first embodiment, and the present disclosure The processes described in the various aspects are combined in combination.

Further, the embodiments of the processes and/or configurations described in the various aspects of the present disclosure are not limited to the above examples. For example, it may be implemented in a device different from the moving image/image encoding device or the moving image/image decoding device disclosed in the first embodiment, or may be implemented separately. The processing and/or composition illustrated in the aspects. Further, the processes and/or configurations already described in the different aspects may be combined. [Summary of coding device]

First, an outline of an encoding apparatus according to the first embodiment will be described. Fig. 1 is a block diagram showing the functional configuration of an encoding apparatus 100 according to the first embodiment. The encoding device 100 is a moving image/image encoding device that encodes moving images/images in block units.

As shown in FIG. 1, the encoding apparatus 100 is an apparatus for encoding an image in units of blocks, and includes a division unit 102, a subtraction unit 104, a conversion unit 106, a quantization unit 108, an entropy coding unit 110, and an inverse quantization unit 112. The inverse conversion unit 114, the addition unit 116, the block memory 118, the loop filter unit 120, the frame memory 122, the intra prediction unit 124, the inter prediction unit 126, and the prediction control unit 128.

The encoding device 100 is realized by, for example, a general-purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor functions as the dividing unit 102, the subtracting unit 104, the converting unit 106, the quantization unit 108, the entropy encoding unit 110, the inverse quantization unit 112, and the counter. The conversion unit 114, the addition unit 116, the loop filter unit 120, the intra prediction unit 124, the inter prediction unit 126, and the prediction control unit 128 function. Further, the encoding device 100 may correspond to the division unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy coding unit 110, the inverse quantization unit 112, the inverse conversion unit 114, the addition unit 116, and the loop filter unit 120. The internal prediction unit 124, the inter prediction unit 126, and the prediction control unit 128 are implemented by one or more dedicated electronic circuits.

Hereinafter, each component included in the encoding device 100 will be described. [Division Department]

The division unit 102 divides each picture included in the input moving image into a plurality of blocks, and outputs each block to the subtraction unit 104. For example, the dividing unit 102 first divides the picture into blocks of a fixed size (for example, 128 × 128). This fixed size block is sometimes referred to as a coding tree unit (CTU). Next, the dividing unit 102 divides each of the fixed-size blocks into variable sizes (for example, 64×64 or less) according to the recursive quadtree and/or binary tree partitioning. ) The block. This variable size block is sometimes referred to as a coding unit (CU), a prediction unit (PU), or a conversion unit (TU). Further, in the present embodiment, it is not necessary to distinguish between CU, PU, and TU, and some or all of the blocks in the picture may be processing units of CU, PU, and TU.

Fig. 2 is a view showing an example of block division in the first embodiment; In Fig. 2, the solid line indicates the block boundary obtained by the division of the quaternary tree block, and the broken line indicates the block boundary obtained by the division of the binary tree block.

Here, block 10 is a square block of 128 x 128 pixels (128 x 128 blocks). The 128 x 128 block 10 is first divided into 64 square 64 blocks (quaternary tree block partitioning) divided into 4 squares.

The upper left 64×64 block is a 32×64 block that is further vertically divided into two rectangles, and the left 32×64 block is a 16×64 block that is further vertically divided into two rectangles (binary tree area) Block split). As a result, the upper left 64x64 block is divided into two 16x64 blocks 11, 12, and 32x64 blocks 13.

The upper right 64x64 block is a 64x32 block 14, 15 (binary tree block partition) that is horizontally divided into two rectangles.

The lower left 64x64 block is a 32x32 block (quaternary tree block partition) that is divided into 4 squares. Among the four 32×32 blocks, the upper left block and the lower right block are further divided. The upper left 32×32 block is a 16×32 block that is vertically divided into two rectangles, and the right 16×32 block is further horizontally divided into two 16×16 blocks (binary tree block division). . The lower right 32x32 block is horizontally divided into two 32x16 blocks (binary tree block partitioning). As a result, the lower left 64×64 block is divided into one 16×32 block 16, two 16×16 blocks 17, 18, two 32×32 blocks 19, 20, and two 32×. Block 16 21, 22.

The 64×64 block 23 at the lower right is not divided.

As above, in Fig. 2, the block 10 is divided into 13 variable-sized blocks 11 to 23 according to the recursive quadtree and binary tree block division. This division is sometimes referred to as QTBT (quad-tree plus binary tree) segmentation.

In addition, in FIG. 2, one block is divided into four or two blocks (quaternary tree or binary tree block division), and the division is not limited thereto. For example, one block can also be divided into three blocks (three-dimensional tree block division). Such division including ternary tree block division is sometimes referred to as MBT (multi type tree) division. [Subtraction Department]

The subtraction unit 104 subtracts the prediction signal (predicted sample) from the original signal (original sample) in the block unit divided by the division unit 102. In other words, the subtraction unit 104 is a prediction error (also referred to as a residual) for calculating a coding target block (hereinafter referred to as a current block). Next, the subtraction unit 104 outputs the calculated prediction error to the conversion unit 106.

The original signal is an input signal of the encoding device 100, and is a signal (for example, a luma signal and two chroma signals) indicating an image of each picture constituting the moving image. In the following, the signal representing the image is also referred to as a sample. [conversion department]

The conversion unit 106 converts the prediction error of the spatial region into a conversion coefficient of the frequency region, and outputs the conversion coefficient to the quantization unit 108. Specifically, the conversion unit 106 performs, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST) that has been determined in advance for the prediction error of the spatial region.

Further, the conversion unit 106 may adaptively select a conversion pattern from among a plurality of conversion patterns, and convert a prediction error into a conversion coefficient using a transform basis function corresponding to the selected conversion pattern. Such a conversion is sometimes referred to as an EMT (explicit multiple core transform) or an AMT (adaptive multiple transform).

A plurality of conversion patterns include, for example, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII. Figure 3 is a table showing the conversion basis functions corresponding to the respective conversion patterns. In Figure 3, N is the number of input pixels displayed. The selection of the conversion pattern from among the plurality of conversion patterns may be based on, for example, the type of prediction (internal prediction and inter prediction) or the intra prediction mode.

Information indicating whether such EMT or AMT is applicable (for example, referred to as an AMT flag) and information indicating the selected conversion pattern is signaled at the CU level. In addition, the signalization of such information is not necessarily limited to the CU level, but may also be other levels (eg, sequence level, picture level, slice level, tile level, or CTU). grade).

Further, the conversion unit 106 may reconvert the conversion coefficient (conversion result). Such reconversion is sometimes referred to as AST (adapive secondary transform) or NSST (non-separable secondary transform). For example, the conversion unit 106 performs reconversion on each sub-block (for example, a 4 × 4 sub-block) included in the block corresponding to the conversion coefficient of the intra prediction error. The information showing whether NSST is applicable and the information about the conversion matrix used in NSST is signaled by the CU level. In addition, the signalization of such information is not necessarily limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level or CTU level).

Here, the separable conversion refers to a method of separating the input dimensions according to the direction and performing the conversion several times. The non-separable conversion refers to a dimension of 2 or more when the input is multi-dimensional. Consolidation, and regarded as a one-dimensional, and then converted together.

For example, in the case of an inseparable conversion, for example, when a block of 4×4 is input, the block is regarded as an array having 16 elements, and the array is 16 The conversion matrix of ×16 performs conversion processing.

Also, similarly, a 4×4 input block is regarded as having an entire array of one of 16 elements, and then the frame is subjected to several Givens rotation (Hypercube Givens Transform). Conversion) is also an example of a non-separable (Non-Separable) conversion. [Quantization Department]

The quantization unit 108 quantizes the conversion coefficients output from the conversion unit 106. Specifically, the quantization unit 108 scans the conversion coefficient of the current block in a predetermined scanning order, and quantizes the conversion coefficient according to the quantization parameter (QP) corresponding to the scanned conversion coefficient. Then, the quantization unit 108 outputs the quantized conversion coefficients (hereinafter referred to as quantized coefficients) of the current block to the entropy encoding unit 110 and the inverse quantization unit 112.

The predetermined order is the order in which the quantization/dequantization of the conversion coefficients is used. For example, the predetermined scan order is defined by the order of the power of the frequency (from the low frequency to the high frequency) or the order of the power reduction (the order from the high frequency to the low frequency).

The quantization parameter refers to a parameter that defines a quantization step (quantization width). For example, if the value of the quantization parameter increases, the quantization step also increases. That is, if the value of the quantization parameter increases, the quantization error also becomes large. [Entropy coding unit]

The entropy coding unit 110 performs variable length coding on the quantized coefficients input from the quantization unit 108, thereby generating an encoded signal (coded bit stream). Specifically, the entropy coding unit 110 performs binarization of the quantized coefficients, for example, and arithmetically encodes the binarized signals. [Anti-quantization department]

The inverse quantization unit 112 inversely quantizes the input quantized coefficients from the quantization unit 108. Specifically, the inverse quantization unit 112 inversely quantizes the quantized coefficients of the current block in a predetermined scanning order. Then, the inverse quantization unit 112 outputs the inverse quantized conversion coefficient of the current block to the inverse conversion unit 114. [Anti-conversion department]

The inverse conversion unit 114 inversely converts the conversion coefficient input from the inverse quantization unit 112, thereby restoring the prediction error. Specifically, the inverse conversion unit 114 performs inverse conversion corresponding to the conversion performed by the conversion unit 106 on the conversion coefficient, thereby restoring the prediction error of the current block. Then, the inverse conversion unit 114 outputs the restored prediction error to the addition unit 116.

Further, since the restored prediction error is lost due to the progress of the quantization, the prediction error calculated by the subtraction unit 104 does not match. That is, the quantization error is included in the restored prediction error. [Addition Department]

The addition unit 116 adds the prediction error input from the inverse conversion unit 114 to the prediction sample input from the prediction control unit 128, thereby reconstructing the current block. Then, the addition unit 116 outputs the reconstructed block to the block memory 118 and the loop filter unit 120. Reconstituted blocks are sometimes referred to as local decoded blocks. [block memory]

The block memory 118 is a memory for storing a block, wherein the block is a block to be referred to for intra prediction, and is a block within a coded object picture (hereinafter referred to as a current picture). Specifically, the tile memory 118 stores the reconstructed block output from the addition unit 116. [loop filter unit]

The loop filter unit 120 applies loop filtering to the block reconstructed by the pass-through unit 116, and outputs the filtered reconstructed block to the frame memory 122. Loop filtering refers to the filters (intra-loop filters) used in the coding loop, including, for example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF).

In ALF, the least square error filter used to remove the coding distortion is applied, for example, according to the respective 2×2 sub-blocks in the current block, according to the direction of the local gradient and the activity. One filter selected from a plurality of filters is applied.

Specifically, first, sub-blocks (for example, 2×2 sub-blocks) are classified into a plurality of categories (for example, 15 or 25 types). The classification of sub-blocks is based on the direction and activity of the gradient. For example, the classification value C (for example, C=5D+A) is calculated using the gradient direction value D (for example, 0 to 2 or 0 to 4) and the gradient activity value A (for example, 0 to 4). Then, based on the classification value C, the sub-blocks are classified into a plurality of categories (for example, 15 or 25 types).

The direction value D of the gradient is derived, for example, by comparing gradients in a plurality of directions (eg, horizontal, vertical, and 2 diagonal directions). Further, the gradient activity value A is derived, for example, by adding the gradients of the plurality of directions and quantizing the addition result.

Based on the result of such classification, a filter for a sub-block is determined from among a plurality of filters.

For the shape of the filter used in the ALF, for example, a circularly symmetrical shape is utilized. 4A to 4C are diagrams showing a plurality of examples of shapes of filters used in ALF. Fig. 4A shows a 5 x 5 diamond shaped filter, Fig. 4B shows a 7 x 7 diamond shaped filter, and Fig. 4C shows a 9 x 9 diamond shaped filter. The information showing the shape of the filter is signaled at the picture level. In addition, the signalization of the information showing the shape of the filter is not limited to the picture level, and may be other levels (for example, sequence level, slice level, block level, CTU level, or CU level).

The ALF is turned on/off, for example, by picture level or CU level. For example, for brightness, it is determined whether or not ALF is applied by the CU level, and for the color difference, whether or not ALF is applied is determined by the picture level. The information showing the ALF on/off is signalized by the picture level or CU level. In addition, the information indicating the on/off of the ALF is not limited to the picture level or the CU level, and may be other levels (such as sequence level, slice level, block level, or CTU level).

The set of coefficients for a selectable plurality of filters (e.g., filters up to 15 or 25) is signaled at the picture level. In addition, the signalization of the coefficient set need not be limited to the picture level, but may be other levels (such as sequence level, slice level, block level, CTU level, CU level, or sub-block level). [frame memory]

The frame memory 122 is a memory portion for storing reference pictures used for inter prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 122 stores the reconstructed block that has been filtered by the loop filter unit 120. [Internal forecasting department]

The intra prediction unit 124 performs intra prediction (also referred to as intra-picture prediction) of the current block by referring to the block in the current picture stored in the block memory 118 to generate a prediction signal (inter prediction signal). Specifically, the intra prediction unit 124 performs intra prediction by referring to samples (eg, luminance values, color difference values) of the blocks adjacent to the current block to generate an intra prediction signal, and outputs the intra prediction signal to the prediction control. Department 128.

For example, the intra prediction unit 124 performs intra prediction by using one of a plurality of intra prediction modes that have been previously defined. The plurality of intra prediction modes include one or more non-directional prediction modes and a plurality of directional prediction modes.

One or more non-directional prediction modes include, for example, a Planar prediction mode and a direct current (DC) defined by the H.265/HEVC (High-Efficiency Video Coding) specification (Non-Patent Document 1). ) Prediction mode.

A plurality of directional prediction modes include, for example, prediction modes of 33 directions specified by the H.265/HEVC specification. In addition, the plurality of directional prediction modes may further include prediction modes of 32 directions (a total of 65 directional prediction modes) in addition to 33 directions. FIG. 5A is a diagram showing 67 intra prediction modes (two non-directional prediction modes and 65 directional prediction modes) in intra prediction. The solid arrow symbol indicates 33 directions defined by the H.265/HEVC standard, and the dotted arrow symbol indicates the 32 additional directions added.

In addition, in the intra prediction of the color difference block, the luminance block can also be referred to. That is, the color difference component of the current block can also be predicted based on the luminance component of the current block. Such intra prediction is sometimes referred to as CCLM (cross-component linear model) prediction. An intra prediction mode such as a color difference block of the reference luminance block (for example, referred to as CCLM mode) can also be added as an intra prediction mode of one color difference block.

The intra prediction unit 124 may correct the intra-predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction. The intra prediction accompanying the correction like this is sometimes referred to as PDPC (position dependent intra prediction combination). Applicable information showing the presence or absence of a PDPC (for example, referred to as a PDPC flag), for example, is signaled at the CU level. In addition, the signalization of the information need not be limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level, or CTU level). [Inter forecasting department]

The inter prediction unit 126 is a reference reference picture for performing inter-block prediction of the current block (also called inter-picture prediction) to generate a prediction signal (inter-predictive signal), wherein the reference picture is a reference stored by the frame memory 122. Picture, and is a reference picture that is different from the current picture. The inter-prediction is performed in units of a current block or a sub-block within the current block (e.g., a 4x4 block). For example, the inter prediction unit 126 performs motion estimation within the reference picture for the current block or sub-block. Next, the inter prediction unit 126 performs motion compensation using motion information (for example, a motion vector) obtained by motion estimation, thereby generating an inter-prediction signal of the current block or sub-block. Then, the inter prediction unit 126 outputs the generated inter prediction signal to the prediction control unit 128.

The mobile information for motion compensation is signaled. For signalization of motion vectors, a motion vector predictor can also be used. That is, the difference between the motion vector and the motion vector predictor can also be signaled.

In addition, not only the movement information of the current block obtained by the motion estimation but also the movement information of the adjacent block may be used to generate the inter prediction signal. Specifically, the prediction signal according to the mobile information obtained by the motion estimation and the prediction signal according to the mobile information of the adjacent block may be weighted and added, thereby using the sub-block unit in the current block. Generate inter-predictive signals. Such prediction (motion compensation) is sometimes referred to as OBMC (overlapped block motion compensation).

In such an OBMC mode, information showing the size of a sub-block for OBMC (for example, referred to as an OBMC block size) is signaled at a sequence level. Also, information indicating whether or not the OBMC mode is applicable (for example, referred to as an OBMC flag) is signaled by the CU level. In addition, the level of signalization of such information need not be limited to the sequence level and CU level, but may also be other levels (such as picture level, slice level, block level, CTU level, or sub-block level).

The OBMC mode will be described more specifically. 5B and 5C are a flowchart and a conceptual diagram for explaining an outline of a predicted image correction process performed by the OBMC process.

First, a motion vector (MV) assigned to a coding target block is used to obtain a predicted image (Pred) obtained by normal motion compensation.

Next, the motion vector (MV_L) of the encoded left adjacent block is applied to the coding target block, the predicted image (Pred_L) is obtained, and the predicted image and the Pred_L are weighted and superimposed to perform the predicted image. The first correction.

In the same manner, the motion vector (MV_U) of the adjacent block that has been encoded is applied to the coding target block, and the predicted image (Pred_U) is obtained, and the predicted image with the first correction and the Pred_U are weighted. This is superimposed to perform the second correction of the predicted image, and this is used as the final predicted image.

In addition, a method of using the two-stage correction of the left adjacent block and the upper adjacent block is described here, but it is also possible to adopt a configuration in which the right adjacent block or the lower adjacent block is used for more times than the 2 stages. The composition of the correction.

In addition, the superimposed region may be an area that is only a part of the vicinity of the block boundary, rather than the pixel area of the block as a whole.

Here, although the prediction image correction processing from one reference picture is described here, the same applies to the case where the prediction image is corrected from the plurality of reference pictures, and the corrected prediction map is obtained from each reference picture. After the image, the obtained predicted image is further superimposed as the final predicted image.

In addition, the foregoing processing target block may also be a prediction block unit, or may be a sub-block unit that further divides the prediction block.

As a method of determining whether or not the OBMC processing is applied, for example, there is a method of using obmc_flag which is a signal indicating whether or not OBMC processing is applied. In a specific example, in the encoding apparatus, it is determined whether the encoding target block belongs to an area where the movement is complicated, and when the movement is a complex area, the set value is 1 as obmc_flag, and the OBMC processing is applied for encoding, When the movement is a complex area, the value is set to 0 as obmc_flag, and OBMC processing is not applied for encoding. On the other hand, in the decoding apparatus, the obmc_flag described in the stream is decoded, and the OBMC processing is switched depending on the value, and decoding is performed.

In addition, the mobile information can be derived from the decoding device side without being signaled. For example, a merge mode specified by the H.265/HEVC specification may also be employed. Further, for example, the motion estimation may be performed on the decoding device side to derive the mobile information. At this time, the motion estimation is performed without using the pixel value of the current block.

Here, a mode in which the motion estimation is performed on the decoding device side will be described. The mode in which the motion estimation is performed on the decoding device side is sometimes referred to as a PMMVD (pattern matched motion vector derivation) mode or a FRUC (frame rate up-conversion) mode.

An example of FRUC processing is shown in Figure 5D. First, a reference to a motion vector of a coded block that is spatially or temporally adjacent to the current block is generated, and a list of a plurality of candidates (which may also be common to the merge list) is generated, the list of the plurality of candidates each having a motion vector predictor . Next, the best candidate MV is selected from among a plurality of candidate MVs that have been registered in the candidate list. For example, the evaluation value of each candidate included in the candidate list is calculated, and one candidate is selected based on the evaluation value.

Next, the motion vector for the current block is derived based on the selected candidate motion vector. Specifically, for example, the selected candidate motion vector (best candidate MV) is derived as the motion vector for the current block. Also, pattern matching is performed, for example, in a peripheral region of a position within a reference picture, whereby a motion vector for the current block can also be derived, wherein the reference picture is a motion vector corresponding to the selected candidate. That is, for the area around the best candidate MV, the search is performed in the same way, and when there is an MV whose evaluation value is a good number, the best candidate MV is updated to the aforementioned MV, and the MV is regarded as the current block. The last MV is also available. Alternatively, the configuration may be omitted.

When processing in sub-block units, it is also possible to configure the same processing.

Further, the evaluation value is calculated by obtaining a difference value of the reconstructed image by matching the region in the reference picture corresponding to the motion vector with the predetermined region. In addition to the difference value, information other than this can be used to calculate the evaluation value.

For pattern matching, the first pattern matching or the second pattern matching is used. The first pattern matching and the second pattern matching are sometimes referred to as bidirectional matching and template matching, respectively.

In the first pattern matching, pattern matching is performed between two blocks, which are two blocks in two different reference pictures, and are movement trajectories along the current block (motion Trajectory). Therefore, in the first pattern matching, an area in another reference picture along the movement trajectory of the current block is used as a predetermined area for calculating the evaluation value of the candidate.

Fig. 6 is a view for explaining an example of pattern matching (bidirectional matching) between two blocks along a movement trajectory. As shown in FIG. 6, under the first pattern matching, two blocks in the moving trajectory along the current block (Cur block) and two regions in the two different reference pictures (Ref0, Ref1) Among the pair of blocks, the most matching pair is searched, thereby deriving two motion vectors (MV0, MV1). Specifically, for the current block, the reconstructed image in the specified position in the first encoded reference picture (Ref0) specified by the candidate MV is derived, and the candidate MV has been scaled at the display time interval ( The difference between the reconstructed images in the specified position in the second encoded reference picture (Ref1) specified by the symmetric MV of the scaling is calculated using the obtained difference value. Among the plurality of candidate MVs, the candidate MV whose evaluation value is the best is selected as the last MV.

Under the assumption of continuous moving trajectory, the distance between the motion vector (MV0, MV1) of the two reference blocks relative to the current picture (Cur Pic) and the two reference pictures (Ref0, Ref1) is indicated (TD0) TD1) is proportional. For example, when the current picture is temporally located between two reference pictures, when the distance from the current picture to the two reference pictures is equal, on the first pattern matching, a mirror-symmetric two-way motion vector can be derived.

On the second pattern matching, a template in the current picture (a block adjacent to the current block in the current picture (eg, upper and/or left adjacent blocks)) and a block in the reference picture are patterned match. Therefore, in the second pattern matching, a block adjacent to the current block in the current picture is used as a predetermined area for calculation of the above-described candidate evaluation value.

FIG. 7 is a diagram for explaining an example of pattern matching (template matching) between a template in a current picture and a block in a reference picture. As shown in FIG. 7, in the second pattern matching, a block that matches the block closest to the current block (Cur block) within the current picture (Cur Pic) is searched within the reference picture (Ref0), thereby Export the motion vector of the current block. Specifically, for the current block, the reconstructed image of the coded region of the left adjacent and the adjacent two sides or either side is derived in the same position as the encoded reference picture (Ref0) specified by the candidate MV. The difference between the images is reconstructed, and the evaluation value is calculated using the obtained difference value, and the candidate MV whose evaluation value is the optimum value is selected among the plurality of candidate MVs as the best candidate MV.

Such information showing whether the FRUC mode is applicable (for example, referred to as the FRUC flag) is signaled by the CU level. Moreover, when the FRUC mode is applied (for example, when the FRUC flag is true), the information of the method of displaying the pattern matching (the first pattern matching or the second pattern matching) (for example, referred to as the FRUC mode flag) is based on the CU level. Be signaled. In addition, the signalization of such information is not limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level, CTU level or sub-block level).

Here, a description will be given of a mode in which a motion vector is derived from a model, which is a model assumed to be a constant-speed linear motion. This mode is sometimes referred to as BIO (bi-directional optical flow) mode.

Fig. 8 is a view for explaining a model assumed to be a constant-speed linear motion. In Fig. 8, (v _x , v _y ) represents a velocity vector, and τ ₀ and τ _{1 are} each represented as a temporal distance between a current picture (Cur Pic) and two reference pictures (Ref ₀ , Ref ₁ ). (MVx ₀ , MVy ₀ ) is a motion vector indicating a reference picture Ref ₀ , and (MVx ₁ , MVy ₁ ) is a motion vector indicating a reference picture Ref ₁ .

At this time, the velocity vectors (v _x , v _y ) are under the assumption of constant-speed linear motion, and (MVx ₀ , MVy ₀ ) and (MVx ₁ , MVy ₁ ) are expressed as (v _x τ ₀ , v _y τ ₀ And (-v _x τ ₁ , -v _y τ ₁ ), the following optical flow equation (1) is established. (Number 1)

Here, I ^(k) is a luminance value indicating a reference image k (k = 0, 1) after the motion compensation. The optical flow equation is a product that displays (i) time differential of the luminance value, (ii) the horizontal direction velocity, and the horizontal component of the spatial gradient of the reference image, and (iii) the velocity in the vertical direction and the space of the reference image. The sum of the products of the vertical components of the gradient is equal to zero. According to the combination of the optical flow equation and the Hermitian interpolation, the motion vector of the block unit obtained from the merge list or the like is corrected in units of pixels.

Alternatively, the motion vector can be derived on the decoding device side by a method different from the derivation of the motion vector of the model based on the assumed constant velocity linear motion. For example, the motion vector may also be derived in sub-block units based on the motion vectors of the plurality of contiguous blocks.

Here, a mode in which a motion vector is derived in units of sub-blocks based on a motion vector of a plurality of adjacent blocks will be described. This mode is sometimes referred to as an affine motion compensation prediction mode.

Fig. 9A is a diagram for explaining the derivation of a motion vector of a sub-block unit, which is performed based on a motion vector of a plurality of adjacent blocks. In Figure 9A, the current block contains 16 4x4 sub-blocks. Here, the motion vector v ₀ of the upper left corner control point of the current block is derived according to the motion vector of the adjacent block, and the motion vector v ₁ of the upper right corner control point of the current block is derived according to the motion vector of the adjacent subblock. . Next, using two motion vectors v ₀ and v ₁ , the motion vectors (v _x , v _y ) of the respective sub-blocks in the current block are derived via the following equation (2). (number 2)

Here, x and y each represent the horizontal position and the vertical position of the sub-block, and w represents the weight coefficient which has been previously set.

In such an affine motion compensation prediction mode, it is also possible to include several modes in which the motion vectors of the upper left and upper right control points are different. Information showing such an affine motion compensated prediction mode (e.g., referred to as an affine flag) is signaled at the CU level. In addition, the signalization of the information showing the affine motion compensation prediction mode need not be limited to the CU level, but may be other levels (eg, sequence level, picture level, slice level, block level, CTU level, or sub-block level). [Predictive Control Department]

The prediction control unit 128 selects any one of the intra prediction signal and the inter prediction signal, and outputs the selected signal as a prediction signal to the subtraction unit 104 and the addition unit 116.

Here, an example of deriving a motion vector of a coding target picture via a merge mode will be described. Fig. 9B is a diagram for explaining an outline of a motion vector derivation process by a merge mode.

First, a list of predicted MVs of candidates for the registered prediction MV is generated. The candidate for predicting the MV includes: a spatial neighbor prediction MV, which is an MV of a plurality of coded blocks located in the space of the coding target block; the temporal adjacent prediction MV is projected to the encoded reference picture. The MV of the nearby block in the position of the coding target block; the combined prediction MV is the MV generated by combining the MV value of the spatial neighbor prediction MV and the temporal neighbor prediction MV; and the zero prediction MV whose value is zero MV and so on.

Next, one prediction MV is selected from among the plurality of prediction MVs that have been registered in the prediction MV list, and this is determined as the MV of the encoding target block.

Further, in the variable length coding unit, the merge_idx is described and encoded in the stream, wherein the merge_idx is a signal indicating which prediction MV has been selected.

In addition, the predicted MV registered in the predicted MV list illustrated in FIG. 9B is only an example, and may be a number different from the number in the figure, or a type including a part of the predicted MV in the figure, or The structure of the prediction MV other than the type of the prediction MV in the figure is added.

Alternatively, the MV of the encoding target block derived by the merge mode may be used to perform the DMVR processing described later, thereby determining the last MV.

Here, an example in which MV is determined using DMVR processing will be described.

Fig. 9C is a conceptual diagram for explaining an outline of DMVR processing.

First, the most suitable MVP set in the processing target block is used as the candidate MV, and the processed picture from the L0 direction, that is, the first reference picture and the processed picture in the L1 direction, that is, the second reference, according to the candidate MV. The picture is obtained by taking reference pixels and taking the average of each reference pixel to generate a template.

Next, using the template, the peripheral regions of the candidate MVs of the first reference picture and the second reference picture are respectively searched, and the MV whose cost is the smallest is determined as the last MV. Further, the cost value is calculated by using a difference value between each pixel value of the template and each pixel value of the search area, an MV value, and the like.

Further, in the encoding device and the decoding device, the outline of the processing described herein is basically common.

Further, even if it is not the processing content described here, other processing may be used as long as it is a process of searching for the periphery of the candidate MV and deriving the last MV.

Here, a mode in which a predicted image is generated using the LIC processing will be described.

9D is a diagram for explaining an outline of a prediction image generation method using luminance correction processing by LIC processing.

First, the MV is derived from the reference picture, wherein the reference picture is an encoded picture, and the MV is used to obtain a reference picture corresponding to the coding target block.

Next, for the coding target block, the brightness pixel value of the coded peripheral reference area of the left adjacent and upper adjacent, and the brightness pixel value of the same position within the reference picture specified by the MV are used, and the display brightness value is used for reference. The information on how the picture and the encoded object picture change, and the brightness correction parameter is calculated.

For the reference image in the reference picture specified by the MV, the brightness correction processing is performed using the aforementioned luminance correction parameter, thereby generating a predicted image with respect to the encoding target block.

In addition, the shape of the aforementioned peripheral reference area in FIG. 9D is only one example, and other shapes may be used.

Here, the process of generating a predicted image from one reference picture has been described here, but the same is true for the case where a predicted image is generated from a plurality of reference pictures, and the reference image that has been acquired from each reference picture is first used. The brightness correction processing is performed in the same manner, and then the predicted image is generated.

For the method of determining whether or not the LIC processing is applicable, for example, there is a method of using lic_flag, which is a signal indicating whether LIC processing is applicable. In a specific example, the encoding apparatus determines whether the encoding target block belongs to an area where the luminance change occurs, and if it belongs to the area where the luminance change occurs, sets the value to 1 for lic_flag, and encodes the LIC processing. If it is not in the region where the luminance change occurs, set the value to 0 for lic_flag, and encode without applying LIC processing. On the other hand, in the decoding apparatus, the lic_flag described in the stream is decoded, and the LIC processing is switched depending on the value, and decoding is performed.

In another method of determining whether or not the LIC processing is applicable, for example, there is also a method of determining whether or not the LIC processing is applied to the peripheral block. In a specific example, when the coding target block is in the merge mode, it is determined whether the coded block in the vicinity of the MV in the merge mode process is encoded by the LIC process, and the result is whether the switch is performed. Apply LIC processing and encode. In addition, in the case of this example, the processing in decoding is also identical. [Summary of decoding device]

Next, an outline of a decoding apparatus that can decode an encoded signal (encoded bit stream) output from the encoding apparatus 100 will be described. FIG. 10 is a block diagram showing a functional configuration of a decoding device 200 according to the first embodiment. The decoding device 200 is a moving image/image decoding device that decodes a moving image/image in units of blocks.

As shown in FIG. 10, the decoding apparatus 200 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse conversion unit 206, an addition unit 208, a block memory 210, a loop filter unit 212, a frame memory 214, and an intra prediction. The unit 216, the inter prediction unit 218, and the prediction control unit 220.

The decoding device 200 can be realized, for example, by a general purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor functions as the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, and the intra prediction unit 216. The inter prediction unit 218 and the prediction control unit 220 operate. Further, the decoding device 200 may correspond to the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, the intra prediction unit 216, the inter prediction unit 218, and the prediction control unit 220. One or more dedicated electronic circuits are dedicated to implementation.

Hereinafter, each component included in the decoding device 200 will be described. [Entropy decoding unit]

The entropy decoding unit 202 performs entropy decoding on the encoded bit stream. Specifically, the entropy decoding unit 202 performs, for example, arithmetic decoding from a coded bit stream to a binary signal. Next, the entropy decoding unit 202 demultiplexes the binary signal. Thereby, the entropy decoding unit 202 outputs the quantized coefficients to the inverse quantization unit 204 in units of blocks. [Anti-quantization department]

The inverse quantization unit 204 inversely quantizes the quantized coefficients of the decoding target block (hereinafter referred to as the current block) which is an input from the entropy decoding unit 202. Specifically, the inverse quantization unit 204 is for each of the quantized coefficients of the current block, and inversely quantizes the quantized coefficients according to the quantization parameter corresponding to the quantized coefficients. Then, the inverse quantization unit 204 outputs the dequantized quantized coefficients (i.e., conversion coefficients) of the current block to the inverse conversion unit 206. [Anti-conversion department]

The inverse conversion unit 206 inversely converts the conversion coefficient, thereby restoring the prediction error, which is the input from the inverse quantization unit 204.

For example, when the information that has been interpreted from the encoded bit stream is displayed when EMT or AMT is applied (for example, the AMT flag is true), the inverse conversion unit 206 performs the conversion coefficient of the current block according to the information of the converted conversion pattern displayed. Reverse conversion.

Further, for example, when the information that has been interpreted from the encoded bit stream is that the applicable NSST is displayed, the inverse conversion unit 206 applies inverse re-conversion to the conversion coefficient. [Addition Department]

The addition unit 208 adds the prediction error to the prediction block, thereby reconstructing the current block, wherein the prediction error is an input from the inverse conversion unit 206, which is an input from the prediction control unit 220. Then, the addition unit 208 outputs the reconstructed block to the block memory 210 and the loop filter unit 212. [block memory]

The tile memory 210 is a memory portion for storing a block referenced in the intra prediction and for decoding a block in the target picture (hereinafter referred to as a current picture). Specifically, the tile memory 210 stores the reconstructed block output from the addition unit 208. [loop filter unit]

The loop filter unit 212 performs loop filtering on the block that has been reconstructed by the addition unit 208, and outputs the filtered reconstructed block to the frame memory 214, the display device, and the like.

The information showing the on/off of the ALF that has been interpreted from the encoded bit stream is that when the ALF is turned on, one filter is selected from a plurality of filters according to the direction and activity of a part of the gradient. The selected filter is suitable for reconstituting blocks. [frame memory]

The frame memory 214 is a memory for storing reference pictures that are used for inter prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 214 stores the reconstructed block filtered by the loop filter unit 212. [Internal forecasting department]

The intra prediction unit 216 performs intra prediction by referring to the intra-prediction mode that has been interpreted from the encoded bit stream, referring to the block in the current picture stored in the block memory 210, thereby generating a prediction signal (internal prediction signal). Specifically, the intra prediction unit 216 performs intra prediction by referring to samples (for example, luminance values and color difference values) of the blocks adjacent to the current block, thereby generating an intra prediction signal, and outputting the intra prediction signal to the prediction control. Department 220.

Further, in the intra prediction of the chroma block, when the intra prediction mode of the reference luma block is selected, the intra prediction unit 216 may also predict the chroma component of the current block based on the luminance component of the current block.

Further, when the information display PDPC has been interpreted from the encoded bit stream, the intra prediction unit 216 corrects the intra-predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction. [Inter forecasting department]

The inter prediction unit 218 is a reference picture stored by the reference frame memory 214 to predict the current block. The prediction is made in units of the current block or sub-blocks within the current block (eg, 4x4 blocks). For example, the inter prediction unit 218 performs motion compensation using motion information (eg, motion vector) that has been interpreted from the encoded bit stream, thereby generating an inter-prediction signal of the current block or sub-block, and outputting the inter-predicted signal to Prediction control unit 220.

In addition, when the information that has been interpreted from the encoded bit stream is the display of the applicable OBMC mode, the inter prediction unit 218 not only uses the mobile information of the current block obtained through the mobile estimation but also uses the mobile information of the adjacent block to generate Inter prediction signal.

Further, when the information that has been interpreted from the encoded bit stream indicates that the FRUC mode is applied, the inter prediction unit 218 performs motion estimation in accordance with the pattern matching method (bidirectional matching or template matching) that has been interpreted from the encoded stream, thereby Export mobile information. Then, the inter prediction unit 218 performs motion compensation using the derived movement information.

Further, when the BIO mode is applied, the inter prediction unit 218 derives a motion vector based on a model of a hypothetical linear motion. Further, when the information display that has been interpreted from the coded bit stream indicates that the affine motion compensation prediction mode is applied, the inter prediction unit 218 derives the motion vector in units of sub-blocks based on the motion vectors of the plurality of adjacent blocks. [Predictive Control Department]

The prediction control unit 220 selects any one of the intra prediction signal and the inter prediction signal, and outputs the selected signal as a prediction signal to the addition unit 208. [Deblocking Filter Processing]

Next, the deblocking filtering process performed by the encoding device 100 and the decoding device 200 configured as above will be specifically described with reference to the drawings. In the following, the operation of the loop filter unit 120 included in the encoding device 100 will be mainly described, but the operation of the loop filter unit 212 included in the decoding device 200 is also the same.

As described above, the encoding apparatus 100 calculates the prediction error by subtracting the prediction signal generated by the intra prediction unit 124 or the inter prediction unit 126 from the original signal when encoding the image. The encoding device 100 performs orthogonal conversion processing, quantization processing, and the like on the prediction error to generate quantized coefficients. Furthermore, the encoding apparatus 100 inversely quantizes and inversely orthogonalizes the obtained quantized coefficients to restore the prediction error. Here, the quantization process is an irreversible process, and thus the restored prediction error has an error (quantization error) with respect to the prediction error before conversion.

The deblocking filtering process performed by the loop filter unit 120 is one of filtering processes performed for the purpose of reducing the quantization error or the like. The deblocking filtering process is applied to block boundaries in order to remove block noise. In addition, the deblocking filter processing is also simply referred to as filtering processing hereinafter.

FIG. 11 is a flowchart showing an example of the deblocking filtering process performed by the loop filter unit 120. For example, the processing shown in Fig. 11 is performed at each block boundary.

First, the loop filter unit 120 calculates the block boundary strength (Bs) in order to determine the behavior of the deblocking process (S101). Specifically, the loop filter unit 120 determines Bs by using the prediction mode of the block to be filtered or the nature of the motion vector. For example, if at least one of the blocks of the clip boundary is an intra prediction block, it is set to Bs=2. Further, if the block satisfying at least one of the blocks of the so-called (1) clip boundary is an orthogonal transform coefficient having an advantage, and (2) the difference between the motion vectors of the two blocks of the clip boundary is a threshold value or more, And (3) when the number of movement vectors of the two blocks of the clip block boundary or the reference image differs from at least one of the conditions (1) to (3), it is set to Bs=1. If none of the conditions (1) to (3) are met, it is set to Bs=0.

Next, the loop filter unit 120 determines whether or not the set Bs is larger than the first threshold (S102). When Bs is equal to or less than the first threshold (No in S102), the loop filter unit 120 does not perform filtering processing (S107).

On the other hand, when the set Bs is larger than the first threshold (Yes in S102), the loop filter unit 120 calculates the pixel variation d of the boundary region using the pixel values in the blocks on both sides of the block boundary ( S103). This processing will be described using FIG. When the pixel value of the block boundary is defined as shown in FIG. 12, the loop filter unit 120 calculates, for example, d=|p30-2×p20+p10|+|p33-2×p23+p13|+|q30-2×q20+q10|+|q33-2 ×q23+q13|.

Next, the loop filter unit 120 determines whether or not the calculated d is larger than the second threshold (S104). When d is equal to or less than the second threshold (No in S104), the loop filter unit 120 does not perform filtering processing (S107). Further, the first threshold is different from the second threshold.

When the calculated d is larger than the second threshold (Yes in S104), the loop filter unit 120 determines the filter characteristics (S105), and performs filtering processing of the determined filter characteristics (S106). For example, a filter of five taps of (1, 2, 2, 2, 1) / 8 is used. That is, for p10 shown in Fig. 12, an operation of (1 × p30 + 2 × p20 + 2 × p10 + 2 × q10 + 1 × q20) / 8 is performed. Here, at the time of the filtering process, in order to prevent excessive smoothing, a clipping process is performed, and the positional movement is converged within a certain range. The trimming process referred to herein means that, for example, when the threshold value of the trimming process is tc and the pixel value before filtering is q, the filtered pixel value can only be processed by the threshold value of the range of q±tc.

Hereinafter, in the deblocking filtering process performed by the loop filter unit 120 of the present embodiment, an example in which a filter having an asymmetric block boundary is applied will be described.

Fig. 13 is a flow chart showing an example of the deblocking filter processing of the embodiment. Further, the processing shown in FIG. 13 may be performed for each block boundary, or may be performed for each unit pixel including one or more pixels.

First, the loop filter unit 120 acquires the encoding parameters, and determines the filter characteristics in which the clip boundary is asymmetric using the acquired encoding parameters (S111). In the present disclosure, the acquired coding parameters are, for example, assigned error distribution features.

Here, the filter characteristics refer to a filter coefficient, a parameter used for filter processing control, and the like. Further, the encoding parameter may be any one as long as it is a parameter that can be used in the determination of the filter characteristics. The coding parameter may also be information showing the error itself, or information or parameters related to the error (for example, the magnitude relationship of the left and right errors).

Further, the following is a pixel which is determined to have a large or small error according to the encoding parameter, that is, a pixel having a high possibility of being large or small is also abbreviated as a pixel having a large error or small.

Here, it is not necessary to perform the determination processing every time, and the processing may be performed in accordance with a rule that has been previously set and that associates the encoding parameter with the filter characteristic.

On the other hand, in a statistical manner, even in the case of a pixel having a high possibility of small error, the error may become larger than the error of a pixel having a high possibility of large error when viewed in each pixel.

Next, the loop filter unit 120 performs filter processing having the determined filter characteristics (S112).

Here, the filter characteristics determined in step S111 are not necessarily asymmetrical, and a symmetrical design can also be achieved. In addition, a filter having a filter characteristic in which the boundary of the block is asymmetric is also referred to as an asymmetric filter, and a filter having a filter characteristic in which the boundary of the block is symmetric is also referred to as a symmetric filter.

Specifically, in order to make the pixel determined to be small as an error, it is difficult to be affected by a pixel having a large surrounding error, and it is easy to receive a small error in order to make the pixel determined to be large in error. The effect of the pixel is 2 points to determine the filter characteristics. In other words, the filter characteristics are determined such that the larger the error, the larger the influence of the filtering process is. For example, the filter characteristics are determined such that the amount of change in the pixel values before and after the filtering process is increased by the pixel having the larger error. As a result, the value of the pixel having a high possibility of small error is greatly changed, thereby preventing the true value from being removed. On the other hand, it is possible to reduce the error by making the value greatly affected by the pixel having a small error for a pixel having a high possibility of large error.

In addition, the elements of the displacement of the variation filter are defined as the weights of the filters below. In other words, the weight is the extent to which the effect of the filtering process on the symmetric pixel is displayed. Enlarging the weight means increasing the influence of the filtering process on the pixel. In other words, it is said that the filtered pixel value is easily affected by other pixels. Specifically, enlarging the weight means that the amount of change in the pixel value before and after the filtering process is increased, or the filter characteristic is determined in order to facilitate the filtering process.

That is, the loop filter unit 120 enlarges the weight more for the pixel having the larger error. Further, the pixel having a larger error increases the weight more, and is not limited to the case where the weight is continuously changed according to the error, and includes the case where the weight is changed stepwise. In other words, the weight of the first pixel may be smaller than the weight of the second pixel whose error is larger than the first pixel. Also, the following is the same performance.

In addition, in the finally determined filter characteristics, it is not necessary to have a pixel with a large error, and the weight is larger. In other words, the loop filter unit 120 may correct the filter characteristics determined to be the reference by a conventional method, and the weight of the pixel having a larger error may be corrected.

Hereinafter, a plurality of methods for changing the specificity of the weight asymmetrically will be described. Alternatively, any of the following methods may be employed, or a combination of a plurality of methods may be employed.

As a first method, the loop filter unit 120 reduces the filter coefficient to a pixel having a larger error. For example, the loop filter unit 120 reduces the filter coefficient of the pixel having a large error, and amplifies the filter coefficient of the pixel having a small error.

For example, an example of the deblocking filtering process performed on the pixel p1 shown in FIG. 12 will be described. Hereinafter, this technique will not be applied, but for example, a filter determined by a conventional method will be referred to as a reference filter. The reference filter is a filter of 5 taps perpendicular to the block boundary and is made to extend across (p3, p2, p1, q1, q2). Also, let the filter coefficient be (1, 2, 2, 2, 1) / 8. Further, the possibility that the error of the block P is large is high, and the possibility that the error of the block Q is small is high. At this time, the filter coefficient is set so that the block P having a large error is easily affected by the block Q having a small error. Specifically, it is set such that the filter coefficient of the pixel used in the small error is large, and the filter coefficient of the pixel used in the error is set to be small. For example, in terms of filter coefficients, (0.5, 1.0, 1.0, 2.0, 1.5) / 6 is used.

In another example, 0 can also be used for the filter coefficients of pixels with small errors. For example, in terms of filter coefficients, (0, 0, 1, 2, 2) /5 can also be used. That is, the filter tap can also be changed. Conversely, the filter coefficient that is now 0 may be set to a value other than zero. For example, in terms of filter coefficients, (1, 2, 2, 2, 1, 1) / 9, etc. can also be used. That is, the loop filter unit 120 can also extend the filter tap to the side where the error is small.

Further, the reference filter is the same as (1, 2, 2, 2, 1)/8 described above, but it may not be a filter that is bilaterally symmetrical about the target pixel. In this case, the loop filter unit 120 further adjusts the filter. For example, the filter coefficient of the reference filter used at the pixel at the left end of the Q block is (1, 2, 3, 4, 5) / 15, and the filter of the reference filter used at the pixel at the right end of the block P is filtered. The coefficient is (5, 4, 3, 2, 1) / 15. That is, at this time, the filter coefficients that have been reversed left and right are used between the pixels at the boundary of the block block. A filter characteristic in which the boundary of the block is reversed symmetrically can also be referred to as a "symmetric filter characteristic of the boundary of the clip block". That is, the filter characteristic with the asymmetrical block boundary is a filter characteristic in which the non-sandwich block boundary is inversely symmetric.

Further, similarly to the above, when the error of the block P is large and the error of the block Q is small, the loop filter unit 120 changes, for example, (5, 4, 3, 2, 1)/15 to (2.5, 2.0). , 1.5, 2.0, 1.0) / 9, where (5, 4, 3, 2, 1) / 15 is the filter coefficient of the reference filter using the pixel at the right end of the block P.

Thus, in the deblocking filtering process, a filter whose filter coefficient is asymmetrically changed with the block boundary is used. For example, the loop filter unit 120 determines a reference filter having a filter characteristic in which the boundary of the clip block is symmetrical in accordance with a predetermined criterion. The loop filter unit 120 changes the reference filter to have a filter characteristic in which the boundary of the clip block is asymmetric. Specifically, the loop filter unit 120 performs at least one of amplifying the filter coefficients of at least one pixel having a small error among the filter coefficients of the reference filter and narrowing down the filter coefficients of at least one pixel having a large error.

Next, the second method of asymmetrically changing the weight will be described. First, the loop filter unit 120 performs a filter operation using a reference filter. Next, the loop filter unit 120 performs asymmetric weighting on the block boundary value Δ0 which is the amount of change in the pixel value before and after the operation of the filter using the reference filter. In addition, hereinafter, for the sake of distinction, a process using a reference filter is referred to as a filter operation, and a series of processes including a filter operation and subsequent correction processing (for example, asymmetric weighting) is referred to as filter processing (deblocking filter processing). ).

For example, the loop filter unit 120 calculates a corrected amount of change Δ1 by multiplying the reference change amount Δ0 by a factor smaller than 1 for a pixel having a small error. Further, the loop filter unit 120 calculates a corrected amount of change Δ1 by multiplying the reference change amount Δ0 by a factor larger than 1 for a pixel having a large error. Next, the loop filter unit 120 adds the corrected amount of change Δ1 to the pixel value before the filter operation, thereby generating the pixel value after the filter processing. Further, the loop filter unit 120 may perform only one of processing for processing a pixel having a small error and processing for a pixel having a large error.

For example, as described above, the error of the block P is made large, and the error of the block Q is small. At this time, the loop filter unit 120 calculates the amount of change Δ1 after the correction by, for example, changing the reference change amount Δ0 by 0.8 times for the pixel included in the block Q having a small error. Further, the loop filter unit 120 calculates the amount of change Δ1 after correction by, for example, changing the reference change amount Δ0 by 1.2 times for the pixel included in the block P having a large error. In this way, the variation of the value of the pixel having a small error can be reduced. Moreover, the variation of the value of the pixel having a large error can be amplified.

Further, there is also a case where 1:1 is selected as a ratio of a coefficient multiplied to the reference change amount Δ0 of the pixel having a small error to a coefficient multiplied by the reference change amount Δ0 of the pixel having a large error. At this point, the filter characteristics are symmetric of the boundary of the clip block.

Further, the loop filter unit 120 can calculate the coefficient multiplied by the reference change amount Δ0 by multiplying the constant by the reference coefficient. At this time, the loop filter unit 120 uses a constant larger than a pixel having a small error for a pixel having a large error. As a result, the amount of change in the pixel value of the pixel having a large error increases, and the amount of change in the pixel value of the pixel which is highly likely to be small in error is reduced. For example, the loop filter unit 120 uses 1.2 or 0.8 as a constant for pixels adjacent to the block boundary, and 1.1 or 0.9 as a constant for pixels that are separated from one pixel adjacent to the block boundary. Further, the reference coefficient is obtained, for example, by (A × (q1 - p1) - B × (q2 - p2) + C) / D. Here, A, B, C, and D are constants. For example, A=9, B=3, C=8, and D=16. Further, p1, p2, q1, and q2 are pixel values of pixels in the positional relationship shown in FIG. 12 with respect to the block boundary.

Next, the third method of asymmetrically changing the weight will be described. Similarly to the second method, the loop filter unit 120 performs a filter operation using filter coefficients of the reference filter. Next, the loop filter unit 120 adds an offset value to the pixel value after the filter operation by adding the clip boundary. Specifically, the loop filter unit 120 is a method in which the value of the pixel having a large error is close to the value of the pixel having a high error, and the displacement of the pixel having the large error is increased. The pixel value is plus a positive offset value. Further, the loop filter unit 120 adds a value of a pixel having a small error to a value of a pixel having a large error, and reduces a displacement of a pixel having a small error, and adds a negative pixel value to a pixel having a small error. Offset value. As a result, the amount of change in the pixel value of the pixel having a large error is increased, and the amount of change in the pixel value of the pixel having a small error is reduced. Further, the loop filter unit 120 may perform only one of processing for a pixel having a small error and processing for a pixel having a large error.

For example, the loop filter unit 120 calculates a corrected amount of change Δ1 by adding a positive offset value (for example, 1) to the absolute value of the reference change amount Δ0 for the pixel included in the block having a large error. Further, the loop filter unit 120 calculates a corrected amount of change Δ1 by adding a negative offset value (for example, -1) to the absolute value of the reference change amount Δ0 for the pixel included in the block having a small error. Next, the loop filter unit 120 adds the corrected amount of change Δ1 to the pixel value before the filter operation, thereby generating the pixel value after the filter processing. Further, the loop filter unit 120 may not add an offset value to the amount of change, but may add an offset value to the pixel value after the filter operation. Also, the offset value may not be symmetric with respect to the boundary of the clip block.

Further, when the filter taps extend across a plurality of pixels from the block boundary, the loop filter unit 120 may change the weight for a specific pixel or may change the weight for all the pixels. Further, the loop filter unit 120 may change the weight in accordance with the distance from the block boundary to the target pixel. For example, the loop filter unit 120 may also set the filter coefficient associated with the correlation between the block boundary and the two pixels to be asymmetric, and set the filter coefficient associated with the pixel after that to be symmetric. Further, the weight of the filter may be common to a plurality of pixels or may be set for each pixel.

Next, the fourth method of asymmetrically changing the weight will be described. The loop filter unit 120 performs a filter operation using filter coefficients of the reference filter. Next, when the amount of change Δ of the pixel value before and after the filtering operation exceeds the clipping width of the reference value, the loop filter unit 120 cuts the amount of change Δ into the clipping width. The loop filter unit 120 sets the clipping width asymmetrically with respect to the block boundary.

Specifically, the loop filter unit 120 amplifies the clipping width of the pixel having a large error so as to be larger than the clipping width of the pixel having a small error. For example, the loop filter unit 120 sets the clipping width of the pixel having a large error to a constant multiple of the clipping width of the pixel having the small error. As a result of the change in the crop width, the value of the pixel having a small error cannot be greatly changed. Moreover, the value of a pixel having a large error can be greatly changed.

Further, the loop filter unit 120 may adjust the absolute value of the trim width without specifying the ratio of the trim width. For example, the loop filter unit 120 fixes the clipping width of the pixel having a large error to a multiple of the predetermined clipping width that has been previously set. The loop filter unit 120 sets the ratio of the clipping width of the pixel having a small error to the clipping width of the pixel having a small error to 1.2:0.8. Specifically, for example, the reference crop width is set to 10, and the amount of change Δ before and after the filter operation is 12. At this time, when the reference trim width is used invariably, the amount of change Δ is corrected to 10 by the threshold processing. On the other hand, when the target pixel is a pixel having a large error, for example, the reference trim width becomes 1.5 times. Thereby, since the trimming width becomes 15, the threshold processing is not performed, and the amount of change Δ becomes 12.

Next, the fifth method of asymmetrically changing the weight will be described. The loop filter unit 102 sets a condition that the clip boundary is asymmetrically set to determine whether or not to perform filtering processing. Here, the condition for determining whether or not to perform the filtering process is, for example, the first threshold or the second threshold shown in FIG. 11 .

Specifically, the loop filter unit 120 sets the conditions so that the pixels having large errors are easily subjected to the filtering process, and sets the conditions so that it is difficult to perform filtering processing on the pixels having small errors. For example, the loop filter unit 120 increases the threshold of the pixel having a small error to be larger than the threshold of the pixel having a large error. For example, the loop filter unit 120 sets the threshold value of the pixel having a small error to a constant multiple of the threshold value of the pixel having a large error.

Further, the loop filter unit 120 can adjust the absolute value of the threshold value not only by specifying the ratio of the threshold values. For example, the loop filter unit 120 may fix the threshold value of the pixel having a small error to a multiple of the predetermined reference threshold value, and set the ratio of the threshold value of the pixel having the small error to the threshold value of the pixel having the large error to 1.2:0.8.

Specifically, the reference threshold of the second threshold in step 104 is set to 10, and d calculated from the pixel values in the block is 12. When the reference threshold value is used as the second threshold value, it is determined that the filtering process is performed. On the other hand, when the target pixel is a pixel having a small error, for example, a value obtained by amplifying the reference threshold by a factor of 1.5 is used as the second threshold. At this time, the second threshold value becomes 15 and becomes larger than d. Thereby, it is determined that the filtering process is not performed.

Further, the constant for displaying the weight or the like may be a value that has been previously predetermined in the encoding device 100 and the decoding device 200, and may be variable, wherein the weight is based on the error used in the first to fifth methods described above. Specifically, the constant is a coefficient multiplied by the filter coefficient in the first method or the filter coefficient of the reference filter, a coefficient multiplied to the reference change amount Δ0 in the second method, or a constant multiplied to the reference coefficient, The offset value in the third method, the constant of the clipping width in the fourth method or the reference clipping width, and the constant multiplied to the threshold value in the fifth method or the reference threshold.

The information indicating the constant may be displayed when the constant is variable, for example, as a parameter of a sequence or a slice unit, and included in the bit stream, and transmitted from the encoding device 100 to the decoding device 200. In addition, the information indicating the constant may be information showing the constant itself, or information showing the ratio or difference between the reference value and the reference value.

Further, a method of changing a coefficient or a constant in response to an error includes, for example, a method of linearly changing, a method of changing a quadratic function, a method of changing an exponential function, or a lookup table using a relationship between display error and a constant. Method etc.

Further, when the error is equal to or higher than the reference, or when the error is below the reference, a fixed value may be used as a constant. For example, the loop filter unit 120 may set the variable to the first value when the error is equal to or less than the predetermined range, and set the variable to the second value when the error is equal to or greater than the predetermined range, and the error is within the predetermined range. The variable is continuously changed from the first value to the second value in response to the error.

Further, when the error exceeds a predetermined reference, the loop filter unit 120 may use a symmetric filter (reference filter) instead of the asymmetric filter.

Further, when the lookup table or the like is used, the loop filter unit 120 may retain a table having both a large error and a small case, or may retain only one of the tables, and the contents of the table are determined in advance. The rule to calculate the constant on the other side.

As described above, the encoding apparatus 100 and the decoding apparatus 200 of the present embodiment use an asymmetric filter, whereby the error of the reconstructed image can be reduced, so that the encoding efficiency can be improved.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (Embodiment 2)

In the second embodiment to the sixth embodiment, a specific example of the encoding parameter to which the error distribution feature described above is given will be described. In the present embodiment, the loop filter unit 120 determines the filter characteristics in accordance with the position in the block of the target pixel.

Fig. 14 is a flow chart showing an example of the deblocking filter processing of the embodiment. First, the loop filter unit 120 acquires information on the position in the block of the display target pixel as an encoding parameter that gives an error distribution characteristic. The loop filter unit 120 determines the filter characteristics in which the slab block boundary is asymmetric based on the position (S121).

Next, the loop filter unit 120 performs a filter process having the determined filter characteristics (S122).

Here, the pixel farther from the intra-predicted reference pixel is more likely to have a larger error than the pixel near the intra-predicted reference pixel. Therefore, the loop filter unit 120 determines the filter characteristics such that the pixel is farther from the intra-predicted reference pixel and the amount of change in the pixel values before and after the filter processing becomes larger.

For example, in H.265/HEVC or JEM, as shown in FIG. 15, a pixel that is closer to a reference pixel refers to a pixel that exists in the upper left of the block, and a pixel that is closer to the reference pixel means that it exists in the block. The bottom right pixel. Accordingly, the loop filter unit 120 determines the filter characteristics such that the weight of the lower right pixel in the block is greater than the weight of the upper left pixel.

Specifically, the loop filter unit 120 determines the filter characteristics such that the influence of the filtering process is increased as described in the first embodiment with respect to the pixels farther from the intra-predicted reference pixels. That is, the loop filter unit 120 amplifies the weight of the pixel farther from the intra-predicted reference pixel. Here, enlarging the weight means that, as described above, at least one of the following is implemented: (1) reducing the filter coefficient, and (2) filtering the pixel at the boundary (ie, the pixel near the intra-predicted reference pixel). The coefficient is amplified, (3) the coefficient multiplied by the amount of change is amplified, (4) the offset value of the amount of change is amplified, (5) the clipping width is enlarged, and (6) the threshold is corrected to make the filtering process easy to perform. On the other hand, the loop filter unit 120 determines the filter characteristics such that the influence of the filtering process is reduced for the pixels near the reference pixels that are out of the inner prediction. That is, the loop filter unit 120 reduces the weight of the pixel that is close to the intra-predicted reference pixel. Here, reducing the weight means that, as described above, at least one of the following is implemented: (1) amplifying the filter coefficient, and (2) a pixel having a boundary (ie, a pixel close to the intra-predicted reference pixel) The filter coefficient is reduced, (3) the coefficient multiplied by the amount of change is reduced, (4) the offset value of the amount of change is reduced, (5) the trimming width is reduced, and (6) the threshold is corrected to make the filtering process difficult to perform.

Alternatively, when the intra prediction is used, the above processing may be performed, and the above processing may not be performed on the block using the inter prediction. However, the nature of the intra-predicted block is sometimes referred to the inter-prediction, so the inter-predictive block can also be processed as described above.

Further, the loop filter unit 120 may arbitrarily designate a position within a specific block and change the weight. For example, the loop filter unit 120 may enlarge the weight of the lower right pixel in the block as described above, and reduce the weight of the upper left pixel in the block. Further, the loop filter unit 120 is not limited to the upper left and lower right, and may arbitrarily specify the position in the block and change the weight.

Further, as shown in FIG. 15, in the adjacent block boundary in the left-right direction, the error of the block on the left side becomes large, and the error of the block on the right side becomes large. Therefore, the loop filter unit 120 may enlarge the weight of the block on the left side and the weight of the block on the right side with respect to the adjacent block boundary in the left-right direction.

Similarly, in the adjacent block boundary in the up and down direction, the error of the upper block becomes larger, and the error of the lower block becomes smaller. Therefore, the loop filter unit 120 may enlarge the weight of the upper block and reduce the weight of the lower block with respect to the adjacent block boundary in the vertical direction.

Further, the loop filter unit 120 may change the weight in accordance with the distance from the reference pixel of the intra prediction. Further, the loop filter unit 120 may determine the weights in units of block boundaries, or may determine the weights in units of pixels. The farther away from the reference pixel, the easier the error becomes. Therefore, the loop filter unit 120 determines the filter characteristics such that the further the distance from the reference pixel is, the steeper the gradient of the weight becomes. Further, the loop filter unit 120 determines the filter characteristics such that the gradient of the weight in the upper side on the right side of the block is more moderate than the gradient of the weight in the lower side.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (Embodiment 3)

In the present embodiment, the loop filter unit 120 determines the filter characteristics in response to the orthogonal conversion base.

Fig. 16 is a flow chart showing an example of the deblocking filter processing of the embodiment. First, the loop filter unit 120 acquires information as an encoding parameter assigned to the error distribution feature, wherein the information displays the orthogonal conversion substrate used by the target block. The loop filter unit 120 determines a filter characteristic in which the boundary of the clip block is asymmetric based on the orthogonal transform base (S131).

Next, the loop filter unit 120 performs filter processing having the determined filter characteristics (S132).

The encoding device 100 selects one orthogonal conversion substrate from among a plurality of candidates, which is a conversion substrate when performing orthogonal conversion. A plurality of candidates, for example, a conversion substrate including 0 times of DCT-II or the like is a flat substrate, and a conversion substrate of 0 times such as DST-VII is not a flat substrate. Figure 17 is a diagram showing a conversion substrate of DCT-II. Figure 18 is a diagram showing a conversion substrate of DCT-VII.

The 0-time base of DCT-II is constant regardless of the position within the block. That is, when the DCT-II is used, the error in the block is fixed. Therefore, the loop filter unit 120 performs filtering processing using a symmetric filter when the blocks on both sides of the clip boundary are converted by DCT-II without using an asymmetric filter.

On the other hand, the 0-time base of DST-VII is the larger the value as the distance from the boundary of the block on the left or above becomes larger. That is, as the distance from the left or upper block boundary is larger, the possibility that the error becomes larger becomes higher. Therefore, the loop filter unit 120 uses an asymmetric filter when at least one of the two blocks at the boundary of the block block is converted by DST-VII. Specifically, the loop filter unit 120 determines the filter characteristics such that the smaller the value in the lower base (for example, 0 times), the smaller the influence of the filtering process.

Specifically, when the blocks on both sides of the boundary of the splicing block are converted by DST-VII, the loop filter unit 120 performs the filtering process on the pixels on the lower right side in the block by the above-described method. The way to increase the filter characteristics. Further, the loop filter unit 120 determines the filter characteristics such that the influence of the filtering process is increased for the upper left pixel in the block.

Further, when DST-VII and DCT-II are vertically adjacent to each other, the loop filter unit 120 also determines the filter characteristics, that is, the pixels of the lower portion of the upper block used in the DST-VII adjacent to the block boundary. The weight of the filter is greater than the weight of the filter of the upper pixel of the lower block used by the DCT-II. However, the difference in amplitude of the low-order base at this time is smaller than the difference in the amplitude of the low-order base when DST-VII is adjacent to each other. Therefore, the loop filter unit 120 sets the filter characteristics such that the gradient of the weight at this time is smaller than the gradient of the weight when the DST-VII is adjacent to each other. The loop filter unit 120 is, for example, set the weight when DCT-II is adjacent to DCT-II to 1:1 (symmetric filter), and sets the weight when DST-VII is adjacent to DST-VII to 1.3:0.7, and DST The weight when -VII is adjacent to DCT-II is set to 1.2:0.8.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (Embodiment 4)

In the present embodiment, the loop filter unit 120 determines the filter characteristics in accordance with the pixel values of the boundary of the block.

Fig. 19 is a flow chart showing an example of the deblocking filter processing of the embodiment. First, the loop filter unit 120 obtains a piece of information as an encoding parameter assigned to the error distribution feature, wherein the information shows the pixel value within the block of the clip block boundary. The loop filter unit 120 determines a filter characteristic in which the boundary of the clip block is asymmetric based on the pixel value (S141).

Next, the loop filter unit 120 performs a filter process having the determined filter characteristics (S142).

For example, the loop filter unit 120 increases the difference in filter characteristics of the block boundary as the pixel value difference d0 increases. Specifically, the loop filter unit 120 determines the filter characteristics such that the difference in the influence of the filtering process becomes large. For example, when the loop filter unit 120 satisfies d0>(quantization parameter)×(constant), the weight is set to 1.4:0.6, and when the above relationship is not satisfied, the weight is set to 1.2:0.8. That is, the loop filter unit 120 compares the difference d0 of the pixel values and the threshold based on the quantization parameter, and when the difference d0 of the pixel values is larger than the threshold value, the difference of the filter characteristics of the clip boundary is amplified by the difference d0 of the pixel value. Greater when it is less than the threshold.

In another example, for example, the larger the average value b0 of the dispersion of the pixel values in the two blocks of the block block boundary is, the larger the difference in the filter characteristics of the block boundary is. Specifically, the loop filter unit 120 may determine the filter characteristics such that the difference in the influence of the filtering process becomes large. For example, when the loop filter unit 120 satisfies b0>(quantization parameter)×(constant), the weight is set to 1.4:0.6, and when the above relationship is not satisfied, the weight is set to 1.2:0.8. That is, the loop filter unit 120 compares the dispersion b0 of the pixel values and the threshold based on the quantization parameter, and when the dispersion b0 of the pixel values is larger than the threshold, the difference in the filter characteristics of the clip boundary is amplified as the dispersion b0 of the pixel values. Greater when it is less than the threshold.

In addition, among the adjacent blocks, which of the blocks is to be enlarged in weight, that is, which block has a larger error, the method of the above-described Embodiment 2 or 3, or the embodiment described later The technique of 6 is specific. That is, the loop filter unit 120 determines the filter characteristics in which the boundary of the clip block is asymmetric in accordance with a predetermined rule (for example, the method of the second, third, or sixth embodiments). Next, the loop filter unit 120 changes the determined filter characteristics such that the difference in filter characteristics at the block boundary is increased in accordance with the difference d0 of the pixel values. In other words, the loop filter unit 120 amplifies the ratio or difference between the weight of the pixel having a large error and the weight of the pixel having a small error.

Here, when the difference d0 of the pixel values is large, there is a possibility that the block boundary coincides with the edge of the object in the image, and therefore, in this case, the difference in filter characteristics of the block boundary is reduced to This can suppress the progress of unnecessary smoothing.

Further, the loop filter unit 120 may be reversed from the above, and the larger the difference d0 between the pixel values, the smaller the difference in filter characteristics at the block boundary. Specifically, the loop filter unit 120 determines the filter characteristics such that the difference in the influence of the filtering process is reduced. For example, when the loop filter unit 120 satisfies d0>(quantization parameter)×(constant), the weight is set to 1.2:0.8, and if the above relationship is not satisfied, the weight is set to 1.4:0.6. In addition, when the above relationship is satisfied, the weight can also be set to 1:1 (symmetric filter). That is, the loop filter unit 120 compares the difference d0 between the pixel values and the threshold value based on the quantization parameter, and when the difference d0 between the pixel values is larger than the threshold value, the filter characteristic of the block boundary is smaller than the difference d0 between the pixel values and the threshold value. The difference is narrowed.

For example, the difference d0 of the pixel value means that the block boundary is conspicuous, so in such an case, the difference in filter characteristics of the block boundary is reduced, whereby the smoothing can be suppressed by the asymmetric filter. Weakened situation.

Alternatively, the two treatments can be performed simultaneously. For example, the loop filter unit 120 may use the first weight when the difference d0 of the pixel values is less than the first threshold, and may use the difference when the difference d0 of the pixel values is equal to or larger than the first threshold and less than the second threshold. When the difference d0 of the pixel value is equal to or greater than the second threshold, the third weight having a smaller weight than the second weight is used.

Further, the difference d0 of the pixel values may also make the difference of the pixel values of the clip boundary itself, or may be the average or dispersion of the difference of the pixel values. For example, the difference d0 of the pixel values is obtained by (A × (q1 - p1) - B × (q2 - p2) + C) / D). Here, A, B, C, and D are constants. For example, A=9, B=3, C=8, D=16. Further, p1, p2, q1, and q2 are pixel values of pixels in the positional relationship shown in FIG. 12 of the block boundary.

Further, the difference d0 of the pixel values and the setting of the weight may be performed in units of pixels, or may be performed in a block boundary unit, or may be performed by a block group unit (for example, an LCU (Largest Coding Unit) unit of a plurality of blocks.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (Embodiment 5)

In the present embodiment, the loop filter unit 120 determines the filter characteristics in accordance with the intra prediction direction and the block boundary direction.

Fig. 20 is a flowchart showing an example of the deblocking filter processing of the embodiment. First, the loop filter unit 120 acquires information indicating the angle between the prediction direction of the intra prediction and the block boundary as an encoding parameter that gives an error distribution characteristic. The loop filter unit 120 determines a filter characteristic in which the slab block boundary is asymmetric based on the angle (S151).

Next, the loop filter unit 120 performs a filter process having the determined filter characteristics (S152).

Specifically, the loop filter unit 120 increases the difference in filter characteristics at the boundary of the block block as the angle is closer to the vertical direction, and the closer the angle is to the horizontal level, the smaller the difference in filter characteristics at the boundary of the block block is. . More specifically, the filter characteristics are determined in such a manner that, when the inner prediction direction is close to the vertical with respect to the block boundary, the difference in weight of the filter with respect to the pixels on both sides of the boundary of the slab block is made. When the inner prediction direction is close to the horizontal with respect to the block boundary, the difference in weight of the filters for the pixels on both sides of the boundary of the slab block is made small. Fig. 21 is a diagram showing an example of a weight which is a relationship between an inner prediction direction and a direction of a block boundary.

In addition, among the adjacent blocks, which weight of the block is to be enlarged, that is, which block has a larger error, the method of Embodiment 2 or 3 described above, or an embodiment to be described later The technique of 6 is specific. That is, the loop filter unit 120 determines the filter characteristics in which the boundary of the clip block is asymmetric in accordance with a predetermined rule (for example, the method of the second, third, or sixth embodiments). Next, the loop filter unit 120 changes the filter characteristics of the clip boundary based on the direction of the intra prediction direction and the block boundary to change the determined filter characteristics.

Further, the encoding device 100 and the decoding device 200 specify the intra prediction direction using, for example, an intra prediction mode.

Further, when the intra prediction mode is a Planar mode or a DC mode, the loop filter unit 120 does not have to consider the direction of the block boundary. For example, when the inner prediction mode is the planar mode or the direct current mode, the loop filter unit 120 may use the difference of the weights or weights that have been previously set, regardless of the direction of the block boundary. Alternatively, the loop filter unit 120 may use a symmetric filter when the intra prediction mode is the planar mode or the direct current mode.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (Embodiment 6)

In the present embodiment, the loop filter unit 120 determines the filter characteristics in accordance with the quantization parameter indicating the width of the quantization.

Fig. 16 is a flowchart showing an example of the deblocking filter processing of the embodiment. First, the loop filter unit 120 acquires information as an encoding parameter that gives an error distribution characteristic, wherein the information displays a quantization parameter used in quantization of the target block. The loop filter unit 120 determines a filter characteristic in which the clip boundary is asymmetric based on the quantization parameter (S161).

Next, the loop filter unit 120 performs filter processing having the determined filter characteristics (S162).

Here, the larger the quantization parameter, the higher the possibility that the error becomes larger. Therefore, the loop filter unit 120 determines the filter characteristics such that the larger the quantization parameter is, the larger the influence of the filtering process becomes.

FIG. 23 is a diagram showing an example of weights for quantization parameters. As shown in FIG. 23, the loop filter section 120 increases the weight of the pixel on the upper left of the tile as the quantization parameter increases. On the other hand, the loop filter unit 120 is an increase in the weight of the pixel to be reduced to the lower right in the block, and the increase of the weight is an increase with the quantization parameter. That is, the loop filter unit 120 determines the filter characteristics in such a manner that the change in the influence of the filtering process with the change of the quantization parameter of the upper left pixel is larger than the filter processing in accordance with the change of the quantization parameter of the upper right pixel. The impact of the change.

Here, the upper left pixel in the block is more susceptible to quantization parameters than the lower right pixel in the block. Therefore, the processing as described above is performed, whereby the error can be appropriately reduced.

Moreover, the loop filter unit 120 may determine the weight of the block based on the quantization parameter of the block for each of the two blocks with the boundary, and may also calculate the average value of the quantization parameters of the two blocks, according to The average is used to determine the weight of the two blocks. Alternatively, the loop filter unit 120 may determine the weights of the two blocks based on the quantization parameter of one of the blocks. For example, the loop filter unit 120 determines the weight of one of the blocks based on the quantization parameter of one of the blocks using the above method. Next, the loop filter unit 120 determines the weight of another block in accordance with the predetermined weight based on the determined weight.

Further, the loop filter unit 120 may use a symmetric filter when the quantization parameters of the two blocks are different or when the difference between the quantization parameters of the two blocks exceeds the threshold.

Further, in FIG. 23, the weight is set using a linear function, but an arbitrary function other than the primary function or a table may be used. For example, a curve showing the relationship between the quantization parameter and the quantization step (quantization width) can also be used.

Further, when the quantization parameter exceeds the threshold value, the loop filter unit 120 may use a symmetric filter instead of an asymmetric filter.

Further, when the quantization parameter is described as the decimal point precision, the loop filter unit 120 may perform rounding, rounding, or rounding off the quantization parameter, and apply the calculated quantization parameter to the above processing. Alternatively, the loop filter unit 120 may perform the above-described processing in consideration of the calculation to the decimal point unit.

As described above, in the second to sixth embodiments, the plurality of methods for determining the error have been individually described, but two or more of the methods may be composed. At this time, the loop filter unit 120 may weight the two or more elements that have been combined.

Modifications will be described below.

It can also be used in addition to the examples of the coding parameters described above. For example, the encoding parameters may also be the type of display orthogonal transform (Wavelet, DFT or repeated conversion, etc.), the block size (the width and height of the block), the direction of the motion vector, the length of the motion vector, or the use of inter-prediction. Information on the number of reference pictures and the characteristics of the reference filter. Also, information such as these can be used in combination. For example, the loop filter unit 120 may use an asymmetric filter only when the length of the block boundary is 16 pixels or less and the filter target pixel is closer to the intra-predicted reference pixel, and in other cases, the symmetry is adopted. filter. By way of another example, asymmetric processing may be performed only when a predetermined one of a plurality of filter candidates is used. For example, the positional shift obtained by the reference filter can be an asymmetrical filter only when it is calculated by transmission (A × (q1 - p1) - B × (q2 - p2) + C) / D. Here, A, B, C, and D are constant numbers. For example, A=9, B=3, C=8, and D=16. Further, p1, p2, q1, and q2 are pixel values of pixels in which the boundary of the block is in the positional relationship shown in FIG.

Further, the loop filter unit 120 may perform the above processing on one of the luminance signal and the color difference signal, or may perform the above processing on both. Further, the loop filter unit 120 can perform common processing on the luminance signal and the color difference signal, and can perform different processing. For example, the loop filter unit 120 may use different weights for the luminance signal and the color difference signal, or may determine the weight according to different rules.

Further, various parameters used in the above-described processing may be determined by the encoding device 100, or may be fixed values set in advance.

Further, the above processing may or may not be performed, or the content of the above processing may be switched in a predetermined unit. The predetermined unit refers to, for example, a slice unit, a block unit, a wavefront division unit, or a CTU unit. Further, the content of the above processing refers to a parameter for displaying any one of the plurality of methods described above, a weight, or the like, or a parameter for determining the same.

Further, the loop filter unit 120 may limit the area where the above processing is performed to the boundary of the CTU, the boundary of the slice, or the boundary of the block.

Also, in the symmetric filter and the asymmetric filter, the number of taps of the filter can be made different.

Further, the loop filter unit 120 may change the content of the above-described processing depending on the type of the frame (I frame, P frame, and B frame).

Further, the loop filter unit 120 may determine whether or not to perform the above processing or determine the content of the above processing depending on whether or not the specific processing of the previous or subsequent stages has been performed.

Further, the loop filter unit 120 may perform different processing depending on the type of prediction mode used by the block, or may perform the above processing only for the block used in the specific prediction mode. For example, the loop filter unit 120 may perform different processing from the block used for the intra prediction, the block used for the inter prediction, and the block that has been merged.

Moreover, the encoding device 100 may also encode the filtering information, wherein the filtering information is a parameter indicating whether to perform the above processing or the content of the processing. That is, the encoding device 100 can also generate a stream of encoded bits including filtered information. The filtering information may include information indicating whether the processing is performed, whether the color difference signal includes information indicating whether the processing is performed, or information indicating whether the display is different for each of the prediction modes.

Further, the decoding device 200 may perform the above processing based on the filter information included in the encoded bit stream. For example, the decoding device 200 may determine whether to perform the above processing or determine the content of the above processing based on the filtering information.

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, the processing described in a part of the flowchart of the present aspect, a part of the configuration of the device, a part of the syntax, and the like may be combined with other aspects. (Embodiment 7)

In each of the above embodiments, each of the functional blocks can be implemented by an MPU, a memory, or the like. Further, the processing performed by each of the function blocks can be realized by executing a software (program) recorded on a recording medium such as a ROM by a program execution unit such as a processor. The software may be distributed by downloading or the like, or may be recorded on a recording medium such as a semiconductor memory. In addition, it is of course also possible to implement each functional block through a hardware (dedicated circuit).

Further, the processing described in the respective embodiments may be realized by performing centralized processing using a single device (system), or may be realized by performing distributed processing using a plurality of devices. Moreover, the processor that executes the above program may be singular or plural. That is, the centralized processing may be performed or the dispersion processing may be performed.

The aspects of the present disclosure are not limited to the above embodiments, and various changes can be made, and variations thereof are also included in the scope of the present disclosure.

Further, an application example of the moving image encoding method (image encoding method) or the moving image decoding method (image decoding method) shown in each of the above embodiments and a system using the same will be described. This system is characterized by an image encoding device using an image encoding method, an image decoding device using an image decoding method, and an image encoding and decoding device having both. The other components in the system can be appropriately changed in accordance with the needs of the situation. [usage]

Fig. 24 is a view showing the overall configuration of a content supply system ex100 that realizes a content distribution service. The area of communication service is divided into desired sizes, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed wireless stations, are provided in each cell.

In the content supply system ex100, each of the machines such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smart phone ex115 is connected via the Internet service provider ex102 or the communication network ex104 and the base stations ex106 to ex110. Connected to the Internet ex101. The content supply system ex100 can be configured to be connected in combination with any of the above elements. It is also possible to connect the machines directly or indirectly via a telephone network or short-range wireless, without going through the base stations ex106 to ex110 of the fixed wireless station. Further, the streaming server ex103 is connected to each of the devices such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smartphone ex115 via the Internet ex101 or the like. Further, the streaming server ex103 is connected to a terminal or the like in the hot spot in the aircraft ex117 via the satellite ex116.

Alternatively, the base station ex106 to ex110 may be replaced by a wireless access point or a hot spot or the like. Further, the streaming server ex103 may be directly connected to the communication network ex104 without via the Internet ex101 or the Internet service provider ex102, or may be directly connected to the aircraft ex117 without via the satellite ex116.

The camera ex113 is a machine that can perform still image shooting and moving image shooting such as a digital camera. In addition, the smart phone ex115 generally refers to a smart phone, a mobile phone, or a PHS (Personal Handyphone System) corresponding to 2G, 3G, 3.9G, 4G, and a mobile communication system called 5G in the future.

The home appliance ex118 is a machine included in a refrigerator or a domestic fuel cell thermoelectric symbiosis system.

In the content supply system ex100, the terminal having the photographing function is connected to the streaming server ex103 via the base station ex106 or the like, thereby enabling live broadcast or the like. In the live broadcast, the terminal (computer ex111, game machine ex112, camera ex113, home appliance ex114, smart phone ex115, terminal in the aircraft ex117, etc.) transmits the data obtained as follows to the stream server ex103. The data is obtained by performing the encoding process described in the above embodiments on the still image or the moving image content captured by the user using the terminal device, and the image data obtained by the encoding and the image corresponding to the image. The voice-coded sound data is obtained by multiplexing. That is, each terminal functions as an image coding apparatus according to one aspect of the present disclosure.

On the other hand, the streaming server ex103 is to stream-distribute the content material to be transmitted to the client having the request. The client is a computer ex111, a game machine ex112, a camera ex113, a home appliance ex114, a smart phone ex115, or a terminal in the aircraft ex117, which can decode the encoded data. Each machine that has received the distributed data decodes the received data and plays it. That is, each device functions as an image decoding device of one aspect of the present disclosure. [Distributed processing]

Further, the stream server ex103 may be a plurality of servers or a plurality of computers, and the data may be distributed or recorded to the distributor. For example, the streaming server ex103 can also be implemented by a CDN (Contents Delivery Network) to realize content distribution by connecting a network between a plurality of edge servers and edge servers scattered around the world. In the CDN, dynamically approaching edge servers are dynamically allocated in response to the client. The content is then cached and distributed by the edge server, which reduces latency. Moreover, when any error occurs or the communication state is changed due to an increase in traffic, etc., a plurality of edge servers may be distributed, or the distribution body may be switched to another edge server, and the network portion where the obstacle has occurred may be performed. It is roundabout to continue the distribution, so high-speed and stable distribution can be achieved.

Further, not only the distribution processing of the distribution itself, but also the encoding processing of the photographed data may be performed at each terminal, or may be performed on the server side or may be performed separately. For example, in the encoding process, the processing loop is generally performed twice. The first cycle detects the complexity or amount of code in the frame or scene unit. In addition, in the second cycle, processing for maintaining image quality and improving coding efficiency is performed. For example, the terminal device performs the first encoding process, and the server side that has received the content performs the second encoding process, thereby reducing the processing load on each terminal device and improving the quality and efficiency of the content. . In this case, if there is a request to be decoded almost in real time, the first encoded data that has been transmitted by the terminal can be received and played back at another terminal, so that softer real-time distribution can be achieved.

For another example, the camera ex113 or the like extracts the feature amount from the image, compresses the material having the feature amount as a meta material, and transmits it to the server. The server determines the importance of the object from the feature amount, for example, and switches the quantization accuracy or the like in accordance with the meaning of the image. The feature quantity data is particularly effective for the accuracy and efficiency improvement of the motion vector prediction when recompressing on the server. Further, it is also possible to perform simple coding such as VLC (variable length coding) in the terminal, and to perform processing with a large load such as CABAC (Context Adaptive Binary Arithmetic Coding) on the server.

Further, in another example, in a stadium, a shopping mall, a factory, or the like, a plurality of pieces of video data in which almost the same scene is captured via a plurality of terminals may exist. In this case, a plurality of terminals that have been photographed and other terminals and servers that are not photographed are used as needed, for example, a GOP (Group of Picture) unit, a picture unit, or a square dividing the picture. Units and the like are assigned code processing to perform distributed processing. This reduces latency and enables better real-time performance.

Moreover, since the plurality of video data are almost the same scene, it is also possible to manage and/or instruct the server to refer to the image data captured by each terminal. Alternatively, the server may receive the encoded data from each terminal, change the reference relationship between the plurality of materials, or correct or replace the image itself to re-encode. Thereby, a stream that improves the quality and efficiency of one piece of data can be generated.

In addition, the server may first perform transcoding to change the encoding mode of the image data, and then distribute the image data. For example, the server can also convert the MPEG-based encoding method into a VP system, or convert H.264 to H.265.

In this way, the encoding process can be performed by a terminal or by one or more servers. In the following, the main body of the processing is a description using a "server" or a "terminal", but some or all of the processing by the server may be performed at the terminal. Part or all of the processing performed by the terminal is performed on the server. Also, regarding these parts, the same applies to the decoding process. [3D, multiple viewing angles]

In recent years, the situation in which different scenes photographed by a plurality of cameras ex113 and/or a smart phone ex115, which are almost simultaneously synchronized, or images or images captured at different angles of the same scene are integrated is also used. too much. The images captured by the respective terminals are integrated based on the relative positional relationship between the separately obtained terminals, or the area where the feature points included in the images match.

The server not only encodes the two-dimensional moving image, but also encodes the still image automatically or at the time specified by the user according to the scene analysis of the moving image, and then transmits the still image to the receiving terminal. Further, when the server can obtain the relative positional relationship between the photographing terminals, it is not only a two-dimensional moving image, but also a three-dimensional shape of the scene based on images taken from the same scene from different viewing angles. In addition, the server can also encode 3D data generated by point cloud or the like, or use 3D data to identify or track the results of people or objects, and shoot from multiple terminals. The image is selected, or reconstructed, to produce an image to be transmitted to the receiving terminal.

In this way, the user can arbitrarily select each of the images corresponding to each of the imaging terminals to view the scene, and to view the contents of the image from the three-dimensional data reconstructed from the plurality of images or images. can. Further, similar to the image, the sound can be collected from a plurality of different viewing angles, and the server can be multiplexed with the image and the sound and image from a specific viewing angle or space.

In addition, in recent years, contents such as Virtual Reality (VR/Virtual Reality) and Augmented Reality (AR/Augmented Reality) have become popular in the relationship between the real world and the virtual world. In the case of an VR image, the server can also make viewpoint images for the right eye and the left eye, and allow reference between the viewpoint images by Multi-View Coding (MVC/Multi-view coding). The encoding may also be encoded as a different stream without referring to each other. When decoding different streams, it is preferable to synchronize and play each other in a manner that reproduces the virtual three-dimensional space in response to the user's viewpoint.

In the case of the image of the AR, the server superimposes the virtual object information on the virtual space on the camera information in the real space according to the position in the 3D or the movement of the user's viewpoint. The decoding device can also acquire or maintain the virtual object information and the three-dimensional data, and generate a two-dimensional image in response to the movement of the user's viewpoint, and smoothly connect to create overlapping data. Alternatively, the decoding device may transmit the movement of the user's viewpoint to the server in addition to the request instruction of the virtual object information, and the server cooperates with the movement of the received viewpoint to create overlapping data from the three-dimensional data held by the server. And the overlapping data is encoded and distributed to the decoding device. Alternatively, the superimposed data may have an alpha value indicating the transmittance in addition to RGB, and the server may set the alpha value of the portion other than the object made from the three-dimensional data to 0, etc., and make the portion penetrate. Encoding in the state of the state. Alternatively, the server may generate data as a chroma key, which sets the RGB value of the predetermined value to the background, and sets the portion other than the object to the background color.

Similarly, the decoding processing of the distributed data may be performed at each terminal of the client, or may be performed on the server side, or may be shared with each other. For example, a terminal device may first send a receiving request to the server, receive the content requested by the other terminal device, perform decoding processing, and send the decoded signal to the device having the display. It is possible to play the content with good picture quality by dispersing the processing and selecting the appropriate content without depending on the performance of the communication terminal itself. Further, in another example, while a large-sized image data is received by a TV or the like, a part of a region such as a square divided by the image may be decoded and displayed on the personal terminal of the viewer. By this, the entire image can be shared, and the area of responsibility for itself or the area to be confirmed in more detail can be confirmed by the side.

In the future, in the case of indoor or outdoor use, it is possible to use a distribution system specification such as MPEG-DASH in a situation where a plurality of types of wireless communication such as a short distance, a medium distance, or a long distance can be used. Data, while receiving content seamlessly, is predictable. Thereby, the user can switch in real time not only by the terminal device but also by a decoding device or a display device such as a display provided indoors or outdoors. Further, it is possible to perform decoding while switching the decoded terminal device and the displayed terminal device based on the own location information and the like. Thereby, it is also possible to make it possible to move the map information while displaying the map information on the wall surface of the building or the part of the ground adjacent to the device in which the display is embedded. Moreover, it is also possible to make it easy to access the encoded material on the network, such as the encoded data may be cached by a server that can be accessed from the receiving terminal in a short time, or copied to the content distribution service ( The edge server in the Contents Delivery Service, etc., to switch the bit rate of the received data. [Adjustable coding]

The switching of the content is described by an adjustable stream shown in Fig. 25, which is a stream compressed and encoded by the moving image encoding method described in each of the above embodiments. Although the server has a plurality of streams having the same content but different qualities as individual streams, it may be configured as follows: a flexible use of time/space type adjustable stream characteristics to switch Content, wherein the time/space type tunable stream is implemented by layering as shown in the figure. In other words, the decoding side determines which layer to decode based on factors such as the inherent factors of performance and the state of the communication band, so that the decoding side can freely switch between low-resolution content and high-resolution content. And decode it. For example, if you want to put a subsequent part of the image that was viewed on the mobile phone ex115 on the mobile home and watch it on a network TV or the like, the machine only needs to decode the same stream to different layers. Therefore, the burden on the server side can be reduced.

Further, as described above, in addition to the configuration in which the picture is encoded in each layer and the scalability of the enhancement layer exists in the upper layer of the base layer, the enhancement layer may contain an interpretation based on image-based statistical information or the like. Information, the decoding side based on the interpretation information, the base layer of the image is super-image analysis, in order to produce high-quality content. The super-image analysis may be any one of the improvement of the SN ratio and the expansion of the resolution at the same resolution. The interpretation information is information including linear or non-linear filter coefficients used for a specific super image analysis process, or filter values used in a specific super image analysis process, mechanical learning, or parameter values in a least squares operation. Information, etc.

Alternatively, the image may be divided into squares or the like in accordance with the meaning of an object or the like in the image, and the decoding side selects a block to be decoded, thereby decoding only a part of the area. In addition, the attributes of the object (person, car, ball, etc.) and the position in the image (coordinate position in the same image, etc.) are stored as interpretation information, so that the decoding side can specify the desired information according to the interpretation information. The position of the object and decide which square to contain the object. For example, as shown in FIG. 26, the interpretation information is stored using a material storage structure different from the pixel material, such as an SEI message in HEVC. The interpretation information is, for example, displaying the position, size, or color of the main object.

Alternatively, the interpretation information may be stored in a unit composed of a plurality of pictures such as a stream, a sequence, or a random access unit. Thereby, the decoding side can obtain the time when the specific person appears in the image, and the information of the picture unit can be used, so that the picture of the specific object and the position of the object in the picture can be specified. [Optimization of web pages]

FIG. 27 is a view showing an example of a display screen of a web page in a computer ex111 or the like. FIG. 28 is a view showing an example of a display screen of a web page of the smart phone ex115 or the like. As shown in FIG. 27 and FIG. 28, the webpage includes a plurality of linked images, wherein the linked images are links to image content, and the manner in which the linked images are viewed depends on the device being viewed. Different. When a plurality of linked images are viewed on the screen, the display device (decoding) is displayed until the user selects the link image or until the link image approaches the center of the screen or the entire link image enters the screen. The device displays a still image or an I picture as a link image, or displays a gif-like image in a plurality of still images or I pictures, or decodes the image only by receiving the base layer. And display.

When the linked image is selected by the user, the display device regards the base layer as the highest priority for decoding. Further, when there is information for displaying the adjustable content in the HTML constituting the web page, the display device can decode until the enhancement layer. Moreover, in order to ensure real-time performance, before the selection or the communication band is extremely narrow, the display device can only decode and display the picture in front of the reference picture (I picture, P picture, only the B picture in front), thereby reducing the head. The delay between the decoding time of the picture and the display time (delay from the start of decoding of the content to the start of display). Moreover, the display device can also ignore the reference relationship of the picture, so that all the B pictures and the P picture are referred to the front, and the decoding is roughly performed first, and then, after a period of time, the normal picture is added as the received picture increases. decoding. [Automatic driving]

In addition, when a static image or a video material such as two-dimensional or three-dimensional map information is transmitted and received for automatic driving or support traveling of a car, the receiving terminal device has image data of one or more layers. It is also possible to receive weather or construction information as an interpretation of the information, and to make the correspondence of the information to be decoded. In addition, the interpretation of information can also belong to the layer, or simply multiplex with the image data.

At this time, since the car, the aerial camera, or the airplane including the receiving terminal moves, the receiving terminal transmits the location information of the receiving terminal when requesting reception, thereby switching the base stations ex106 to ex110 while switching. Achieve seamless reception and decoding. Moreover, the receiving terminal can dynamically switch to which extent the interpretation information is received or to what extent the map information is updated, depending on the user's selection, the user's condition, or the state of the communication band.

As described above, in the content supply system ex100, the client can receive the encoded information transmitted by the user in real time and decode it, and play it. [Distribution of personal content]

Further, in the content supply system ex100, not only the long-term content is transmitted through the high image quality performed by the image distribution company, but also the unicast of the short-term content can be performed with low image quality by the individual. , or multicast for distribution. In addition, the content of the individual is considered to increase in the future. In order to make the personal content into more excellent content, the server can also perform editing processing and then perform encoding processing. This can be realized, for example, as follows.

In the real-time or first-time storage after photography, the server performs recognition processing such as shooting error, scene search, meaning analysis, and object detection from the original image or the encoded data. Then, according to the identification result, the server manually or automatically performs a correction of the out-of-focus or the jitter, or deletes a scene having a low importance such as a scene with a lower brightness or an unfocused image, or emphasizes the edge of the object, Or edit the color tones, etc. The server encodes the edited data based on the edited result. Moreover, it is known that when the shooting time is too long, the viewing rate is lowered, and the server can automatically edit the scenes based on the image processing results, not only for scenes of low importance as described above, but also for scenes with less motion. It becomes content within a specific time range in response to the shadow time. Alternatively, the server may also generate a digest based on the result of the semantic analysis of the scene and encode it.

In addition, in the personal content, if the original content is changed, there are cases in which copyrights, copyrights, or portrait rights are taken, and the scope of sharing exceeds the intended range, which is not appropriate for individuals. Happening. Therefore, for example, the server can also intentionally change the face of the peripheral portion of the screen or the home to an unfocused image to perform encoding. Further, the server can recognize whether or not a face of a person different from the person registered in advance is captured in the image to be encoded, and if it is photographed, a part of the face is subjected to a mosaic or the like. Alternatively, in the pre-processing or post-processing of the encoding, from the viewpoint of copyright, etc., the user specifies the person or background area to be processed for the image, and the server replaces the designated area with another image, or performs The processing of fuzzy focus, etc. is also possible. In the case of a character, in the moving image, it is possible to replace a part of the image of the face while tracking the person.

Further, since the viewing of personal content having a small amount of data is highly demanded in real time, the decoding apparatus first receives the highest priority in the base layer and performs decoding and playback depending on the bandwidth. The decoding device can also receive the enhancement layer during this period, and when there are two or more playbacks in the case of loop playback, the high-quality video is played together with the enhancement layer. If the stream with adjustable coding has been performed in this way, the following experience can be provided, that is, although it is a rough animation when it is not selected or at the beginning of the viewing, it gradually becomes refined and refined. The image is getting better. In addition to the adjustable coding, the same stream can be provided by using a rough stream that is played for the first time and a second stream that is coded with reference to the first animation as one stream. [Other use cases]

Further, these encoding or decoding processes are generally processed in the LSI ex500 included in each terminal. The LSI ex500 may be a single wafer or a plurality of wafers. Alternatively, the software for encoding or decoding moving pictures may be loaded into some recording medium (CD-ROM, floppy disk, or hard disk, etc.) that can be read by a computer ex111 or the like, and encoded using the software or Decoding processing. Further, when the smartphone ex115 is attached to a camera, it can also transmit animation data acquired by the camera. The animation data at this time is data that has been encoded by the LSI ex500 which is included in the smart phone ex115.

In addition, the LSIex500 can also be configured to download an application software program. At this time, first, the terminal determines whether the terminal supports the encoding mode of the content or whether it has the execution capability of the specific service. When the terminal does not support the encoding method of the content, or does not have the execution capability of the specific service, the terminal downloads the codec or the application software program, and then acquires and plays the content.

Further, the content encoding system ex100 via the Internet ex101 is not limited, and at least the moving image encoding device (image encoding device) or the moving image decoding device of the above embodiments may be incorporated in the digital broadcasting system. Any of the decoding devices. Since it is a multiplexed data that has been subjected to multiplex processing of video and audio by radio waves for broadcasting by satellite or the like, and is transmitted and received by the radio wave for broadcasting, it is easy to perform unicast propagation with respect to the content supply system ex100, and digitally. Although the broadcasting system is advantageous for the difference of multicasting, the same application can be applied to the encoding processing and the decoding processing. [Hardware composition]

Fig. 29 is a diagram showing the smartphone ex115. FIG. 30 is a diagram showing an example of the configuration of the smartphone ex115. The smart phone ex115 includes an antenna ex450 for transmitting and receiving radio waves with the base station ex110, a camera portion ex465 for capturing images and a still image, and a display portion ex458 for displaying the camera portion ex465. The captured image and the data decoded by the image received by the antenna ex450. The smart phone ex115 further includes an operation unit ex466, which is a touch panel or the like, a sound output unit ex457, which is a speaker for outputting sound or sound, and an audio input unit ex456, which is a microphone for inputting sound, and the like. Ex467, can save captured images or still images, recorded sounds, received images or still images, mailed and other encoded materials, or decoded data; and slot ex464, and SIMex468 The interface between the SIMex 468 is used for specific users, and the Internet is the first to perform authentication for accessing various materials. Alternatively, an external memory may be used instead of the memory unit ex467.

Further, the main control unit ex460, which is integrally controlled by the display unit ex458 and the operation unit ex466, and the power supply circuit unit ex461, the operation input control unit ex462, the video signal processing unit ex455, the camera interface ex463, and the display control unit ex459 are adjusted. The variable/demodulation unit ex452, the multiplex/separation unit ex453, the audio signal processing unit ex454, the slot unit ex464, and the memory unit ex467 are connected via the bus bar ex470.

When the power switch is turned on by the user's operation, the power supply circuit unit ex461 supplies power to each unit from the battery pack, thereby causing the smartphone ex115 to be activated.

The smartphone ex115 performs processing such as call and data communication based on the control of the main control unit ex460 such as a CPU, a ROM, and a RAM. At the time of the call, the audio signal received by the voice input unit ex456 is converted into a digital audio signal by the audio signal processing unit ex454, and the signal is subjected to spectrum diffusion processing in the modulation/demodulation unit ex452, and the transmission/reception unit ex451 The digital analog conversion processing and the frequency conversion processing are performed, and then transmitted via the antenna ex450. Further, the received data is amplified, and the frequency conversion processing and the analog digital conversion processing are performed, and the spectrum de-diffusion processing is performed in the modulation/demodulation unit ex452, and converted into an analog sound signal by the audio signal processing unit ex454, and then the signal is transmitted from the analog signal. The sound output unit ex457 outputs. In the data communication mode, the text, the still image, or the video data is sent to the main control unit ex460 via the operation input control unit ex462 through the operation of the operation unit ex466 or the like of the main unit, and is similarly transmitted and received. In the data communication mode, when transmitting a video, a still image, or a video or a sound, the video signal processing unit ex455 transmits the video signal stored in the memory unit ex467 or the video signal input from the camera unit ex465 through the respective The moving picture coding method shown in the embodiment performs compression coding, and the coded video data is sent to the multiplex/separation unit ex453. Further, the audio signal processing unit ex454 encodes the audio signal received by the audio input unit ex456 during the imaging of the video or the still image by the camera unit ex465, and sends the encoded audio data to the multiplex/separation unit ex453. . The multiplex/separation unit ex453 multiplexes the encoded video data and the encoded audio data in a predetermined manner, and performs modulation and demodulation (modulation/demodulation circuit unit) ex452 and transmission. The receiving unit ex451 performs the modulation processing and the conversion processing, and transmits it via the antenna ex450.

When receiving an image attached to an e-mail or a chat or connecting to a web page or the like, the multiplexer/extension unit ex453 performs the multiplexed data in order to decode the multiplexed data received via the antenna ex450. Separating, thereby dividing the multiplexed data into a bit stream of the image data and a bit stream of the sound data, and supplying the encoded image data to the image signal processing unit ex455 via the synchronous bus ex470, and encoding the data The sound data is supplied to the audio signal processing unit ex454. The video signal processing unit ex455 decodes the video signal by the video decoding method corresponding to the video encoding method described in each of the above embodiments, and transmits the video signal to the display unit ex458 via the display control unit ex459. The image or still image contained in the dynamic image file. Further, the audio signal processing unit ex454 decodes the audio signal and outputs the sound from the audio output unit ex457. In addition, since real-time streaming has become widespread, depending on the user's situation, sound playback may also occur in socially inappropriate scenes. For this reason, as an initial value, the audio signal is not played, and only the composition of the video material is more desirable. It is also possible to play the sound synchronously only when the user performs an operation, such as clicking on image data.

Here, the smart phone ex115 has been described as an example. The terminal device can also be considered in the following three installation forms, except for the signal transceiving type terminal having both the encoder and the decoder. A messaging terminal having an encoder and a receiving terminal having only a decoder. Further, in the digital broadcasting system, the multiplexed data for performing multiplex processing on the video data is received or transmitted, but the multiplexed data may be combined with the sound data. The image has associated text data and other multiplex processing, and can also receive or send the image data itself, rather than multiplex data.

Further, the case where the main control unit ex460 including the CPU controls the encoding process or the decoding process will be described, but the case where the terminal has the GPU is also large. Therefore, the configuration of the GPU can be flexibly utilized by using a memory that is shared between the CPU and the GPU, or a memory that manages the address to form a state that can be used in common. Consolidate to work together. Thereby, the encoding time can be shortened, real-time performance can be ensured, and low latency can be realized. In particular, it is efficient to use the CPU instead of the GPU to perform motion estimation, deblocking filter, SAO (Sample Adaptive Offset), and conversion and quantization processing in units of pictures and the like. .

This aspect can also be implemented in combination with at least a portion of other aspects of the disclosure. Further, a part of the processing described in the flowchart of the present aspect, a part of the configuration of the apparatus, a part of the syntax, and the like may be combined with other aspects. (industrial use)

The present disclosure is applicable to an encoding device, a decoding device, an encoding method, and a decoding method.

100‧‧‧ coding device

102‧‧‧ Division

104‧‧‧Subtraction Department

106‧‧‧Transition Department

108‧‧‧Quantity Department

110‧‧‧ Entropy Coding Department

112, 204‧‧‧Anti-Quantization Department

114, 206‧‧‧Anti-conversion department

116, 208‧‧ Addition Department

118, 210‧‧‧ Block memory

120, 212‧‧‧Circuit Filtering Department

122, 214‧‧‧ frame memory

124, 216‧‧ Internal Forecasting Department

126, 218‧ ‧ forecasting department

128, 220‧‧‧Predictive Control Department

200‧‧‧ decoding device

202‧‧‧ Entropy Decoding Department

Ex100‧‧‧Content Supply System

Ex101‧‧‧Internet

Ex102‧‧‧Internet Service Provider

Ex103‧‧‧Streaming server

Ex104‧‧‧Communication Network

Ex106 to ex110‧‧‧ base station

Ex111‧‧‧ computer

Ex112‧‧‧game machine

Ex113‧‧‧Camera

Ex114‧‧‧Home appliances

Ex115‧‧‧Smart mobile phone

Ex116‧‧‧ satellite

Ex117‧‧ aircraft

Ex450‧‧‧Antenna

Ex451‧‧‧Send/Receive Department

Ex452‧‧‧Modulation/Demodulation Department

Ex453‧‧‧Multiplex/Separation Department

Ex454‧‧‧Sound Signal Processing Department

Ex455‧‧‧Image Signal Processing Department

Ex456‧‧‧Sound Input Department

Ex457‧‧‧Sound Output Department

Ex458‧‧‧Display Department

Ex459‧‧‧Display Control Department

Ex460‧‧‧Main Control Department

Ex461‧‧‧Power Circuit Department

Ex462‧‧‧Operation Input Control Department

Ex463‧‧‧Camera face

Ex464‧‧‧Slots

Ex465‧‧‧ camera department

Ex466‧‧‧Operation Department

Ex467‧‧‧Memory Department

Ex468‧‧‧SIM

Ex470‧‧‧ busbar

Fig. 1 is a block diagram showing a functional configuration of an encoding apparatus according to a first embodiment.

Fig. 2 is a view showing an example of block division in the first embodiment;

Figure 3 is a table showing the conversion basis functions corresponding to the respective conversion patterns.

Fig. 4A is a view showing an example of the shape of a filter used in ALF.

Fig. 4B is a view showing another example of the shape of the filter used in the ALF.

Fig. 4C is a view showing another example of the shape of the filter used in the ALF.

Figure 5A is a graph showing 67 intra prediction modes in intra prediction.

FIG. 5B is a flowchart for explaining an outline of predicted image correction processing by OBMC processing.

FIG. 5C is a conceptual diagram for explaining an outline of a predicted image correction process by OBMC processing.

Fig. 5D is a diagram showing an example of FRUC.

Fig. 6 is a diagram for explaining pattern matching (bidirectional matching) between two blocks along a movement trajectory.

FIG. 7 is a diagram for explaining pattern matching (template matching) between a template in a current picture and a block in a reference picture.

Fig. 8 is a view for explaining a model assuming constant-speed linear motion.

FIG. 9A is a diagram for explaining the derivation of a motion vector of a sub-block unit whose motion vector is based on a motion vector of a plurality of adjacent blocks.

Fig. 9B is a diagram for explaining an outline of a motion vector derivation process in a merge mode.

Fig. 9C is a conceptual diagram for explaining an outline of DMVR processing.

9D is a diagram for explaining an outline of a method of generating a predicted image, which is a brightness correction process using LIC processing.

Fig. 10 is a block diagram showing a functional configuration of a decoding apparatus according to the first embodiment.

Fig. 11 is a flow chart showing the deblocking filter processing in the first embodiment.

Fig. 12 is a view showing an example of arrangement of pixels in a block boundary in the first embodiment;

Fig. 13 is a flow chart showing the deblocking filter processing in the first embodiment.

Fig. 14 is a flowchart showing the deblocking filter processing in the second embodiment.

Fig. 15 is a view showing the relationship between the pixel position and the error in the block of the second embodiment;

Fig. 16 is a flow chart showing the deblocking filter processing in the third embodiment.

Fig. 17 is a view showing a conversion substrate of DCT-II of the third embodiment.

Fig. 18 is a view showing a conversion substrate of DST-VII of the third embodiment.

Fig. 19 is a flow chart showing the deblocking filter processing in the fourth embodiment.

Fig. 20 is a flow chart showing the deblocking block filtering process of the fifth embodiment.

Fig. 21 is a view showing an example of a weight which is a direction in which the prediction direction and the block boundary are in accordance with the fifth embodiment.

Figure 22 is a flowchart showing the deblocking filter processing in the sixth embodiment.

Fig. 23 is a view showing an example of weights which are quantization parameters according to the sixth embodiment.

Fig. 24 is a view showing the overall configuration of a content supply system for realizing a content distribution service.

Fig. 25 is a view showing an example of a coding structure in the case of adjustable coding.

Fig. 26 is a view showing an example of a coding structure in the case of adjustable coding.

Fig. 27 is a view showing an example of a display screen of a web page.

28 is a diagram showing an example of a display screen of a web page.

Figure 29 is a diagram showing an example of a smart phone.

Fig. 30 is a block diagram showing a configuration example of a smart phone.

Claims (8)

An encoding device includes: a processor; and a memory, wherein the processor uses the memory to determine a filter characteristic that the boundary of the clip block is asymmetric according to a position in a block of the target pixel, and the object is The pixel performs deblocking filtering processing having the aforementioned filter characteristics determined. An encoding apparatus according to claim 1, wherein the filter characteristic is determined in such a manner that the filter characteristic is determined by a pixel which is farther from the reference pixel which is predicted from the inside, and the influence of the filtering process becomes larger. . The encoding apparatus of claim 1, wherein the determination of the filter characteristic is performed in such a manner that the effect of the de-blocking filtering process of the lower right pixel is greater than the effect of the de-blocking filtering process of the upper left pixel Determine the aforementioned filter characteristics. A decoding apparatus includes: a processor; and a memory, wherein the processor uses the memory to determine a filter characteristic that the boundary of the clip block is asymmetric according to a position in a block of the target pixel, and the object is The pixel performs deblocking filtering processing having the aforementioned filter characteristics determined. The decoding device of claim 4, wherein in the determining of the filter characteristic, the filter characteristic is determined in such a manner that the influence of the filtering process becomes larger as the pixel is farther from the intra-predicted reference pixel. . The decoding apparatus of claim 4, wherein in the determining of the filter characteristic, the influence of the foregoing deblocking filtering process of the lower right pixel is greater than the influence of the foregoing deblocking filtering process of the upper left pixel Determine the aforementioned filter characteristics. An encoding method comprising the steps of: determining, according to a position within a block of a target pixel, a filter characteristic that is asymmetric of a boundary of the slab block, and performing a solution region having the determined filter characteristic for the target pixel Block filtering processing. A decoding method includes the following steps: determining, according to a position in a block of a target pixel, a filter characteristic that is asymmetric of a boundary of the clip block, and performing a solution region having the determined filter characteristic on the target pixel Block filtering processing.