JP2020092456A - Image encoding device, image encoding method and image encoding program, and image decoding device, image decoding method and image decoding program - Google Patents

Abstract

To provide an image encoding device capable of improving encoding efficiency by performing block division suitable for image encoding and decoding.SOLUTION: In an image encoding device, a primary signal block division unit that divides an image into blocks and performs encoding in the divided block units divides a primary signal of the image into rectangles having a predetermined size to generate primary signal blocks. A secondary signal block division unit divides a secondary signal of the image into rectangles having a predetermined size to generate secondary signal blocks. A primary signal prediction unit predicts the primary signal, and a secondary signal prediction unit predicts the secondary signal. The secondary signal prediction unit can perform inter-component prediction of predicting the secondary signal from the encoded primary signal, and restricts the inter-component prediction on the basis of at least one of the size of the primary signal block and the size of the secondary signal block.SELECTED DRAWING: None

Description

The present invention relates to a technique of dividing an image into blocks and performing encoding and decoding in units of the divided blocks.

In image encoding and decoding, an image is divided into blocks that are a set of a predetermined number of pixels, and encoding and decoding are performed in block units. By performing appropriate block division, the coding efficiency of intra-picture prediction (intra prediction) and inter-picture prediction (inter prediction) is improved. In intra prediction, the coding efficiency is improved by predicting the secondary signal from the decoded image of the primary signal.

JP, 2013-90015, A

However, when the secondary signal is predicted from the decoded image of the primary signal, the amount of processing increases, and the dependency between the processing on the primary signal and the processing on the secondary signal occurs, which makes parallel processing difficult.

The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for improving coding efficiency by performing block division suitable for image coding and decoding.

In order to solve the above problems, an image encoding device according to an aspect of the present invention is an image encoding device that divides an image into blocks and performs encoding in units of divided blocks, and a primary signal of the image. A primary signal block dividing unit that divides a rectangular signal of a predetermined size to generate a primary signal block, a secondary signal block dividing unit that generates a secondary signal block by dividing a secondary signal of the image into a rectangular shape of a predetermined size, and a primary A primary signal prediction unit that predicts a signal, and a secondary signal prediction unit that predicts a secondary signal, the primary signal block and the secondary signal block are each independently divided, the primary signal is a luminance signal, the secondary The signal is a color difference signal, the secondary signal prediction unit is capable of inter-component prediction to predict a secondary signal from the encoded primary signal, if the size of the primary signal block is a predetermined size or more, between the components Prohibit the use of forecasts.

Another aspect of the present invention is an image coding method. This method divides the image into blocks,
An image encoding method for performing encoding in units of divided blocks, a primary signal block dividing step of generating a primary signal block by dividing a primary signal of the image into rectangles of a predetermined size, and a secondary signal of the image. Including a secondary signal block dividing step of generating a secondary signal block by dividing into a rectangular of a predetermined size, a primary signal prediction step of predicting a primary signal, and a secondary signal prediction step of predicting a secondary signal, wherein the primary signal block And the secondary signal block are each independently divided, the primary signal is a luminance signal, the secondary signal is a color difference signal, the secondary signal prediction step is a component for predicting the secondary signal from the encoded primary signal When inter prediction is possible and the size of the primary signal block is equal to or larger than a predetermined size, use of inter component prediction is prohibited.

Yet another aspect of the present invention is an image decoding apparatus. This device is an image decoding device that performs decoding in units of blocks into which an image is divided, and a primary signal block division unit that divides a primary signal of the image into rectangles of a predetermined size to generate a primary signal block, A secondary signal block division unit that generates a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size, a primary signal prediction unit that predicts the primary signal, and a secondary signal prediction unit that predicts the secondary signal. , The primary signal block and the secondary signal block are each independently divided, the primary signal is a luminance signal, the secondary signal is a color difference signal, the secondary signal prediction unit, the secondary from the decoded primary signal Inter-component prediction for predicting a signal is possible, and when the size of the primary signal block is equal to or larger than a predetermined size, use of the inter-component prediction is prohibited.

Yet another aspect of the present invention is an image decoding method. This method is an image decoding method for performing decoding on a block-by-block basis in which an image is divided, and a primary signal block division step of dividing the primary signal of the image into rectangles of a predetermined size to generate a primary signal block, A secondary signal block dividing step of dividing the secondary signal of the image into rectangles of a predetermined size to generate a secondary signal block, a primary signal prediction step of predicting a primary signal, and a secondary signal prediction step of predicting a secondary signal. , The primary signal block and the secondary signal block are each independently divided, the primary signal is a luminance signal, the secondary signal is a color difference signal, the secondary signal prediction step is secondary from the decoded primary signal Inter-component prediction for predicting a signal is possible, and when the size of the primary signal block is equal to or larger than a predetermined size, use of the inter-component prediction is prohibited.

It should be noted that any combination of the above constituent elements, and the expression of the present invention converted between a method, a device, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

According to the present invention, it is possible to perform block division suitable for image encoding and decoding, improve encoding efficiency, and provide image encoding and decoding with a small processing amount.

It is a block diagram of the image coding apparatus which concerns on 1st Embodiment. It is a block diagram of the image decoding apparatus which concerns on 1st Embodiment. 7 is a flowchart illustrating division into tree blocks and division inside a tree block. It is a figure which shows a mode that the input image is divided into tree blocks. It is a figure explaining z-scan. It is a figure which divided a tree block into four horizontally and vertically. It is the figure which divided the tree block into two in the horizontal direction. It is the figure which divided the tree block into two in the perpendicular direction. It is a flow chart explaining processing of each divided block when a tree block is divided into four horizontally and vertically. It is a flow chart explaining processing of each divided block when a tree block is divided into two in the horizontal direction. It is a figure which shows the mode of subdivision of the block which was divided|segmented when the division|segmentation of a tree block is divided into two in the horizontal direction. It is a flow chart explaining processing of each divided block when a tree block is divided into two in the perpendicular direction. It is a figure which shows the mode of subdivision of the divided block when the tree block is divided into two in the vertical direction. It is a figure which shows the example of the syntax regarding the block division of 1st Embodiment. It is a figure explaining intra prediction. It is a figure explaining inter prediction. It is a figure explaining a color difference format. It is a figure explaining luma color difference intra prediction. It is a flowchart explaining a luminance color difference intra prediction. It is a figure explaining the case where the size of a color difference block is larger than the size of a luminance block. It is a figure explaining the case where the size of a color difference block is smaller than the size of a luminance block. It is a figure explaining the example of the syntax of intra color difference prediction mode. It is a figure explaining the example which replaces a peripheral pixel and limits intra color difference prediction. It is a figure explaining the luminance color difference intra prediction by the difference in the size of a luminance block.

The embodiment of the present invention divides an image into rectangular blocks and encodes the divided blocks.
An image encoding technique for decoding is provided.

(First embodiment)
The image coding apparatus 100 and the image decoding apparatus 200 according to Embodiment 1 of the present invention will be described.

FIG. 1 is a configuration diagram of an image encoding device 100 according to the first embodiment. Here, FIG. 1 shows only the data flow regarding the image signal, and each additional component other than the image signal, such as the motion vector and the prediction mode, is supplied to the encoded bit string generation unit 105 by each component and the corresponding encoding is performed. Although data is generated, the flow of data regarding additional information is not shown.

The block division unit 101 divides an image into coding target blocks that are processing units for coding, and supplies the image signal in the coding target block to the residual signal generation unit 103. The block division unit 101 also supplies the image signal of the encoding target block to the predicted image generation unit 102 in order to evaluate the degree of coincidence of the predicted images.

The block division unit 101 recursively divides an image into rectangles of a predetermined size to generate a coding target block. The block division unit 101 divides the target block in the recursive division into four in the horizontal and vertical directions to generate four blocks, and the target block in the recursive division into two in the horizontal direction or the vertical direction. And a two-part dividing unit for generating two blocks. The detailed operation of the block division unit 101 will be described later.

The predicted image generation unit 102 uses the decoded image signal supplied from the decoded image memory 108 to predict intra picture (intra prediction) or inter picture prediction (inter prediction) based on the prediction mode.
And a predicted image signal is generated. The image signal in the coding target block supplied from the block division unit 101 is used for evaluation of intra prediction and inter prediction. In the intra prediction, the image signal of the encoding target block supplied from the block dividing unit 101 and the surroundings close to the encoding target block existing in the same picture as the encoding target block supplied from the decoded image memory 108. A predicted image signal is generated using the image signal of the coded block of. In inter prediction, the image signal of the encoding target block supplied from the block dividing unit 101 is stored in the decoded image memory 108 that is located in front or behind in time series of the picture (encoding picture) including the encoding target block. Use the coded picture as a reference picture,
A block matching degree evaluation such as block matching is performed between the coded picture and the reference picture to obtain a motion vector indicating a motion amount, and motion compensation is performed from the reference picture based on the motion amount to generate a predicted image signal. .. The prediction image generation unit 102 supplies the prediction image signal thus generated to the residual signal generation unit 103.

The residual signal generation unit 103 subtracts the image signal to be encoded from the prediction signal generated by the prediction image generation unit 102 to generate a residual signal, and supplies the residual signal to the orthogonal transformation/quantization unit 104.

The orthogonal transformation/quantization unit 104 performs orthogonal transformation/quantization on the residual signal supplied from the residual signal generation unit 103, and the orthogonal transformation/quantized residual signal is encoded bit string generation unit 105 and inverse quantization. -Supply to the inverse orthogonal transform unit 106.

The encoded bit string generation unit 105 generates an encoded bit string for the orthogonal transform/quantized residual signal supplied from the orthogonal transform/quantization unit 104. Also, the encoded bit string generation unit 1
Reference numeral 05 generates a corresponding encoded bit string for additional information such as a motion vector, a prediction mode, and block division information.

The inverse quantization/inverse orthogonal transformation unit 106 performs inverse quantization/inverse orthogonal transformation on the orthogonal transformation/quantized residual signal supplied from the orthogonal transformation/quantization unit 104, and performs inverse quantization/inverse orthogonal transformation. The residual signal is supplied to the decoded image signal superimposing unit 107.

The decoded image signal superimposing unit 107 superimposes the predicted image signal generated by the predicted image generating unit 102 on the residual signal that has been inversely quantized and inversely orthogonally transformed by the inverse quantization/inverse orthogonal transforming unit 106 to obtain a decoded image. It is generated and stored in the decoded image memory 108. The decoded image may be stored in the decoded image memory 108 after being subjected to a filtering process for reducing block distortion due to encoding.

FIG. 2 is a configuration diagram of the image decoding apparatus 200 according to the first embodiment. Here, FIG. 2 shows only the data flow regarding the image signal, and the additional information other than the image signal such as the motion vector and the prediction mode is supplied to the respective constituent elements by the bit string decoding unit 201 and used for the corresponding processing. The data flow relating to the additional information is not shown.

The bit string decoding unit 201 decodes the supplied encoded bit string, and supplies the orthogonal transform/quantized residual signal to the block dividing unit 202.

The block division unit 202 determines the shape of the decoding target block based on the decoded block division information, and inversely quantizes and inversely orthogonalizes the orthogonal transformation/quantized residual signal of the determined decoding target block. It is supplied to the conversion unit 203.

The block division unit 202 recursively divides the image into rectangles of a predetermined size based on the decoded block division information to generate a decoding target block. The block dividing unit 202 divides the target block in the recursive division into four in the horizontal and vertical directions to generate four blocks, and divides the target block in the recursive division into two in the horizontal direction or the vertical direction. And a two-part dividing unit for generating two blocks. The detailed operation of the block division unit 202 will be described later.

The inverse quantization/inverse orthogonal transformation unit 203 performs inverse orthogonal transformation and inverse quantization on the supplied orthogonal transformation/quantized residual signal to obtain the inverse orthogonal transformation/inverse quantized residual signal. To get

The predicted image generation unit 204 generates a predicted image signal from the decoded image signal supplied from the decoded image memory 206, and supplies the predicted image signal to the decoded image signal superposition unit 205.

The decoded image signal superimposing unit 205 superimposes the predicted image signal generated by the predicted image generating unit 204 and the residual signal that is inversely orthogonally transformed/inversely quantized by the inverse quantization/inverse orthogonal transformation unit 203 on each other. , A decoded image signal is generated, output, and stored in the decoded image memory 206.
The decoded image may be stored in the decoded image memory 206 after being subjected to filtering processing for reducing block distortion and the like due to encoding.

The operation of the block division unit 101 of the image encoding device 100 will be described in detail. FIG. 3 is a flowchart for explaining division into tree blocks and division inside tree blocks.

First, the input image is divided into tree blocks of a predetermined size (S1000). For example, the tree block has 128 pixels×128 pixels. However, the tree block is 128
The size is not limited to pixels×128 pixels, and any size and shape may be used as long as it is rectangular. Also, the size and shape of the tree block may be set to fixed values between the encoding device and the decoding device, but it is determined by the encoding device and recorded in the encoded bit stream for decoding. The device may be configured to use the recorded block size. FIG. 4 shows how the input image is divided into tree blocks. The tree blocks are encoded and decoded in raster scan order, left to right, top to bottom.

The inside of the tree block is further divided into rectangular blocks. The inside of the tree block is encoded and decoded in z-scan order. FIG. 5 shows the z-scan sequence. In z-scan, encoding and decoding are performed in the order of upper left, upper right, lower left, lower right. The inside of the tree block can be divided into four and two. The four divisions are divided in the horizontal direction and the vertical direction. Two
The division is performed in the horizontal direction or the vertical direction. Figure 6 shows treeblocks 4 horizontally and vertically
FIG. FIG. 7 is a diagram in which a tree block is horizontally divided into two parts. FIG. 8 is a diagram in which a tree block is vertically divided into two parts.

Referring back to FIG. It is determined whether the inside of the tree block is divided into four in the horizontal and vertical directions (S1001).

When it is determined that the inside of the tree block is divided into four (S1001: Yes), the inside of the tree block is divided into four (S1002), and each processing of the blocks divided into four in the horizontal and vertical directions is performed (S1003). The subdivision processing of the block divided into four will be described later (FIG. 9).

When it is determined that the inside of the tree block is not divided into four (S1001: No), it is determined whether or not the inside of the tree block is divided into two (S1004).

When it is determined that the inside of the tree block is divided into two (S1004: Yes), it is determined whether the direction of the two divisions is the horizontal direction (S1005).

When it is determined that the direction of dividing into two is the horizontal direction (S1005: Yes), the inside of the tree block is divided into two in the horizontal direction (S1006), and each processing of the block divided into two in the horizontal direction is performed (S1007). The subdivision processing of the block divided into two in the horizontal direction will be described later (FIG. 10).

When it is determined that the direction to be divided into two is the vertical direction instead of the horizontal direction (S1005: No), the inside of the tree block is vertically divided into two (S1008), and each processing of the vertically divided block is performed (S1009). .. The subdivision processing of the block divided into two in the horizontal direction will be described later (FIG. 11).

When it is determined that the inside of the tree block is not divided into two (S1004: No), the inside of the tree block is not divided into blocks and the block division processing ends (S1010).

Next, the processing of each divided block when the tree block is divided into four in the horizontal and vertical directions will be described using the flowchart in FIG.

It is determined whether or not the inside of the block is again divided into four in the horizontal and vertical directions (S1101).
.

When it is determined that the inside of the block is divided into four again (S1101: Yes), the inside of the block is again divided into four (S1102), and each processing of the block divided into four in the horizontal and vertical directions is performed (S1103).

When it is determined that the inside of the block is not divided into four again (S1101: No), it is determined whether or not the inside of the block is divided into two (S1104).

When it is determined that the inside of the block is divided into two (S1104: Yes), it is determined whether the direction of the two divisions is the horizontal direction (S1105).

When it is determined that the direction of dividing into two is the horizontal direction (S1105: Yes), the inside of the block is divided into two in the horizontal direction (S1106), and each processing of the block divided into two in the horizontal direction is performed (S1).
107).

When it is determined that the direction of dividing into two is the vertical direction instead of the horizontal direction (S1105: No), the inside of the block is divided into two in the vertical direction (S1108), and each processing of the block divided into two in the vertical direction is performed (S1109).

When it is determined that the inside of the block is not divided into two (S1104: No), the inside of the block is not divided into blocks and the block division processing is ended (S1110).

The process shown in the flowchart of FIG. 9 is executed for each of the four divided blocks. The inside of the block divided into four is also encoded and decoded in the z-scan order.

Next, processing of each divided block when the tree block is divided into two in the horizontal direction will be described with reference to the flowchart of FIG.

When the tree block is divided into two in the horizontal direction, each of the two divided blocks first determines whether or not the inside of the block is divided into four in the horizontal and vertical directions (S1201).

When it is determined that the inside of the block is divided into four (S1201: Yes), the inside of the block is divided into four.
Each block is divided (S1202) and each block is divided into four blocks in the horizontal and vertical directions (S12).
03).

When it is determined that the inside of the block is not divided into four (S1201: No), it is determined whether the inside of the block is divided into two again (S1204).

When it is determined that the block is divided into two again (S1204: Yes), the inside of the block is vertically divided (S1205), and each processing of the vertically divided block is performed (S1206).

When it is determined that the block is not divided into two again (S1204: No), the inside of the block is not re-divided and the block division processing is ended (S1207).

FIG. 11 shows a state of subdivision of a divided block when the treeblock is divided into two in the horizontal direction. Here, when the tree block, which is the parent block, is divided into two in the horizontal direction, the re-division into two of the divided blocks allows only two divisions in the vertical direction and automatically divides the two in the vertical direction. Further, when the tree block, which is the parent block, is divided into two, it is possible to completely prohibit even four divisions in the child block. As a result, it is possible to prevent the block from being divided in the same direction as that of the parent block, so that it is possible to prevent block division into a rectangle that is elongated in the lateral direction, and the encoding/decoding processing is facilitated.

The process shown in the flowchart of FIG. 10 is executed for each block divided into two in the horizontal direction. The inside of the block divided into two is also encoded and decoded in the order from top to bottom.

Next, the processing of each divided block when the tree block is divided into two in the vertical direction will be described using the flowchart of FIG.

When the tree block is divided into two in the vertical direction, each of the divided two blocks first determines whether the inside of the block is divided into four in the horizontal and vertical directions (S1301).

When it is determined that the inside of the block is divided into four (S1301: Yes), the inside of the block is divided into four.
Each block is divided (S1302), and each block is divided into four blocks in the horizontal and vertical directions (S13).
03).

When it is determined that the inside of the block is not divided into four (S1301: No), it is determined whether or not the inside of the block is divided into two again (S1304).

When it is determined that the block is divided into two again (S1304: Yes), the inside of the block is divided in the horizontal direction (S1305), and each processing of the block divided into two in the horizontal direction is performed (S1306).

When it is determined that the block is not divided into two again (S1304: No), the inside of the block is not re-divided and the block division processing is ended (S1307).

FIG. 13 shows a state of subdivision of a divided block when the treeblock is divided into two in the vertical direction. Here, when the tree block, which is the parent block, is divided into two in the vertical direction, the subdivision of the divided block allows only two in the horizontal direction, and automatically divides into two in the horizontal direction. Further, when the tree block, which is the parent block, is divided into two, it is possible to completely prohibit even four divisions in the child block. As a result, it is possible to prevent the block from being divided in the same direction as the parent block, so that it is possible to prevent the block from being divided into a vertically elongated rectangle, which facilitates the encoding/decoding process.

The process shown in the flowchart of FIG. 12 is executed for each block vertically divided into two. The inside of the divided block is also encoded and decoded in the order of left and right.

Although the re-division of the divided block when the tree block is divided has been described, the parent block may not be the tree block. For example, a tree block (128x1
28) is divided into four, and the block (64×64) divided into four is further divided into four or into two, the above processing is applied to the division of the re-divided block.

Next, the operation of the block division unit 202 of the image decoding device 200 will be described. Blocks are divided by the same processing procedure as the block division unit 101 of the image encoding device 100, but the block division unit 101 of the image encoding device 100 selects a block division pattern and outputs the selected block division information. On the other hand, the block division unit 202 of the image decoding device
In a situation where a block is divided using the block division information decoded from the coded bitstream and when the block division information is decoded from the coded bitstream, redivision in the same direction is prohibited. The difference is that information with no options has a syntax structure that is not transmitted in the bitstream.

FIG. 14 shows an example of syntax (syntax rule of coded bitstream) related to block division according to the first embodiment. For the internal division of the tree block, first, a flag (4_division_flag) as to whether or not to divide into four is transmitted and received. When dividing into 4 (4_division_flag is 1)
Ends the processing by dividing the tree block into four. After that, the inside of the block divided into four is again divided by the syntax shown in FIG. When not divided into 4 (4_division_f
When the lag is 0), a flag (2_division_flag) indicating whether to divide into two is transmitted and received. When dividing into two (2_division_flag is 1), a flag (2_division_d) indicating the direction of further dividing into two.
irection). If 2_division_direction is 1, it indicates division in the vertical direction, and 2
When _division_direction is 0, it indicates horizontal division. Thereafter, the inside of the block is divided again with the syntax shown in FIG. 14 for the divided two blocks. If not divided into two (2_division_flag is 0), the process is terminated without dividing the tree block.

Here, a process of re-dividing the inside of a block divided into four or two will be described.
The syntax shown in FIG. 14 is also used for the process of redistributing the inside of the block, but the difference is that the splitting direction is limited when the treeblock is split into two, compared to when the treeblock is split. That is, in the case where the tree block is divided into two, when the inside of the divided block is redone, it is prohibited to divide the tree block in the same direction as the dividing direction.
As a result, it is possible to prevent the divided blocks from becoming more elongated rectangles and prevent an increase in the memory band required for intra prediction and inter prediction. Details of preventing the increase of the memory band will be described later.

Further, it is of course possible to count the number of times divided into two in the same direction and restrict the division in the same direction when the number of times exceeds a predetermined number. For example, two divisions in the same direction are permitted up to two times, but two divisions in the same direction are prohibited from the third time.

In FIG. 14, the 4 divisions are selected with priority, and the information indicating whether or not the 4 divisions are performed is a syntax that is transmitted and received before the information indicating whether or not the 2 divisions are performed. On the other hand, in the case of preferentially selecting the two divisions, it is possible to use the syntax of transmitting or receiving the information of whether or not to divide into two before the information of whether or not to divide into four. This is because the amount of codes to be transmitted as a bit stream is smaller when an event that is more likely to occur stochastically is transmitted and received first. That is, 4 divisions and 2 in advance
Which of the divisions is more likely to occur is estimated, and the division information that is more likely to occur may be used as the syntax for transmission and reception. For example, the encoding device adaptively determines the number of priority divisions having a high encoding efficiency by transmitting and receiving whether to prioritize 4 divisions or 2 divisions in the header information of the image. It is also possible to divide the inside of the tree block with a syntax based on the selected priority division number.

In the image encoding device 100 and the image decoding device 200, intra prediction and inter prediction are performed using the divided blocks. Both intra and inter prediction involve copying pixels from memory.

15A to 15D show an example of intra prediction. 15(a) and 15(
b) shows the prediction direction and mode number of intra prediction. As shown in FIGS. 15C and 15D, intra prediction is performed by performing coding/decoding by copying pixels from coded/decoded pixels that are close to the coding/decoding target block. Generate a predicted image of the target block. In intra prediction, since the prediction image generation and the encoding/decoding pixel generation are repeated in block units, the processing order becomes sequential in block units, and the smaller the inside of the block is, the larger the load of the entire process becomes. In addition, the processing of pixel copy from the memory increases as the shape of the block becomes longer and narrower. In addition, since the residual signal is subjected to orthogonal transformation for encoding/decoding, the number of types of orthogonal transformation required increases as the size of the rectangle increases, resulting in an increase in circuit scale. Therefore, when dividing the inside of a block into two,
By limiting the division into two in the same direction as the parent block division method, it is possible to prevent an increase in the memory bandwidth required for intra prediction.

FIG. 16 shows an example of inter prediction. In inter-prediction, a predicted image of an encoding/decoding target block is generated by copying pixels in block units from pixels included in an encoded/decoded image. In inter prediction, when a pixel is copied in block units from a reference image, it is often a device configuration that requires acquisition in a management unit of a memory that includes a required pixel. Therefore, as the block is divided into smaller blocks, and the shape of the block becomes longer and narrower, the overall processing load increases. In addition, when performing decimal precision motion compensation using an interpolation filter for a reference image, it is necessary to copy pixels obtained by adding several pixels to the pixels included in the block, and the smaller the block size, The relative ratio of several pixels to be added becomes large, and the overall processing load becomes large. Therefore, when the inside of the block is divided into two, it is possible to prevent an increase in the memory band required for inter prediction by limiting the division into the same direction as the division direction of the parent block.

Next, the relationship between the luminance signal and the color difference signal in intra prediction will be described. As a format between the luminance signal and the color difference signal, 4:2:0, 4:2:2, 4:4:4 and the like have been conventionally known. In the 4:2:0 format shown in FIG. 17A, the luminance signal samples two pixels in both the horizontal and vertical directions, whereas the chrominance signal samples one pixel in both the horizontal and vertical directions. Since the human eye can perceive the luminance signal more sensitively than the color difference signal, the amount of information of the color difference signal is smaller than that of the luminance signal. In the 4:2:2 format shown in FIG. 17B, the luminance signal samples two pixels in the horizontal direction, while the color difference signal samples one pixel in the horizontal direction. In the vertical direction, the luminance signal is sampled in two pixels in the vertical direction, whereas the color difference signal is sampled in two pixels in the vertical direction. FIG. 17(c)
In the 4:4:4 format shown in (1), the luminance signal is sampled in two pixels in the horizontal/vertical directions, whereas the chrominance signal is sampled in two pixels in the horizontal/vertical directions.

This embodiment will be described by taking the 4:2:0 format most widely used in image coding as an example. The block division units 101 and 202 include a luminance block division unit that divides an image luminance signal to generate a luminance block and a color difference block division unit that divides an image color difference signal to generate a color difference block. In, the luminance signal and the color difference signal are independently divided into blocks. That is, in intra prediction, the size of the luminance block and the size of the color difference block are independently determined. In intra prediction, since the luminance signal and the color difference signal each copy the pixel value from the peripheral pixels, the prediction efficiency is improved when the luminance signal and the color difference signal are independently divided into blocks. On the other hand, for inter prediction, the luminance signal and the color difference signal are treated together and divided into blocks. That is, in inter prediction, the size of the luminance block and the size of the color difference block are the same. This is because it is not necessary to distinguish luminance and color difference in motion compensation in inter prediction.

The prediction image generation units 102 and 204 include a luminance signal prediction unit that predicts a luminance signal and a color difference signal prediction unit that predicts a color difference signal, and the color difference signal prediction unit increases the prediction efficiency of the color difference signal in intra prediction. , Luminance-chrominance intra prediction for predicting a color-difference signal from pixels that have been encoded/decoded for a luminance signal. In luminance/chrominance intra prediction, a primary signal is encoded/decoded before a secondary signal, and the secondary signal is predicted using the encoded/decoded primary signal. Here, in the 4:2:0 format and the 4:2:2 format, since the information amount of the luminance signal is larger than that of the color difference signal, the luminance signal is used as the primary signal and the color difference signal is used as the secondary signal. In the 4:4:4 format, the information amounts of the luminance signal and the color difference signal are the same, but it is common to use the luminance signal as the primary signal and the color difference signal as the secondary signal according to other formats.

FIG. 18 is a diagram for explaining the luminance/color difference intra prediction, and FIG. 19 is a flowchart for explaining the luminance/color difference intra prediction.

As shown in FIG. 18, the luminance/color difference intra prediction is performed by encoding/decoding peripheral pixels 12 a and 12 b of the luminance block 10 and encoding/decoding peripheral pixels 16 a of the color difference block 14.
16b based on the degree of correlation with 16b. Luminance color difference intra prediction is for predicting color difference signals, and therefore the peripheral pixels for which the degree of correlation is calculated are defined with reference to the peripheral pixels of the color difference block to be encoded/decoded. That is, the peripheral pixels of the luminance block located at the same position as the peripheral pixels determined for the color difference block are targets for calculating the degree of correlation.

First, the degree of correlation between the peripheral pixels of the encoded/decoded luminance signal and the peripheral pixels of the color difference signal is calculated (S1901). Subsequently, the encoded/decoded luminance signal of the encoding/decoding target block is downsampled (S1902). Here, a plurality of filter types to be downsampled may be prepared so that the filter type can be selected. For example, filters having different intensities may be prepared and a plurality of filter types may be selected, or filters having different tap numbers may be prepared and a plurality of filter types may be selected. The filter type may be automatically selected using the degree of correlation between neighboring pixels, or the filter type may be encoded/decoded in the bitstream and transmitted. Also, unless the downsampling filter of the luminance signal of the encoding/decoding target block is determined using the degree of correlation between neighboring pixels, the processes of step S1901 and step S1902 are out of order, and step S1901
And step S1902 can be processed in parallel.

Finally, a color difference signal is predicted from the downsampled luminance signal based on the degree of correlation between peripheral pixels (S1903). In the case of the 4:2:0 format, the down sample becomes 1/2 in the horizontal/vertical direction. In the case of the 4:2:2 format, it becomes 1/2 in the horizontal direction and is not downsampled in the vertical direction. In the case of 4:4:4 format, down sampling is not performed in the horizontal/vertical directions.

In the luminance/chrominance intra prediction, the chrominance signal prediction process can be started after the luminance/signal decoding of the luminance signal of the encoding/decoding target block is completed. Therefore, the timing when the prediction process of the color difference block can be started depends on the size of the luminance block and the size of the color difference block.

FIG. 20A and FIG. 20B are diagrams for explaining the luminance/color difference intra prediction when the size of the color difference block is larger than the size of the luminance block. The number of pixels of the first to fourth luminance blocks 20a, 20b, 20c, 20d divided into four shown in FIG. 20(a) is 16×16, and the number of pixels of the color difference block 20e shown in FIG. 20(b) is 16×16. ..

Here, the comparison of the size of the luminance block and the size of the color difference block is not the comparison in the number of pixels in the block but the comparison in the area considering the color difference format. That is, in the 4:2:0 format, the area occupied by the luminance block is 1/2 of the area occupied by the color difference block, so that the number of pixels of the luminance block is 16x16 and the number of pixels of the color difference block is 16x16. , The size of the luminance block is smaller. In the 4:2:0 format, when the number of pixels of the luminance block is 16x16 and the number of pixels of the color difference block is 8x8, the areas occupied by both blocks are the same, and the luminance block and the color difference block have the same size. is there.

To compare the size of the luminance block and the color difference block not by comparing the area of the block but by comparing the number of pixels in the block, the number of pixels in the color difference block is determined by the ratio of the luminance signal and the color difference signal in the color difference format. It can be converted into the number of pixels. In the case of the 4:2:0 format, the number of pixels of the luminance signal is twice the number of pixels of the color difference signal, so the number of vertical and horizontal pixels of the color difference block is doubled to be converted into the number of vertical and horizontal pixels of the luminance block. For example, in the 4:2:0 format, if the size of the luminance block is 16x16 and the size of the chrominance block is 16x16, the converted size of the chrominance block converted to the number of pixels of the luminosity block is 32x32. You can see that the size is larger.

In the intra prediction, after the decoding process is completed in block units, the prediction process of the subsequent blocks becomes possible. That is, after the decoding of the first luminance block 20a is completed, the second luminance block 2
0b can be decoded, the second luminance block 20b can be decoded, the third luminance block 20c can be decoded, and the fourth luminance block 20d can be decoded. Decryption is possible.

When the size of the color difference block is larger than the size of the luminance block, the color difference block 20e
The peripheral pixels necessary for the prediction process of 4 are four luminance blocks 20a for both luminance pixels and chrominance pixels.
20b, 20c, and 20d exist before decoding, and four luminance blocks 20a, 20b,
The degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the color difference signal in step S1901 of FIG. 19 can be calculated without waiting for the decoding of 20c and 20d.

Next, after the decoding of the first luminance block 20a is completed, the luminance signal down-sampling in step S1902 of FIG. 19 is executed. The pixel of the color difference block 20e corresponding to the position of the first luminance block 20a can be predicted without waiting for the completion of decoding of the second luminance block 20b. Similarly,
After the decoding of the second luminance block 20b is completed, the luminance signal is downsampled, and the pixel of the color difference block 20e corresponding to the position of the second luminance block 20b is predicted without waiting for the completion of the decoding of the third luminance block 20c. .. Further, after the decoding of the third luminance block 20c is completed, the luminance signal is downsampled, and the pixels of the color difference block 20e corresponding to the position of the third luminance block 20c are not waited for until the decoding of the fourth luminance block 20d is completed. Predict. Finally, after the decoding of the fourth luminance block 20d is completed, downsampling of the luminance signal is performed, and the fourth luminance block 2
The pixel of the color difference block 20e corresponding to the position of 0d is predicted.

FIG. 21A and FIG. 21B are diagrams for explaining the luminance/color difference intra prediction when the size of the color difference block is smaller than the size of the luminance block. The number of pixels of the luminance block 21a shown in FIG. 21(a) is 16×16, and the number of pixels of the first to fourth color difference blocks 21b, 21c, 21d, 21e divided into four shown in FIG. 21(b) is 4×4. ..

In the 4:2:0 format, the area occupied by the luminance block is 1/2 of the area occupied by the color difference block. Therefore, if the number of pixels of the luminance block is 16x16 and the number of pixels of the color difference block is 4x4, The block size is smaller. To compare the size of the luminance block and the color difference block by comparing the number of pixels in the block instead of comparing the area of the block, in the case of 4:2:0 format, double the number of vertical and horizontal pixels of the color difference block. Then, it is converted into the number of vertical and horizontal pixels of the luminance block. In the 4:2:0 format, when the size of the luminance block is 16x16 and the size of the color difference block is 4x4, the converted size of the color difference block converted into the number of pixels of the brightness block is 8x8, and the size of the color difference block is You can see that it is smaller.

If the size of the color difference block is smaller than the size of the luminance block, the first color difference block 2
The peripheral pixels of 1b can be used for both the luminance pixel and the color difference pixel before the decoding of the luminance block 21a, but the peripheral pixels of the second to fourth color difference blocks 21c, 21d, and 21e are the luminance block 21.
It cannot be used unless the decryption of a is completed. That is, when the decoding of the luminance block 21a is completed and the decoding of the first color difference block 21b is not completed, FIG.
In step S1901, the degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the color difference signal cannot be calculated. Similarly, for the third color difference block 21d, if the decoding of the luminance block 21a is completed and the decoding of the first and second color difference blocks 21b and 21c is not completed, the third color difference block 2
For 1d, the degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the color difference signal cannot be calculated. Similarly, for the fourth color difference block 21e, decoding of the luminance block 21a is completed, and
~ If the decoding of the third color difference blocks 21b, 21c, 21d is not completed, the fourth color difference block 2
For 1e, the degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the color difference signal cannot be calculated.

As described above, when the size of the color difference block is smaller than the size of the luminance block, when the luminance color difference intra prediction is performed, there is a processing dependency relationship between the luminance block and the color difference blocks and between the color difference blocks in the color difference block prediction processing. Not suitable for parallel processing. Therefore, when the size of the color difference block is smaller than the size of the luminance block, the luminance color difference intra prediction is limited. As a method of limiting the luma color difference intra prediction, (1) the syntax is used, (
There are 2) replacement of the intra-color difference mode, (3) replacement of peripheral pixels, and the like.

FIG. 22 shows an example of the syntax of the intra color difference prediction mode. When the color difference prediction mode number is 0, the same intra prediction mode as the luminance prediction mode is used for the color difference prediction mode. For example, when the luminance prediction mode is the horizontal prediction mode, the color difference prediction mode is also the horizontal prediction mode. When the color difference prediction mode number is 1, the average value mode (DC mode) is used. In the DC mode, intra prediction is performed using the average value of peripheral pixels. When the color difference prediction mode number is 2, the luminance color difference intra prediction mode is used.

As a method of limiting the luminance/chrominance intra prediction, when the (1) syntax is used, mode 2 is prohibited because it represents the luminance/color difference intra prediction. That is, mode 2 is not transmitted, and the intra color difference mode is selected from mode 0 and mode 1.

As a method of limiting the luma color difference intra prediction, (2) when replacing the intra color difference mode, when the color difference prediction mode number is designated as mode 2, the luma color difference intra prediction mode is used instead of the vertical prediction mode. To do. However, the replacement prediction mode is not limited to the vertical prediction mode and may be another prediction mode. Further, it is more preferable that the luminance prediction mode of mode 0 and the mode to be replaced by mode 2 are the same so that the modes to be used do not overlap.

As a method of limiting the luminance/color difference intra prediction, (3) in the case of replacing the peripheral pixels,
In step S1901 of 9, the peripheral pixels for calculating the degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the color difference signal are replaced. The second to fourth color difference blocks 21c, 21d of FIG.
Also in 21e, the degree of correlation of peripheral pixels can be calculated without waiting for the decoding of the luminance block 21a.

FIG. 23 shows an example of replacement of peripheral pixels. Although the pixels in the first color difference block 21b are normally used as the peripheral pixels on the left side of the second color difference block 21c, the pixels in the first color difference block 21b are used in order to use the pixels in the luminance block 21a and the first color difference block 21b. It is necessary to wait for the completion of decryption. Therefore, the usable area 21f is used as a peripheral pixel of the second color difference block 21c without waiting for the completion of decoding of the luminance block 21a. Similarly, for the third color difference block 21d, the usable area 21f is used as a peripheral pixel of the third color difference block 21d without waiting for the completion of decoding of the luminance block 21a. Similarly, for the fourth color difference block 21e,
The area 21f that can be used without waiting for the completion of decoding of the luminance block 21a is set to the fourth color difference block 21.
It is used as a peripheral pixel of e.

As described above, in the first embodiment, when the size of the color difference block is smaller than the size of the brightness block when performing the brightness color difference intra prediction, the brightness color difference block and the color difference block are limited by limiting the brightness color difference intra prediction. It becomes possible to relax the processing dependency between them. As a result, the luminance block and the color difference block can be processed in parallel, and the processing amount of encoding/decoding can be reduced.

(Second embodiment)
A second embodiment of the present invention will be described. In the second embodiment, it is possible to limit the luminance/color-difference intra prediction by independently evaluating the luminance block size and the color-difference block size independently of the size relationship between the luminance block and the color-difference block. Unlike the embodiment, the other configurations and operations are the same as those of the first embodiment.

First, the limitation based on the size of the luminance block will be described. 24(a) to 24(
In c), the luminance/color difference intra prediction due to the difference in the size of the luminance block is shown. FIG. 24(a),
As shown in FIG. 24B, the first luminance block 24a corresponds to the position of the first color difference block 24e, the second luminance block 24b corresponds to the position of the second color difference block 24f, and the third luminance block 24c is Corresponding to the position of the third color difference block 24g, the fourth luminance block 24d corresponds to the fourth position.
It corresponds to the position of the color difference block 24h. The luminance block 24i in FIG. 24C corresponds to the positions of the first to fourth color difference blocks 24e, 24f, 24g, and 24h in FIG. 24B.

The dependency relationship between the luminance block and the color difference block will be described. When the size of the luminance block is small as shown in FIG. 24A, when the decoding of the first luminance block 24a is completed, the decoding of the first color difference block 24e becomes possible. When the decoding of the second luminance block 24b is completed, the second luminance block 24b
The color difference block 24f can be decoded. When the decoding of the third luminance block 24c is completed,
It becomes possible to decode the third color difference block 24g. When the decoding of the fourth luminance block 24d is completed, the decoding of the fourth color difference block 24h becomes possible.

On the other hand, when the size of the luminance block is large as shown in FIG. 24C, the luminance block 24i
If the decoding of No. 1 is not completed, all the decoding of the first to fourth color difference blocks 24e, 24f, 24g, 24h will not be possible.

When the division of the luminance block and the division of the color difference block are independently determined, if the absolute size of the luminance block is large, the size of the color difference block is more likely to be smaller than the size of the luminance block. Therefore, when the absolute size of the luminance block is equal to or larger than the predetermined size, the luminance color difference intra prediction of the corresponding color difference block is limited.

Similarly, if the absolute size of the color difference block is small, the size of the color difference block is more likely to be smaller than the size of the luminance block. Therefore, when the absolute size of the color difference block is equal to or smaller than the predetermined size, the luminance color difference intra prediction of the color difference block is limited.

The method of limiting the luminance/color difference intra prediction is the same as that in the first embodiment.

As described above, in the second embodiment, the luminance/color difference intra prediction is limited when the absolute size of the luminance block is larger than the threshold value or when the absolute size of the color difference block is smaller than the threshold value. As a result, it is possible to predict when the size of the color difference block is smaller than the size of the luminance block, limit the luminance color difference intra prediction, and stochastically relax the processing dependency between the luminance block and the color difference block. become.

(Third Embodiment)
A third embodiment of the present invention will be described. The third embodiment is different from the first embodiment in that the block division unit 101 divides the color difference blocks so that the size of the color difference blocks does not become smaller than the size of the luminance blocks, and other configurations are provided. The operation is the same as that of the first embodiment. When dividing the color difference block, the block division unit 101 prohibits division so that the size of the color difference block is smaller than the size of the luminance block. This prevents the size of the color difference block from becoming smaller than the size of the brightness block in the brightness/color difference intra prediction, and always enables parallel processing of the brightness block and the color difference block.

The encoded bitstream of the image output by the image encoding device according to the above-described embodiment is
It has a specific data format so that it can be decoded according to the encoding method used in the embodiment, and the image decoding device corresponding to the image encoding device encodes this specific data format. The encoded bitstream can be decoded.

When a wired or wireless network is used to exchange the encoded bitstream between the image encoding device and the image decoding device, the encoded bitstream is converted into a data format suitable for the transmission mode of the communication path. May be transmitted. In that case, a transmission device that converts the encoded bit stream output by the image encoding device into encoded data in a data format suitable for the transmission mode of the communication path and transmits the encoded data to the network, and receives the encoded data from the network. And a receiver for restoring the encoded bitstream and supplying it to the image decoder.

The transmission device includes a memory that buffers the encoded bitstream output from the image encoding device, a packet processing unit that packetizes the encoded bitstream, and a transmission unit that transmits the packetized encoded data via a network. Including and The reception device receives a packetized coded data via a network, a memory that buffers the received coded data, and packetizes the coded data to generate a coded bitstream, And a packet processing unit provided to the image decoding apparatus.

In addition, by adding a display unit that displays the image decoded by the image decoding device to the configuration,
It can also be a display device. In that case, the display unit reads the decoded image signal generated by the decoded image signal superimposing unit 205 and stored in the decoded image memory 206, and displays it on the screen.

It is also possible to make an image pickup device by adding an image pickup unit to the configuration and inputting the picked-up image to the image coding device. In that case, the imaging unit inputs the captured image signal to the block division unit 101.

The above-mentioned processing relating to encoding and decoding can be realized not only as a transmission, storage and reception device using hardware, but also as stored in a ROM (Read Only Memory), a flash memory or the like. It can also be realized by existing firmware or software such as a computer. Recording the firmware program or software program in a computer-readable recording medium and providing it, providing it from a server through a wired or wireless network, or providing it as a terrestrial or satellite digital broadcast data broadcast. Is also possible.

The present invention has been described above based on the embodiment. It is understood by those skilled in the art that the embodiments are exemplifications, that various modifications can be made to the combinations of the respective constituent elements and the respective processing processes, and that the modifications are within the scope of the present invention. ..

100 image coding apparatus, 101 block division unit, 102 predicted image generation unit,
103 residual signal generation unit, 104 orthogonal transformation/quantization unit, 105 coded bit string generation unit, 106 inverse quantization/inverse orthogonal transformation unit, 107 decoded image signal superposition unit, 108
Decoded image memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block division unit, 203 Dequantization/inverse orthogonal transformation unit, 204 Predicted image generation unit, 20
5 Decoded image signal superimposing unit, 206 Decoded image memory

Claims (6)

An image encoding device that divides an image into blocks and encodes each divided block,
A primary signal block dividing unit that generates a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size,
A secondary signal block division unit that generates a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal predictor for predicting a primary signal,
And a secondary signal prediction unit that predicts a secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction unit is capable of inter-component prediction to predict the secondary signal from the encoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image encoding device characterized by the above. An image encoding method for dividing an image into blocks and performing encoding in units of divided blocks,
A primary signal block dividing step of generating a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size;
A secondary signal block dividing step of generating a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal prediction step of predicting a primary signal,
A secondary signal prediction step of predicting a secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction step, inter-component prediction is possible to predict a secondary signal from the encoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image coding method characterized by the above. An image coding program that divides an image into blocks and performs coding on a divided block basis,
A primary signal block dividing step of generating a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size;
A secondary signal block dividing step of generating a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal prediction step of predicting a primary signal,
Let the computer perform the secondary signal prediction step of predicting the secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction step, inter-component prediction is possible to predict a secondary signal from the encoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image coding program characterized by the above. An image decoding device that performs decoding in block units obtained by dividing an image,
A primary signal block dividing unit that generates a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size,
A secondary signal block division unit that generates a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal predictor for predicting a primary signal,
And a secondary signal prediction unit that predicts a secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction unit is capable of inter-component prediction to predict a secondary signal from the decoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image decoding device characterized by the above. An image decoding method for performing decoding in block units obtained by dividing an image,
A primary signal block dividing step of generating a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size;
A secondary signal block dividing step of generating a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal prediction step of predicting a primary signal,
A secondary signal prediction step of predicting a secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction step, inter-component prediction is possible to predict a secondary signal from the decoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image decoding method characterized by the above. An image decoding program that decodes an image in block units,
A primary signal block dividing step of generating a primary signal block by dividing the primary signal of the image into rectangles of a predetermined size;
A secondary signal block dividing step of generating a secondary signal block by dividing the secondary signal of the image into rectangles of a predetermined size,
A primary signal prediction step of predicting a primary signal,
Let the computer perform the secondary signal prediction step of predicting the secondary signal,
The primary signal block and the secondary signal block are each independently divided,
The primary signal is a luminance signal, the secondary signal is a color difference signal,
The secondary signal prediction step, inter-component prediction is possible to predict a secondary signal from the decoded primary signal, if the size of the primary signal block is a predetermined size or more, prohibit the use of the inter-component prediction,
An image decoding program characterized by the above.

Images