WO2020255688A1 - Dispositif de codage d'image, dispositif et procédé de décodage d'image et programme - Google Patents

Dispositif de codage d'image, dispositif et procédé de décodage d'image et programme Download PDF

Info

Publication number
WO2020255688A1
WO2020255688A1 PCT/JP2020/021614 JP2020021614W WO2020255688A1 WO 2020255688 A1 WO2020255688 A1 WO 2020255688A1 JP 2020021614 W JP2020021614 W JP 2020021614W WO 2020255688 A1 WO2020255688 A1 WO 2020255688A1
Authority
WO
WIPO (PCT)
Prior art keywords
subblock
attention
prediction
image
basic block
Prior art date
Application number
PCT/JP2020/021614
Other languages
English (en)
Japanese (ja)
Inventor
真悟 志摩
Original Assignee
キヤノン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by キヤノン株式会社 filed Critical キヤノン株式会社
Publication of WO2020255688A1 publication Critical patent/WO2020255688A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • the present invention relates to an image coding device, an image decoding device, a method, and a program.
  • HEVC High Efficiency Video Coding
  • HEVC High Efficiency Video Coding
  • a basic block having a size larger than that of a conventional macroblock (16 ⁇ 16 pixels) has been adopted.
  • This large-sized basic block is called a CTU (Coding Tree Unit), and its size is a maximum of 64 ⁇ 64 pixels.
  • the CTU is further divided into sub-blocks that serve as units for prediction and conversion.
  • Patent Document 1 discloses a technique for predicting such a motion vector.
  • VVC Very Video Coding
  • VVC is also considering the introduction of prediction technology using encoded block pixels and motion vectors.
  • the pixels and motion vectors of blocks adjacent to the left and above, upper left and upper right of the block to be encoded are used, but especially when parallel processing is performed in block column units, some blocks There were cases where pixels and motion vectors could not be referenced. As a result, such prediction processing may hinder the implementation of parallel processing.
  • the present invention has been made to solve the above-mentioned problems, and an object of the present invention is to reduce the complexity of implementation by limiting the blocks that can be used for prediction processing to some blocks.
  • the image coding apparatus of the present invention has the following configuration. That is, An image coding device that encodes an image.
  • a division means for dividing a frame image to be encoded into basic blocks of a predetermined size and further dividing the basic block into a plurality of subblocks. It has a prediction means for executing intra-prediction or inter-prediction for the subblocks obtained by the division means.
  • the prediction means is In the case where the intra prediction is executed for the attention subblock in the attention basic block, and the subblock located at the upper right of the attention subblock is in the basic block located at the upper right of the attention basic block.
  • the subblock located at the upper right of the attention subblock is always excluded from the reference target in the intra prediction for the attention subblock, and the intra prediction is performed.
  • the complexity of implementation can be reduced by limiting the blocks that can be used for prediction processing to some blocks depending on the positional relationship between the blocks and adjacent blocks.
  • the block block diagram of the image coding apparatus in 1st Embodiment The block block diagram of the image decoding apparatus in 1st Embodiment.
  • a block of interest such as a block to be processed at present is referred to as a block of interest.
  • the basic block to be processed at present is called a attention block.
  • the subblock to be processed at present is referred to as a attention subblock.
  • restrictions on subblocks that can be used in intra-prediction and restrictions on sub-blocks that can be selected as motion vector predictors in inter-prediction will be described. Only one of these restrictions may be applied, or both may be applied. Even when only one of them is applied, it is possible to contribute to the reduction of implementation complexity and the reduction of memory usage.
  • FIG. 1 is a block composition of the image coding apparatus of this embodiment.
  • the image coding device has a control unit 150 that controls the entire device.
  • the control unit 150 includes a CPU, a ROM for storing a program executed by the CPU, and a RAM used as a work area of the CPU.
  • the image coding apparatus includes an input terminal 101, a block division unit 102, a prediction unit 103, a conversion / quantization unit 104, an inverse quantization / inverse conversion unit 105, an image reproduction unit 106, a frame memory 107, and an in-loop filter unit. It has 108, a coding unit 109, an integrated coding unit 110, an output terminal 111, and a motion vector memory 112.
  • the input terminal 101 inputs the image data to be encoded supplied from the image data supply unit (not shown) in frame units.
  • the image data generation source may be of any type, such as an image pickup device that generates image data to be encoded, a file server or a storage medium that stores the image data to be encoded.
  • the output terminal 112 outputs the coded data to the output destination device, but the output destination device is not particularly limited to a storage medium, a file server, or the like.
  • the output terminal 112 may be a network interface, or may output encoded data to the outside via the network.
  • the block division unit 102 divides the input frame image into a plurality of basic blocks, and outputs the image data of each basic block unit to the prediction unit 103 in the subsequent stage. Specifically, the block division unit 102 outputs the image data of the basic block to the prediction unit 103 in the order of raster scan with the basic block as a unit.
  • the prediction unit 103 divides the image data in basic block units into sub-blocks, performs intra-frame prediction, which is intra-frame prediction, inter-prediction, which is inter-frame prediction, and the like, and generates prediction image data. .. Further, the prediction unit 103 calculates and outputs a prediction error from the attention subblock and the prediction image data in the input image data. For example, the prediction error is the difference between the attention subblock and the prediction image data of the attention subblock. Further, the prediction unit 103 also outputs information necessary for prediction, for example, information such as sub-block division, prediction mode, motion vector, and the like together with the prediction error. Hereinafter, the information necessary for this prediction is referred to as prediction information. This prediction information will be used for prediction in other blocks and for generating prediction image data on the decoding side.
  • the conversion / quantization unit 104 obtains a conversion coefficient by orthogonally converting the prediction error input from the prediction unit 103 in units of subblocks. Further, the conversion / quantization unit 104 performs quantization on the obtained conversion coefficient to obtain a quantization coefficient. It should be noted that the function of performing orthogonal conversion and the function of performing quantization may be configured separately.
  • the inverse quantization / inverse conversion unit 105 inversely quantizes the quantization coefficient output from the conversion / quantization unit 104 to reproduce the conversion coefficient, and further performs inverse orthogonal conversion to reproduce the prediction error.
  • the function of performing inverse quantization and the function of performing inverse orthogonal transformation may be configured separately.
  • the image reproduction unit 106 Based on the prediction information output from the prediction unit 103, the image reproduction unit 106 generates the prediction image data by appropriately referring to the frame memory 107, and generates the reproduction image data from the generated prediction image data and the input prediction error. Generate and store in frame memory 107. For example, the image reproduction unit 106 reproduces the reproduced image data by adding a prediction error to the predicted image data.
  • the in-loop filter unit 108 performs in-loop filter processing such as a deblocking filter and a sample adaptive offset on the reproduced image data stored in the frame memory 107, and re-stores the filtered image data in the frame memory 107. ..
  • the motion vector memory 112 holds the motion vector of the encoded subblock. Therefore, the motion vector memory 112 holds the motion vector in the prediction information output by the prediction unit 103 in association with the position of the subblock to be encoded.
  • the coding unit 109 encodes the quantization coefficient output from the conversion / quantization unit 104 and the prediction information output from the prediction unit 103, generates code data, and outputs the code data.
  • the integrated coding unit 110 generates header code data encoded in the header part of the sequence or picture, which is necessary for decoding the bit stream on the decoding side. Further, the integrated coding unit 110 forms a bit stream in which the code data output from the coding unit 109 is integrated with the header code data, and outputs the bit stream via the output terminal 111.
  • moving image data is input in frame units.
  • the block division unit 102 will be described as being divided into basic blocks of 64 ⁇ 64 pixels, but the present invention is not limited to this.
  • the size of the basic block may be 128 ⁇ 128 pixels.
  • a notation such as 64 x 64 pixels indicates that the height of the block in the vertical direction is 64 pixels and the width in the horizontal direction is 64 pixels.
  • the notation 32 ⁇ 64 pixels indicates that the height of the block in the vertical direction is 32 pixels and the width in the horizontal direction is 64 pixels.
  • the same notation shall indicate the vertical height and the horizontal width of the block as in the above example.
  • the image data for one frame input via the input terminal 101 is supplied to the block dividing unit 102.
  • the block division unit 102 divides the input image data into a plurality of basic blocks, and outputs the image data in basic block units to the prediction unit 103. In this embodiment, it is assumed that an image of a basic block unit of 64 ⁇ 64 pixels is output.
  • the prediction unit 103 executes prediction processing on the image data of the basic block unit input from the block division unit 102. Specifically, the prediction unit 103 determines the sub-block division that divides the basic block into smaller sub-blocks, and further refers to the motion vector memory 112 and the frame memory 107 in sub-block units for intra-prediction and inter-prediction. Determine the prediction mode such as.
  • FIGS. 7A to 7F An example of the subblock division method is shown with reference to FIGS. 7A to 7F.
  • the thick frames of blocks 700 to 705 in FIGS. 7A to 7F represent basic blocks (64 ⁇ 64 pixels in the embodiment), and each quadrangle divided by a thin line inside thereof represents a subblock.
  • FIG. 7A is an example in which the basic block 700 is a subblock. That is, this is an example in which the basic block is not divided and the size of the subblock is 64 ⁇ 64 pixels.
  • FIG. 7B shows a division example in which the basic block 701 is divided into four square sub-blocks, and the size of one sub-block is 32 ⁇ 32 pixels.
  • 7C to 7F show an example of rectangular subblock division.
  • FIG. 7C shows that the basic block 702 is divided into two sub-blocks (longitudinal in the vertical direction) having a size of 32 ⁇ 64 pixels.
  • FIG. 7D shows that the basic block 703 is divided into two subblocks (horizontally longitudinal) having a size of 64 ⁇ 32 pixels.
  • the basic blocks 704 and 705 of FIGS. 7E and 7F although the division directions are different, they are divided into three rectangular subblocks at a ratio of 1: 2: 1. In this way, not only squares but also rectangular sub-blocks are used for encoding processing.
  • the basic block 701 of 64 ⁇ 64 pixels is divided into quadtrees and divided into four sub-blocks of 32 ⁇ 32 pixels.
  • the subblock division method is not limited to this.
  • a binary tree division as shown in FIGS. 7C and 7D, a ternary tree division as shown in FIGS. 7E and 7F, or an undivided tree as shown in FIG. 7A may be used.
  • the prediction process by the prediction unit 103 will be described in detail.
  • image coding technology such as HEVC
  • the pixels of the encoded subblock are used to encode the subblock to be encoded.
  • the process of predicting the pixels of In the prediction process intra-prediction using pixels of the coded subblock in the frame in which the subblock to be encoded exists, or a sub of the coded frame different from the frame in which the subblock to be coded exists.
  • inter-predictions that use block pixels. Also in this embodiment, these two types of prediction methods are used.
  • the prediction unit 103 When performing intra prediction, the prediction unit 103 is spatially located in the periphery of the subblock to be encoded, and uses the encoded pixels to generate prediction pixels of the subblock to be encoded (hereinafter, hereinafter, The subblock to be encoded is also called a subblock of interest, and the basic block including the subblock to be encoded is also called a basic block of interest).
  • the prediction unit 103 describes an intra prediction method such as horizontal prediction using the coded pixels of the subblock adjacent to the left of the subblock of interest and vertical prediction using the coded pixels of the subblock adjacent above.
  • the method used for the prediction of the subblock of interest is determined from the inside, and the information indicating it is generated as the intra prediction mode.
  • peripheral sub-blocks used for intra-prediction are not limited to those located on the left and above, and those located on the lower left, upper left, and upper right are also used if they are encoded.
  • the relationship between the subblock to be encoded and the surrounding subblocks used for intra-prediction will be described in more detail with reference to FIGS. 8A to 8E.
  • the thick frame shows a basic block of 64 ⁇ 64 pixels
  • the thin frame obtained by dividing the thick frame into quadtrees shows a subblock of 32 ⁇ 32 pixels.
  • 8B to 8E show a subblock of interest (C: Current) to be encoded and five subblocks ⁇ BL: Bottom Left, L: Left, located in the periphery ⁇ lower left, left, upper left, upper, upper right ⁇ . It shows the relationship with UL: Upper Left, U: Upper, UR: Upper Right ⁇ .
  • FIG. 8B shows the positional relationship with the five surrounding subblocks when the attention subblock (C) is located at the upper left of the attention basic block. Whether or not the attention subblock (C) is located at the upper left of the attention basic block can be determined by whether or not the attention subblock (C) includes the upper left pixel of the attention basic block. it can.
  • FIG. 8B since the coding processing of the five subblocks UL, U, UR, L, and BL is completed at the time of the prediction processing of the coding target subblock (C), these five subblocks are used for the intra. It is possible to perform prediction processing. Depending on the prediction method, the intra-prediction processing of the attention subblock (C) can be performed using all of these five subblocks, or the attention subblock using only a part of the five subblocks. The intra prediction process of (C) may be performed.
  • FIG. 8C shows the positional relationship with the five surrounding subblocks when the attention subblock (C) is located in the upper right of the attention basic block. Whether or not the attention subblock (C) is located at the upper right of the attention basic block can be determined by whether or not the attention subblock (C) includes the pixel at the upper right end of the attention basic block.
  • the coding processing of the lower left (BL) subblock among the five subblocks is not completed. Therefore, the subblock (BL) cannot be used for the intra prediction process.
  • the upper right (UR) subblock is coded when the coding process is sequentially performed.
  • each basic block line (block line) is executed in parallel with a delay of one basic block.
  • FIG. 9 shows that the four parallel coding processes are performed by delaying the four basic block lines from left to right by one basic block. The four shaded areas in the figure indicate the basic blocks encoded in parallel.
  • the sub is located in the upper right of the attention subblock. It is conceivable to perform prediction processing without using a block (UR). However, only in such a case, if the use of the subblock (UR) located at the upper right of the subblock of interest in the prediction processing is prohibited (restricted), the difficulty level in implementing the coding method is correspondingly increased. There are problems such as high speed, complicated coding process, slowdown of coding process, and increase of memory usage.
  • the prediction unit 103 of the present embodiment always considers the attention subblock when the attention subblock is a subblock located at the upper right of the attention basic block, regardless of whether or not the coding processing is executed in parallel. It is assumed that the prediction process of the subblock of interest is performed without using the subblock (UR) located at the upper right of. In other words, if the upper right subblock of the attention subblock is in the basic block located at the upper right of the attention basic block, the upper right subblock is excluded from the subblocks used in the prediction process. As a result, it is possible to reduce the mounting cost such as holding the pixel value caused by the reference of the upper right basic block regardless of whether or not the coding processing is executed in parallel.
  • the prediction unit 103 excludes the upper right (UR) and the lower left (BL) during the prediction processing (these Intra-prediction processing is performed using three sub-blocks (restricted from using blocks), left (L), upper left (UL), and upper (U).
  • FIG. 8D shows the positional relationship between the attention subblock (C) and the surrounding five subblocks when it is located at the lower left of the basic block. Whether or not the attention subblock (C) is located at the lower left in the attention basic block can be determined by whether or not the attention subblock (C) includes the pixel at the lower left end in the attention basic block.
  • the coding processing of the lower left (BL) subblock is not completed. Therefore, the subblock (BL) cannot be used for the intra prediction process.
  • the remaining four subblocks, left (L), upper left (UL), upper (U), and upper right (UR), have been encoded. Therefore, the prediction unit 103 does not use the lower left (BL) subblock, but refers to these four subblocks of left (L), upper left (UL), upper (U), and upper right (UR) for intra-prediction. Perform processing.
  • FIG. 8E shows the positional relationship between the attention subblock (C) and the surrounding five subblocks when it is located at the lower right of the basic block. Whether or not the attention subblock (C) is located at the lower right of the attention basic block can be determined by whether or not the attention subblock (C) includes the lower right pixel in the attention basic block. ..
  • the coding processing of the lower left (BL) and upper right (UR) subblocks is not completed. Therefore, the subblock (BL) and the subblock (UR) cannot be used for the intra prediction process.
  • the prediction unit 103 does not use the lower left (BL) and upper right (UR) subblocks, and the remaining three subblocks, left (L), upper left (UL), and upper (U). Intra-prediction processing is performed using.
  • inter-prediction is a process of predicting the pixels of the block of interest (sub-block to be encoded) by referring to the pixels of the encoded frame (frame different from the frame to which the sub-block to be encoded belongs). For example, if there is no movement between the coded frame of the reference target and the coded frame of the reference target, the pixels of the coded target subblock are predicted using the pixels at the same position of the coded frame of the reference target. .. In such a case, a motion vector (0, 0) indicating that there is no motion is generated. On the other hand, when motion is generated between frames for the subblock to be encoded, a motion vector (MVx, MVy) indicating the motion is generated.
  • Image coding standards such as HEVC employ a technique called "motion vector prediction" in order to further reduce the amount of data related to this motion vector.
  • This is a technique for predicting the motion vector of the subblock to be encoded by using the motion vector of another encoded subblock, and has the effect of reducing the amount of data of the motion vector.
  • the sub-block C in FIG. 8B is a part of a moving object
  • the sub-blocks L, U, UR, L, BL, etc. around the sub-block C are moved by a motion vector predictor. It is selected as a candidate, and the motion vector closest to the motion vector of the subblock C is used as the motion vector predictor.
  • the difference between the motion vector predictor and the motion vector of the subblock to be encoded is encoded.
  • the prediction unit 103 of the present embodiment limits the sub-blocks used for motion vector prediction according to the positional relationship between the sub-block C of interest to be encoded and the sub-blocks around it. In other words, the subblocks from which the motion vector can be selected (enabled) as the motion vector predictor are made different according to the positional relationship with the surrounding subblocks.
  • the prediction unit 103 performs motion vector prediction using the motion vectors of all five subblocks as candidates for the motion vector predictor.
  • these sub-blocks may not have a motion vector, in which case the motion vector of that sub-block cannot be used. For example, if a subblock is encoded using intra-prediction, that block does not have a motion vector. This also applies to the following description of FIGS. 8C to 8E.
  • the maximum number of subblocks that can be used for is not limited to these five subblocks, but may be a part of the five subblocks, or other subblocks may be added. However, in any case, as will be described later, the subblocks that can be used are limited depending on the position of the subblock (C) to be encoded.
  • the prediction unit 103 uses the unencoded lower left (BL) sub as in the case of intra prediction.
  • the motion vector of the block and the motion vector of the upper right (UR) subblock belonging to the upper right basic block of the current basic block are not always used for the motion vector prediction of the coded subblock (C).
  • the prediction unit 103 performs motion vector prediction using the motion vectors of the remaining three sub-blocks (L), upper left (UL), and upper (U) as candidates for the motion vector predictor.
  • the motion vector of the lower left (BL) subblock and the motion vector of the upper right (UR) subblock are different regardless of whether or not the coding processing is executed in parallel. Since it is prohibited to be a candidate for a motion vector predictor, it is possible to reduce the implementation cost such as holding the pixel value caused by the reference of the upper right basic block.
  • the prediction unit 103 uses the unencoded lower left (BL) motion vector to predict the motion vector. Not used. That is, in this case, it is prohibited (restricted) that the motion vector at the lower left (BL) is a candidate for the motion vector predictor. Therefore, the prediction unit 103 performs motion vector prediction using the motion vectors of the remaining four sub-blocks (L), upper left (UL), upper (U), and upper right (UR) as candidates for the motion vector predictor.
  • the prediction unit 103 uses the unencoded lower left (BL) motion vector and upper right (UR).
  • Motion vector is not used for motion vector prediction. That is, in this case, it is prohibited (restricted) that the motion vector of the lower left (BL) block is a candidate for the motion vector predictor. It is also prohibited (restricted) to use the motion vector of the upper right (UR) block as a candidate for the motion vector predictor. Therefore, the prediction unit 103 uses the motion vectors of the remaining three sub-blocks (L), upper left (UL), and upper (U) as candidates for the motion vector predictor to perform motion vector prediction.
  • the prediction unit 103 generates prediction image data from the intra prediction mode determined as described above, the motion vector, and the encoded pixels, and further generates a prediction error from the input image data and the prediction image data. Output to the conversion / quantization unit 104. Further, the prediction unit 103 outputs information such as subblock division, intra prediction mode, and motion vector as prediction information to the coding unit 109, the image reproduction unit 106, and the motion vector memory 112.
  • the conversion / quantization unit 104 performs orthogonal conversion / quantization on the input prediction error and generates a quantization coefficient. Specifically, the conversion / quantization unit 104 first performs an orthogonal conversion process corresponding to the size of the subblock to generate an orthogonal conversion coefficient. Next, the conversion / quantization unit 104 quantizes the orthogonal conversion coefficient and generates the quantization coefficient. Then, the conversion / quantization unit 104 outputs the generated quantization coefficient to the coding unit 109 and the inverse quantization / inverse conversion unit 105.
  • the inverse quantization / inverse conversion unit 105 inversely quantizes the input quantization coefficient to reproduce the conversion coefficient, and further performs inverse orthogonal conversion of the reproduced conversion coefficient to reproduce the prediction error. Then, the inverse quantization / inverse conversion unit 105 outputs the reproduced prediction error to the image reproduction unit 106.
  • the image reproduction unit 106 appropriately refers to the frame memory 107 and the motion vector memory 112 based on the prediction information input from the prediction unit 103, and reproduces the predicted image.
  • the image reproduction unit 106 reproduces the predicted image using the pixels of the peripheral encoded subblocks stored in the frame memory 107.
  • the subblocks used for intra-prediction are limited according to the positional relationship between the coded subblock and the peripheral coded subblocks, and the limiting method is the limiting method in the prediction processing of the prediction unit 103. Since the same is true, the description here will be omitted.
  • the image reproduction unit 106 uses the pixels of the encoded subblocks of other frames stored in the frame memory 107 based on the motion vector determined by the prediction unit 103. Is used to generate a predicted image. In this case, the subblocks used for motion vector prediction are limited according to the positional relationship between the coded subblock and the peripheral coded subblocks, and the limiting method is the limiting method in the prediction process of the prediction unit 103. Since it is the same as the above, the description here is omitted.
  • the image reproduction unit 106 then reproduces the image data from the reproduced predicted image and the reproduced prediction error input from the inverse quantization / inverse conversion unit 105, and stores the reproduced image data in the frame memory 107.
  • the in-loop filter unit 108 reads the reproduced image from the frame memory 107 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 108 re-stores the filtered image data in the frame memory 107.
  • the coding unit 109 entropy-encodes the quantization coefficient generated by the conversion / quantization unit 104 and the prediction information input from the prediction unit 103 in block units to generate code data.
  • the method of entropy coding is not particularly limited, but Golomb coding, arithmetic coding, Huffman coding and the like can be used.
  • the coding unit 109 outputs the generated code data to the integrated coding unit 110.
  • the integrated coding unit 110 generates header code data encoded in the header part of the sequence or picture, which is necessary for decoding the bit stream on the decoding side.
  • the integrated coding unit 110 further multiplexes (integrates) the code data and the like input from the coding unit 109 so as to follow the header code data to form a bit stream. Then, the coding unit 109 outputs the formed bit stream to the outside via the output terminal 111.
  • FIG. 6 shows an example of the data structure of the bitstream output in the first embodiment.
  • the picture data which is the coded data for each picture
  • FIG. 3 is a flowchart showing the coding process for one frame of the control unit 150 in the image coding device of the embodiment.
  • control unit 150 controls the integrated coding unit 110 to determine the size of the image (the number of pixels in the horizontal and vertical directions of one frame), the size of the basic block, and the like.
  • the header information necessary for encoding the image data and decoding the bit stream on the decoding side is encoded and output.
  • control unit 150 controls the block division unit 102, divides the input frame image into basic block units, and outputs one of them to the prediction unit 103.
  • control unit 150 controls the prediction unit 103 to execute prediction processing on the image data of the basic block unit generated in S302, and performs prediction information such as subblock division information and prediction mode and prediction image data. Generate.
  • the prediction unit 103 further calculates a prediction error from the input image data and the predicted image data, and outputs the prediction error.
  • control unit 150 controls the conversion / quantization unit 104 to perform orthogonal conversion and quantization.
  • the conversion / quantization unit 104 executes orthogonal conversion with respect to the prediction error calculated in S303 and generates a conversion coefficient.
  • the conversion / quantization unit 104 performs quantization with respect to the conversion coefficient to generate a quantization coefficient.
  • control unit 150 controls the inverse quantization / inverse conversion unit 105 to perform inverse quantization and inverse orthogonal conversion.
  • the inverse quantization / inverse conversion unit 105 dequantizes the quantization coefficient generated in S304 and reproduces the conversion coefficient. Further, the inverse quantization / inverse conversion unit 105 performs inverse orthogonal conversion with respect to the conversion coefficient and reproduces the prediction error.
  • control unit 150 controls the image reproduction unit 106 to reproduce the image. Specifically, the image reproduction unit 106 reproduces the predicted image based on the prediction information generated in S303. Further, the image reproduction unit 106 reproduces the image data from the reproduced predicted image and the prediction error generated in S305, and stores the image data in the frame memory 107.
  • control unit 150 controls the coding unit 109 to encode the image. Specifically, the coding unit 109 encodes the prediction information generated in S303 and the quantization coefficient generated in S304 to generate code data. In addition, the coding unit 109 generates a bit stream including other code data.
  • control unit 150 determines whether or not the coding of all the basic blocks in the frame of interest is completed, and if so, proceeds to S309 if the coding is completed, and there is an uncoded basic block. If it is determined, the process is returned to S302 in order to encode the basic block.
  • control unit 150 controls the in-loop filter unit 108 to execute the filter processing. Specifically, the in-loop filter unit 108 performs in-loop filter processing on the image (image data reproduced in S306) stored in the frame memory 107, and transfers the filtered image data to the frame memory 107. Restore and finish this process.
  • the basic block to which the subblock to be encoded (the subblock of interest) belongs regardless of whether or not each block line composed of a plurality of basic blocks is encoded in parallel.
  • FIG. 2 is a block configuration diagram of an image decoding device that performs intra-predictive decoding and inter-predictive decoding in sub-block units from the coded image data generated by the image coding device in the embodiment.
  • the configuration and its operation related to the decoding process will be described with reference to the figure.
  • the restrictions on the subblocks that can be used in the above-mentioned intra-prediction and the restrictions on the sub-blocks that can be selected as the motion vector predictor in the inter-prediction are the same.
  • the image decoding device has a control unit 250 that controls the entire device.
  • the control unit 250 has a CPU, a ROM for storing a program executed by the CPU, and a RAM used as a work area of the CPU.
  • the image decoding device includes an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization / inverse conversion unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and the like. It has a motion vector memory 209.
  • the input terminal 201 inputs a coded bit stream
  • the input source is, for example, a storage medium storing the coded stream, but it may be input from a network, and the type thereof does not matter.
  • the separation / decoding unit 202 separates the bitstream into code data related to information related to decoding processing and coefficients, and decodes the code data existing in the header part of the bitstream.
  • the separation / decoding unit 202 performs the reverse operation of the integrated coding unit 110 of FIG.
  • the decoding unit 203 decodes the code data output from the separation decoding unit 202 and reproduces the quantization coefficient and the prediction information. Then, the decoding unit 203 outputs the obtained quantization coefficient and prediction information to the inverse quantization / inverse conversion unit 204 and the image reproduction unit 205. Further, the decoding unit 203 outputs and stores the motion vector information included in the prediction information in the motion vector memory 209.
  • the inverse quantization / inverse conversion unit 204 performs inverse quantization on the quantization coefficient input from the decoding unit 203 to obtain a conversion coefficient. Further, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal transformation on the conversion coefficient and reproduces the prediction error.
  • the image reproduction unit 205 appropriately refers to the frame memory 206 based on the input prediction information to generate the prediction image data. Then, the image reproduction unit 205 generates the reproduction image data from the predicted image data and the prediction error reproduced by the inverse quantization / inverse conversion unit 204, and outputs (stores) the reproduced image data to the frame memory.
  • the in-loop filter unit 207 performs an in-loop filter process such as a deblocking filter on the reproduced image (stored in the frame memory 206), and the filtered image. Is output.
  • an in-loop filter process such as a deblocking filter on the reproduced image (stored in the frame memory 206), and the filtered image. Is output.
  • the output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside.
  • the output destination is generally a display device, but other devices may be used.
  • the encoded bit stream is input in frame units.
  • the separation / decoding unit 202 inputs or receives a bit stream for one frame via the input terminal 201.
  • the separation / decoding unit 202 separates the input bit stream into information related to the decoding process and code data related to the coefficient, and decodes the code data existing in the header unit of the bit stream. Subsequently, the separation / decoding unit 202 reproduces the code data of the basic block unit of the picture data and outputs the code data to the decoding unit 203.
  • the decoding unit 203 decodes the code data and reproduces the quantization coefficient and the prediction information. Then, the decoding unit 203 outputs the quantization coefficient to the inverse quantization / inverse conversion unit 204 and the prediction information to the image reproduction unit 205, respectively. Further, the decoding unit 203 outputs and stores the motion vector in the prediction information to the motion vector memory 209 so as to correspond to the position of the subblock.
  • the inverse quantization / inverse conversion unit 204 performs inverse quantization on the input quantization coefficient to generate an orthogonal conversion coefficient. Further, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal transformation on the generated orthogonal transformation coefficient to reproduce the prediction error. Then, the inverse quantization / inverse conversion unit 204 outputs the reproduced prediction error to the image reproduction unit 205.
  • the image reproduction unit 205 reproduces the predicted image by referring to the prediction information and the motion vector memory 209 input from the decoding unit 203 and appropriately referring to the frame memory 206.
  • two types of prediction methods intra-prediction and inter-prediction, are used as in the prediction unit 103 of the image coding apparatus.
  • the subblocks used for the prediction processing are limited according to the positional relationship between the decoding target subblock and the peripheral decoded subblocks.
  • the image reproduction unit 106 in the image coding apparatus Since the same is true, the description thereof will be omitted.
  • the image reproduction unit 205 reproduces the image data from the predicted image and the prediction error input from the inverse quantization / inverse conversion unit 204, and stores the reproduced image data in the frame memory 206.
  • the stored image data will be used as a reference when making a prediction.
  • the image reproduction unit 205 also limits the sub-blocks that can be used in the intra-prediction, as in the prediction unit 103 of the image coding device. Therefore, it is not necessary to store unnecessary data for reference in other blocks, and it is easy to determine whether to store data for other blocks, so that the memory usage amount It is possible to reduce the complexity of implementation and reduce the complexity of implementation. Further, similarly to the prediction unit 103 of the image coding device, the subblocks that can be selected as the motion vector predictor in the inter-prediction are also limited. This also contributes to the reduction of implementation complexity and the reduction of memory usage.
  • the in-loop filter unit 207 reads the reproduced image data from the frame memory 206 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 207 stores the filtered image in the frame memory 206 again. The reproduced image stored in the frame memory 206 and filtered is finally output from the terminal 208 to the outside (display or the like).
  • FIG. 4 is a flowchart showing a one-frame image decoding process of the control unit 250 in the image decoding device according to the embodiment. The decoding process will be described below with reference to the same figure.
  • control unit 250 controls the separation / decoding unit 202 to separate the bit stream into information related to the decoding process and code data of the image data, acquire information required for decoding, and code data of the image. Is output to the decoding unit 203.
  • control unit 250 controls the decoding unit 203, decodes the code data separated in S401, and reproduces the quantization coefficient and the prediction information.
  • control unit 250 controls the inverse quantization / inverse conversion unit 204 to perform inverse quantization and inverse orthogonal conversion. Specifically, the inverse quantization / inverse conversion unit 204 performs inverse quantization on the quantization coefficient of each subblock under the control of the control unit 250 to obtain a conversion coefficient, and further, the conversion coefficient is obtained. The prediction error is reproduced by performing the inverse orthogonal transformation on the.
  • control unit 250 controls the image reproduction unit 205 to reproduce the image data. Specifically, the image reproduction unit 205 reproduces the predicted image based on the predicted information. Then, the image reproduction unit 205 reproduces the image data by adding the prediction error generated in S404 to the predicted image.
  • control unit 250 determines whether or not the decoding process of the frame of interest has been completed, that is, whether or not the decoding of all the basic blocks in the frame of interest and all the subblocks inside the frame has been completed. Then, when the decoding of the frame of interest is completed, the control unit 250 proceeds to S406, and if there is an undecrypted basic block or subblock, the control unit 250 decodes the next basic block or subblock. , The process is returned to S402.
  • control unit 250 controls the in-loop filter unit 207, performs in-loop filter processing on the image data (image data reproduced in S404) stored in the frame memory 206, and filters the image. Is generated and the process ends.
  • the subblock to be decoded belongs to the upper right of the basic block. It is possible to decode a bitstream that has undergone prediction processing without referring to subblocks that belong to adjacent basic blocks. As a result, the complexity of implementation is reduced, and the feasibility of parallel processing of decoding can be improved.
  • FIG. 5 is a block diagram showing a configuration example of computer hardware applicable to the image coding device and the image decoding device according to the above embodiment.
  • the CPU 501 controls the entire computer by using the computer programs and data stored in the RAM 502 and the ROM 503, and executes each of the above-described processes as performed by the image processing device according to the above embodiment. That is, the CPU 501 functions as each processing unit shown in FIGS. 1 and 2.
  • the RAM 502 has an area for temporarily storing programs and data acquired from the outside via the external storage device 506 and the I / F (interface) 507. Further, the RAM 502 is also used as a work area used by the CPU 501 when executing various processes.
  • the RAM 502 can be allocated as a frame memory, for example, or various other areas can be provided as appropriate.
  • the ROM 503 stores the setting data of this computer, the boot program, and the like.
  • the operation unit 504 is composed of a keyboard, a mouse, and the like, and can be operated by a user of the computer to input various instructions to the CPU 501.
  • the display unit 505 displays the processing result by the CPU 501.
  • the display unit 505 is composed of, for example, a liquid crystal display.
  • the external storage device 506 is a large-capacity information storage device represented by a hard disk drive device.
  • the external storage device 506 stores an OS (operating system) and a computer program (application program) for realizing the functions of the respective parts shown in FIGS. 1 and 2 in the CPU 501. Further, each image data as a processing target may be stored in the external storage device 506.
  • the computer programs and data stored in the external storage device 506 are appropriately loaded into the RAM 502 according to the control by the CPU 501, and are processed by the CPU 501.
  • a network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 507, and the computer acquires and sends various information via the I / F 507. Can be done.
  • Reference numeral 508 is a bus connecting the above-mentioned parts.
  • the CPU 501 executes the boot program stored in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and executes the boot program. Then, under the control of the OS, the CPU 501 loads the application program related to encoding or decoding from the external storage device 506 into the RAM 502 and executes it.
  • the CPU 501 functions as each processing unit of FIG. 1 or 2, and the present device functions as an image coding device or an image decoding device.
  • the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
  • a circuit for example, ASIC
  • the present invention is used in a coding device / decoding device that encodes / decodes still images / moving images.
  • it can be applied to coding methods and decoding methods that use intra-prediction and inter-prediction.

Abstract

La présente invention réduit la complexité de montage en limitant les blocs pouvant être utilisé dans un traitement de prédiction à certains blocs en fonction de relations de position des blocs. A cet effet, un dispositif de codage d'image, qui code une image animée selon la présente invention, comprend : une unité de division qui sépare, dans des images de trame dans l'image mobile à coder, des blocs de base d'une taille prédéfinie dans un ordre de balayage de trame et divise en outre les blocs de base en une pluralité de sous-blocs ; et une unité de codage qui code en prédiction intra et qui code en prédiction inter les sous-blocs obtenus à partir de l'unité de division. Ainsi, lorsqu'un sous-bloc d'intérêt dans un bloc de base d'intérêt est codé en prédiction intra et un sous-bloc en haut à droite du sous-bloc d'intérêt est dans un bloc de base en haut à droite du bloc de base d'intérêt, l'unité de codage réalise un codage en prédiction intra en excluant le sous-bloc en haut à droite du sous-bloc d'intérêt d'une cible de référence du codage en prédiction intra.
PCT/JP2020/021614 2019-06-20 2020-06-01 Dispositif de codage d'image, dispositif et procédé de décodage d'image et programme WO2020255688A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019114933A JP2021002722A (ja) 2019-06-20 2019-06-20 画像符号化装置及び画像復号装置及び方法及びプログラム
JP2019-114933 2019-06-20

Publications (1)

Publication Number Publication Date
WO2020255688A1 true WO2020255688A1 (fr) 2020-12-24

Family

ID=73994300

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/021614 WO2020255688A1 (fr) 2019-06-20 2020-06-01 Dispositif de codage d'image, dispositif et procédé de décodage d'image et programme

Country Status (2)

Country Link
JP (1) JP2021002722A (fr)
WO (1) WO2020255688A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008182527A (ja) * 2007-01-25 2008-08-07 Matsushita Electric Ind Co Ltd 画像符号化装置及び方法、並びに撮像システム
WO2013069557A1 (fr) * 2011-11-10 2013-05-16 ソニー株式会社 Dispositif et procédé de traitement d'image

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3152906B1 (fr) * 2014-07-07 2021-06-16 HFI Innovation Inc. Procédé de recherche de copie intrabloc et de plage de compensation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008182527A (ja) * 2007-01-25 2008-08-07 Matsushita Electric Ind Co Ltd 画像符号化装置及び方法、並びに撮像システム
WO2013069557A1 (fr) * 2011-11-10 2013-05-16 ソニー株式会社 Dispositif et procédé de traitement d'image

Also Published As

Publication number Publication date
JP2021002722A (ja) 2021-01-07

Similar Documents

Publication Publication Date Title
JP2024019496A (ja) 画像符号化装置、画像復号装置、画像符号化方法、画像復号方法
WO2020184229A1 (fr) Dispositif de codage d'image, procédé de codage d'image, et programme
WO2020255688A1 (fr) Dispositif de codage d'image, dispositif et procédé de décodage d'image et programme
WO2020255689A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, procédé, et programme
WO2020246125A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, procédé, et programme
KR102599267B1 (ko) 화상 부호화 장치, 화상 부호화 방법 및 프로그램, 화상 복호 장치, 화상 복호 방법 및 프로그램
WO2020003740A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, procédés de commande et programme correspondants
KR20220051022A (ko) 화상 부호화 장치, 화상 부호화 방법, 화상 복호 장치, 화상 복호 방법, 및 비일시적인 컴퓨터 판독가능한 저장 매체
WO2020255820A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, et programme d'ordinateur
CN113196760B (zh) 图像编码设备、图像解码设备、控制方法和存储介质
WO2021049277A1 (fr) Dispositif de codage d'image et dispositif de décodage d'image
WO2020179365A1 (fr) Dispositif de codage d'images, procédé de codage d'images, dispositif de décodage d'images et procédé de décodage d'images
TWI788268B (zh) 圖像編碼裝置、圖像解碼裝置、其等之控制方法及程式
WO2021187023A1 (fr) Dispositif de codage d'image, procédé et programme de codage d'image, dispositif de décodage d'image, et procédé et programme de décodage d'image
WO2020184223A1 (fr) Décodeur d'image, procédé de décodage d'image, et programme
WO2020183859A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, procédé de codage d'image, procédé de décodage d'image, et programme
WO2020184227A1 (fr) Décodeur d'image, procédé de décodage d'image et programme
CN118018737A (en) Image encoding apparatus, image decoding apparatus, and control method
CN118018736A (en) Image encoding apparatus, image decoding apparatus, and control method
CN118018733A (en) Image encoding apparatus, image decoding apparatus, and control method
CN118018734A (en) Image encoding apparatus, image decoding apparatus, and control method
CN118018735A (en) Image encoding apparatus, image decoding apparatus, and control method
JP2023140150A (ja) 画像符号化装置、画像復号装置、画像符号化方法、および画像復号方法
KR20220055495A (ko) 화상 부호화 장치, 화상 부호화 방법, 화상 복호 장치, 화상 복호 방법, 및 컴퓨터 프로그램

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20826935

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20826935

Country of ref document: EP

Kind code of ref document: A1