WO2020050705A1 - Procédé de décodage et de codage d'image destiné au traitement d'un paramètre de quantification d'unité de groupe - Google Patents

Procédé de décodage et de codage d'image destiné au traitement d'un paramètre de quantification d'unité de groupe Download PDF

Info

Publication number
WO2020050705A1
WO2020050705A1 PCT/KR2019/011651 KR2019011651W WO2020050705A1 WO 2020050705 A1 WO2020050705 A1 WO 2020050705A1 KR 2019011651 W KR2019011651 W KR 2019011651W WO 2020050705 A1 WO2020050705 A1 WO 2020050705A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
quantization
block
picture
quantization parameter
Prior art date
Application number
PCT/KR2019/011651
Other languages
English (en)
Korean (ko)
Inventor
임화섭
임정윤
Original Assignee
가온미디어 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 가온미디어 주식회사 filed Critical 가온미디어 주식회사
Publication of WO2020050705A1 publication Critical patent/WO2020050705A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present invention relates to video encoding and decoding, and more particularly, to a method of performing prediction and quantization by dividing a video picture into a plurality of blocks.
  • one picture is divided into a plurality of blocks having a predetermined size, and encoding is performed.
  • inter prediction and intra prediction techniques that remove redundancy between pictures are used to increase compression efficiency.
  • a residual signal is generated using intra prediction and inter prediction, and the reason for obtaining the residual signal is that when coding with the residual signal, the amount of data is small, so the data compression rate is high, and the better the prediction, the better the residual signal. This is because the value of.
  • the intra prediction method predicts data of the current block using pixels around the current block.
  • the difference between the actual value and the predicted value is called the residual signal block.
  • the intra prediction method increases from 9 prediction modes used in the existing H.264 / AVC to 35 prediction modes to further refine the prediction.
  • the current block is compared with blocks in neighboring pictures to find the most similar block.
  • the location information (Vx, Vy) for the found block is called a motion vector.
  • the difference of pixel values in a block between a current block and a prediction block predicted by a motion vector is called a residual signal block (motion-compensated residual block).
  • the quantization processing process according to the current block structure may be inefficient.
  • the present invention is to solve the above problems, is suitable for encoding and decoding of high-resolution images, and the image processing method for processing a more efficient quantization process according to a complex encoding characteristic change, and an image decoding and encoding method using the same
  • the purpose is to provide.
  • a picture is a plurality of basic units in which inter prediction or intra prediction is performed.
  • Decoding the coding unit is divided into a coding unit (Coding Unit) of the picture or the divided coding unit into a quad tree (quad tree), binary tree (binary tree) or ternery tree (ternery tree) structure
  • a plurality of pictures are basic units in which inter prediction or intra prediction is performed.
  • Decoding the coding unit is divided into a coding unit (Coding Unit) of the picture or the divided coding unit into a quad tree (quad tree), binary tree (binary tree) or ternery tree (ternery tree) structure
  • a characteristic adaptive quantization parameter determining unit selectively adaptively determining a quantization parameter according to the quantization process of the target block, based on the group quantization information of the target block.
  • a plurality of pictures are basic units in which inter prediction or intra prediction is performed.
  • Decoding the coding unit is divided into a coding unit (Coding Unit) of the picture or the divided coding unit into a quad tree (quad tree), binary tree (binary tree) or ternery tree (ternery tree) structure
  • a coding unit which is a basic unit in which inter prediction or intra prediction is performed, may be divided into a composite tree structure including a quad tree, a binary tree, and a ternary tree, and the target block group of the target block
  • quantization efficiency corresponding to a diversified block shape can be improved, and coding efficiency for a high-resolution image can be improved.
  • the derivation process of the quantization parameter and the quantization parameter derived therefrom are selectively adaptively determined using the detailed characteristic information of the image, the motion compensation efficiency corresponding to the diversified block form and The filtering effect can be improved, and coding efficiency for a high resolution image can be improved.
  • FIG. 1 is a block diagram showing the configuration of an image encoding apparatus according to an embodiment of the present invention.
  • 2 to 5 are diagrams for describing a first embodiment of a method of dividing and processing an image in block units.
  • FIG. 6 is a block diagram illustrating an embodiment of a method for performing inter prediction in an image encoding apparatus.
  • FIG. 7 is a block diagram showing the configuration of an image decoding apparatus according to an embodiment of the present invention.
  • FIG. 8 is a block diagram illustrating an embodiment of a method of performing inter prediction in an image decoding apparatus.
  • FIG. 9 is a view for explaining a second embodiment of a method of dividing and processing an image in block units.
  • FIG. 10 is a diagram illustrating an embodiment of a syntax structure used to process an image by dividing it into blocks.
  • FIG. 11 is a diagram for describing a third embodiment of a method of dividing and processing an image in block units.
  • FIG. 12 is a diagram for explaining an embodiment of a method of configuring a transformation unit by dividing a coding unit into a binary tree structure.
  • FIG. 13 is a view for explaining a fourth embodiment of a method of dividing and processing an image in block units.
  • 14 to 16 are diagrams for explaining other embodiments of a method of dividing and processing an image in block units.
  • 17 and 18 are diagrams for explaining embodiments of a method of determining a split structure of a transform unit by performing rate distortion optimization (RDO).
  • RDO rate distortion optimization
  • 19 is a view for explaining a composite partition structure according to another embodiment of the present invention.
  • 20 is a block diagram illustrating a quantization unit according to an embodiment of the present invention in more detail.
  • 21 is a diagram for explaining a characteristic adaptive quantization parameter determiner according to an embodiment of the present invention in more detail.
  • FIG. 22 is a flowchart illustrating a process of determining an initial quantization parameter corresponding to a CTU group unit according to an embodiment of the present invention.
  • 23 is a view for explaining that an initial quantization parameter is determined for each group quantization unit according to an embodiment of the present invention.
  • 24 is a flowchart illustrating a decoding method for processing inverse quantization according to an embodiment of the present invention.
  • 25 is a view for explaining slice type based quantization parameter derivation according to an embodiment of the present invention.
  • 26 is a flowchart illustrating an entire process according to an embodiment of the present invention.
  • 27 to 29 are diagrams for explaining a high level syntax according to an embodiment of the present invention.
  • first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.
  • first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may be referred to as a first component.
  • each component shown in the embodiments of the present invention are shown independently to indicate different characteristic functions, and do not mean that each component is composed of separate hardware or one software component. That is, for convenience of description, each component is listed and included as each component, and at least two components of each component are combined to form one component, or one component is divided into a plurality of components to perform functions.
  • the integrated and separated embodiments of the components are also included in the scope of the present invention without departing from the essence of the present invention.
  • the components are not essential components for performing essential functions in the present invention, but may be optional components for improving performance.
  • the present invention can be implemented by including only components necessary for realizing the essence of the present invention, except for components used for performance improvement, and structures including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.
  • the image encoding apparatus 10 includes a picture splitter 110, a transform unit 120, a quantization unit 130, and scanning.
  • the picture splitter 110 analyzes an input video signal, divides a picture into coding units, determines a prediction mode, and determines the size of a prediction unit for each coding unit.
  • the picture splitter 110 sends the prediction unit to be encoded to the intra prediction unit 150 or the inter prediction unit 160 according to a prediction mode (or prediction method). Also, the picture splitter 110 sends a prediction unit to be encoded to the subtractor 190.
  • a picture of an image may be composed of a plurality of bricks, tiles, or slices, and the bricks, tiles, or slices are a plurality of coding tree units, which are basic units for dividing a picture ( Coding Tree Unit (CTU).
  • CTU Coding Tree Unit
  • each tile, brick, or slice may form a CTU group unit described according to an embodiment of the present invention.
  • the tile may be a rectangular CTU unit partition defined by rows and columns
  • the brick may be a rectangular partition composed of CTU row units or a substructure included in a specific tile.
  • a slice may be composed of an integer number of bricks formed continuously, and configuration information may be exclusively included in one network abstraction layer unit (NAL unit).
  • NAL unit network abstraction layer unit
  • the slice may include one or more entire tiles, or may include only consecutive bricks included in one tile.
  • one picture may be divided into a plurality of bricks, which are rectangular regions of a CTU row group, and the picture includes the bricks and is divided into one or more columns. It may be divided into tiles, divided into tiles including the bricks and divided into one or more horizontal columns, or divided into tiles including the bricks and divided into one or more vertical columns and one or more horizontal columns.
  • the picture may be equally divided into tiles of the same size based on the lengths of the horizontal and vertical columns in the picture, or may be divided into tiles of different sizes.
  • the coding tree unit may be divided into one or two or more coding units (CUs), which are basic units in which inter prediction or intra prediction is performed.
  • CUs coding units
  • the coding unit CU may be divided into one or more prediction units (PUs), which are basic units for which prediction is performed.
  • PUs prediction units
  • the encoding apparatus 10 determines one of inter prediction and intra prediction for each of the divided coding units (CUs) as a prediction method, but differently predicts blocks for each prediction unit (PU). Can be created.
  • the coding unit CU may be divided into one or two or more transform units (TUs), which are basic units for transforming a residual block.
  • TUs transform units
  • the picture division unit 110 may transmit the image data to the subtraction unit 190 in a block unit (eg, a prediction unit (PU) or a transformation unit (TU)) divided as described above.
  • a block unit eg, a prediction unit (PU) or a transformation unit (TU)
  • a coding tree unit (CTU) having a maximum size of 256x256 pixels is divided into a quad tree structure, and may be divided into four coding units (CUs) having a square shape.
  • Each of the four coding units (CUs) having the square shape may be re-divided into a quad tree structure, and the depth (Depth) of the coding unit (CU) divided into a quad tree structure may be 0 to 3 as described above. It can have one integer value.
  • the coding unit CU may be divided into one or two or more prediction units PU according to a prediction mode.
  • the prediction unit PU may have the size of 2Nx2N shown in FIG. 3A or NxN shown in FIG. 3B. have.
  • the prediction unit PU when the size of the coding unit CU is 2Nx2N, the prediction unit PU is 2Nx2N shown in FIG. 4A, 2NxN shown in FIG. 4B, and FIG. 4B Nx2N shown in (c), NxN shown in FIG. 4 (d), 2NxnU shown in FIG. 4 (e), 2NxnD shown in FIG. 4 (f), and shown in FIG. 4 (g) It may have a size of any one of nLx2N and nRx2N shown in Figure 4 (h).
  • the coding unit is divided into a quad tree structure, and can be divided into four transform units (TUs) having a square shape.
  • the four transform units (TUs) having the square shape may be re-divided into quad tree structures, and the depth (Depth) of the transform unit (TU) divided into quad tree structures may be 0 to 3 as described above. It can have one integer value.
  • the prediction unit PU and the transform unit TU split from the corresponding coding unit CU may have independent splitting structures.
  • the transform unit TU divided from the coding unit CU cannot be larger than the size of the prediction unit PU.
  • the conversion unit (TU) divided as described above may have a maximum size of 64x64 pixels.
  • the conversion unit 120 converts the residual block, which is a residual signal between the original block of the input prediction unit PU and the prediction block generated by the intra prediction unit 150 or the inter prediction unit 160, wherein the conversion is conversion
  • the unit TU may be performed as a basic unit.
  • different transformation matrices may be determined according to a prediction mode (intra or inter), and since the residual signal of intra prediction has directionality according to the intra prediction mode, the transformation matrix may be adaptively determined according to the intra prediction mode. have.
  • the transformation unit may be transformed by two (horizontal, vertical) one-dimensional transformation matrices, for example, in the case of inter prediction, one predetermined transformation matrix may be determined.
  • a DCT-based integer matrix is applied in the vertical direction, and DST-based or in the horizontal direction.
  • KLT-based integer matrix When the intra prediction mode is vertical, a DST-based or KLT-based integer matrix may be applied in the vertical direction, and a DCT-based integer matrix may be applied in the horizontal direction.
  • a DCT-based integer matrix can be applied to both directions.
  • a transform matrix may be adaptively determined based on the size of the transform unit (TU).
  • the quantization unit 130 determines a quantization step size for quantizing the coefficients of the residual block transformed by the transform matrix, and the quantization step size may be determined for each quantization unit having a predetermined size or more.
  • the size of the quantization unit may be 8x8 or 16x16, and the quantization unit 130 quantizes coefficients of a transform block using a quantization matrix determined according to a quantization step size and a prediction mode.
  • the quantization unit 130 may use a quantization step size of a quantization unit adjacent to the current quantization unit as a quantization step size predictor of the current quantization unit.
  • the quantization unit 130 may search the left quantization unit, the upper quantization unit, and the left upper quantization unit of the current quantization unit to generate a quantization step size predictor of the current quantization unit using one or two valid quantization step sizes. have.
  • the quantization unit 130 may determine a valid first quantization step size retrieved in the order as a quantization step size predictor, or determine an average value of two valid quantization step sizes retrieved in the order as a quantization step size predictor, or If only one quantization step size is valid, it can be determined as a quantization step size predictor.
  • the quantization unit 130 transmits the difference between the quantization step size of the current quantization unit and the quantization step size predictor to the entropy encoding unit 140.
  • all of the left coding unit, the upper coding unit, and the left upper coding unit of the current coding unit do not exist.
  • quantization units adjacent to the current coding unit and the quantization step size of the previous quantization unit in the coding order may be candidates in the largest coding unit.
  • priority is set in the order of 1) the left quantization unit of the current coding unit, 2) the upper quantization unit of the current coding unit, 3) the upper left quantization unit of the current coding unit, and 4) the order of the previous quantization unit in the coding order.
  • the quantized transform block as described above is transferred to the inverse quantization unit 135 and the scanning unit 131.
  • the scanning unit 131 scans the coefficients of the quantized transform block and converts them into one-dimensional quantized coefficients.
  • the scanning method is based on the intra prediction mode. It can be decided accordingly.
  • the coefficient scanning method may be differently determined according to the size of the transform unit, and the scan pattern may be changed according to the directional intra prediction mode, and in this case, the scan order of the quantization coefficients may be scanned in the reverse direction.
  • the same scan pattern may be applied to quantization coefficients in each subset, and a zigzag scan or diagonal scan may be applied to the scan pattern between subsets.
  • the scan pattern is preferably scanned from the main subset including DC to the remaining subsets in the forward direction, but the reverse direction is also possible.
  • a scan pattern between subsets may be set in the same manner as the scan pattern of quantized coefficients in the subset, and the scan pattern between subsets may be determined according to the intra prediction mode.
  • the encoding apparatus 10 includes a decoding apparatus by including in the bitstream information indicating the position of the last non-zero quantization coefficient and the position of the last non-zero quantization coefficient in each subset in the transform unit (PU) ( 20).
  • the inverse quantization unit 135 inverse quantizes the quantized quantized coefficients as described above, and the inverse transform unit 125 performs inverse transformation in units of transform units (TU) to restore the inverse quantized transform coefficients to a residual block in a spatial domain. can do.
  • TU transform units
  • the adder 195 may generate a reconstructed block by combining the residual block reconstructed by the inverse transform unit 125 with the predicted block received from the intra predictor 150 or the inter predictor 160.
  • the post-processing unit 170 deblocking filtering process to remove the blocking effect occurring in the reconstructed picture, sample adaptive offset (Sample Adaptive Offset: to compensate for the difference value from the original image in units of pixels) SAO) application process and a coding unit can perform post-processing such as an adaptive loop filtering (ALF) process to compensate for a difference value from the original image.
  • ALF adaptive loop filtering
  • the deblocking filtering process may be applied to a boundary of a prediction unit (PU) or a transform unit (TU) having a size equal to or greater than a predetermined size.
  • PU prediction unit
  • TU transform unit
  • the deblocking filtering process may include determining a boundary to be filtered, determining a boundary filtering strength to be applied to the boundary, and determining whether to apply a deblocking filter, If it is determined to apply the deblocking filter, it may include selecting a filter to be applied to the boundary.
  • whether or not the deblocking filter is applied includes i) whether the boundary filtering intensity is greater than 0, and ii) the degree of change in pixel values at the boundary of two block (P block, Q block) adjacent to the boundary to be filtered. It may be determined by whether the value indicated is smaller than the first reference value determined by the quantization parameter.
  • At least two said filters are preferable.
  • a filter that performs relatively weak filtering is selected.
  • the second reference value is determined by the quantization parameter and the boundary filtering intensity.
  • sample adaptive offset (SAO) application process is to reduce the difference (distortion) between the pixel and the original pixel in the image to which the deblocking filter is applied, and the sample adaptive offset (SAO) application process is performed in units of pictures or slices. It can be decided whether or not to perform.
  • the picture or slice may be divided into a plurality of offset areas, and an offset type may be determined for each offset area, wherein the offset type is a predetermined number (eg, 4) edge offset types and 2 band offsets. Type.
  • the offset type is an edge offset type
  • an edge type to which each pixel belongs is determined and an offset corresponding thereto is applied
  • the edge type may be determined based on a distribution of two pixel values adjacent to the current pixel. have.
  • the adaptive loop filtering (ALF) process may perform filtering based on a value obtained by comparing a reconstructed image and an original image that have undergone a deblocking filtering process or an adaptive offset application process.
  • the picture storage unit 180 receives the post-processed image data from the post-processing unit 170 and restores and stores the image in picture units, and the picture may be a frame unit image or a field unit image.
  • the inter prediction unit 160 may perform motion estimation using at least one reference picture stored in the picture storage unit 180 and determine a reference picture index and a motion vector indicating the reference picture.
  • a prediction block corresponding to a prediction unit to be encoded may be extracted from a reference picture used for motion estimation among a plurality of reference pictures stored in the picture storage unit 180 according to the determined reference picture index and motion vector. have.
  • the intra prediction unit 150 may perform intra prediction encoding using reconstructed pixel values inside a picture in which the current prediction unit is included.
  • the intra prediction unit 150 may receive the current prediction unit to be predictively encoded and select one of a preset number of intra prediction modes according to the size of the current block to perform intra prediction.
  • the intra prediction unit 150 adaptively filters the reference pixel to generate the intra prediction block, and when the reference pixel is not available, the reference pixels may be generated using the available reference pixels.
  • the entropy encoding unit 140 may entropy encode quantization coefficients quantized by the quantization unit 130, intra prediction information received from the intra prediction unit 150, and motion information received from the inter prediction unit 160. .
  • FIG. 6 is a block diagram showing an embodiment of a configuration in which the encoding apparatus 10 performs inter prediction, and the illustrated inter prediction encoder is a motion information determination unit 161 and a motion information encoding mode determination unit 162.
  • the motion information determination unit 161 determines motion information of a current block, motion information includes a reference picture index and a motion vector, and the reference picture index is any one of pictures that have been previously encoded and reconstructed. Can represent
  • the current block is one-way inter-prediction coded, it indicates any one of the reference pictures belonging to list 0 (L0), and when the current block is two-way predictive-coded, it refers to one of the reference pictures in list 0 (L0).
  • An index and a reference picture index indicating one of the reference pictures of list 1 (L1) may be included.
  • an index indicating one or two pictures of reference pictures of the composite list LC generated by combining list 0 and list 1 may be included.
  • the motion vector indicates a position of a prediction block in a picture indicated by each reference picture index, and the motion vector may be a pixel unit (integer unit) or a sub pixel unit.
  • the motion vector may have a precision of 1/2, 1/4, 1/8, or 1/16 pixels, and when the motion vector is not an integer unit, a prediction block is generated from pixels of the integer unit. You can.
  • the motion information encoding mode determiner 162 may determine an encoding mode for motion information of a current block, and the encoding mode may be exemplified as one of a skip mode, a merge mode, and an AMVP mode.
  • the skip mode is applied when there is a skip candidate having the same motion information as the motion information of the current block, and the residual signal is 0.
  • the current block which is the prediction unit (PU)
  • the merge mode is applied when there is a merge candidate having the same motion information as the motion information of the current block, and the merge mode has a residual signal when the size of the current block is different from the coding unit (CU) or the same size. In case it applies. Meanwhile, the merge candidate and the skip candidate may be the same.
  • the AMVP mode is applied when the skip mode and the merge mode are not applied, and an AMVP candidate having a motion vector most similar to the motion vector of the current block can be selected as the AMVP predictor.
  • the encoding mode is a process other than the above-described method, and may include a more fine-grained motion compensation prediction encoding mode.
  • the adaptively determined motion compensation prediction mode includes the above-described AMVP mode, merge mode, and skip mode, as well as FRUC (FRAME RATE UP-CONVERSION) mode, BIO (BI-DIRECTIONAL OPTICAL FLOW), which is currently proposed as a new motion compensation prediction mode.
  • AMP AFFINE MOTION PREDICTION
  • OBMC OverLAPPED BLOCK MOTION COMPENSATION
  • DMVR DECODER-SIDE MOTION VECTOR REFINEMENT
  • ATMVP Alternative temporal motion vector prediction
  • STMVP Sepatial-temporal motion vector prediction
  • LIC Local Illumination Compensation
  • the motion information encoding unit 163 may encode motion information according to a method determined by the motion information encoding mode determiner 162.
  • the motion information encoding unit 163 may perform a merge motion vector encoding process when the motion information encoding mode is a skip mode or a merge mode, and may perform an AMVP encoding process in the AMVP mode.
  • the prediction block generation unit 164 generates a prediction block using motion information of the current block, and when the motion vector is an integer unit, copies a block corresponding to a position indicated by the motion vector in the picture indicated by the reference picture index and copies the current block To generate predictive blocks.
  • the prediction block generator 164 may generate pixels of the prediction block from integer unit pixels in the picture indicated by the reference picture index.
  • a prediction pixel may be generated using an 8-tap interpolation filter for a luminance pixel, and a prediction pixel may be generated using a 4-tap interpolation filter for a chrominance pixel.
  • the residual block generator 165 generates a residual block using the current block and the prediction block of the current block, and when the size of the current block is 2Nx2N, the residual block is generated using the 2Nx2N prediction block corresponding to the current block and the current block You can create blocks.
  • the size of the current block used for prediction is 2NxN or Nx2N
  • a final prediction block of 2Nx2N size is obtained by using the 2 2NxN prediction blocks. Can be created.
  • a 2Nx2N size residual block may be generated using the 2Nx2N size prediction block, and overlap smoothing may be applied to pixels of the boundary part to resolve discontinuity of the boundary part of the 2 prediction blocks having 2NxN size. You can.
  • the residual block encoder 166 divides the residual block into one or more transform units (TUs), so that each transform unit (TU) can be transform-encoded, quantized, and entropy-encoded.
  • the residual block encoder 166 may transform the residual block generated by the inter prediction method using an integer-based transform matrix, and the transform matrix may be an integer-based DCT matrix.
  • the residual block encoder 166 uses a quantization matrix to quantize the coefficients of the residual block transformed by the transform matrix, and the quantization matrix can be determined by a quantization parameter.
  • the quantization parameter is determined for each coding unit (CU) having a predetermined size or more, and if the current coding unit (CU) is smaller than the predetermined size, the first coding unit (in coding order) among coding units (CU) within the predetermined size ( Only the quantization parameter of CU) is coded, and the quantization parameter of the remaining coding unit CU is the same as the above parameter, and thus may not be coded.
  • coefficients of the transform block may be quantized using a quantization matrix determined according to the quantization parameter and a prediction mode.
  • the quantization parameter determined for each coding unit (CU) having a predetermined size or more may be predictively coded using the quantization parameter of the coding unit (CU) adjacent to the current coding unit (CU).
  • a valid first quantization parameter retrieved in the above order may be determined as a quantization parameter predictor, and a valid first quantization parameter may be quantized by searching in the order of the left coding unit (CU) and the previous coding unit (CU) in the coding order. It can be determined as a parameter predictor.
  • the coefficients of the quantized transform block are scanned and converted into one-dimensional quantized coefficients, and the scanning method may be set differently according to the entropy coding mode.
  • inter-prediction-encoded quantization coefficients can be scanned in one predetermined manner (zigzag, or raster scan in a diagonal direction), and when encoded with CAVLC, scanning in a different way from the above method Can be.
  • the scanning method may be determined according to a zigzag case in the case of inter and an intra prediction mode in case of intra, and the coefficient scanning method may be differently determined according to the size of a transform unit.
  • the scan pattern may vary according to the directional intra prediction mode, and the scan order of quantization coefficients may be scanned in the reverse direction.
  • the multiplexer 167 multiplexes the motion information encoded by the motion information encoding unit 163 and the residual signals encoded by the residual block encoding unit 166.
  • the motion information may vary according to an encoding mode, and for example, in the case of skip or merge, only the index indicating the predictor may be included, and in the case of AMVP, the reference picture index, the differential motion vector, and the AMVP index of the current block may be included. .
  • the intra prediction unit 150 receives the prediction mode information and the size of the prediction unit PU from the picture division unit 110, and the picture storage unit determines a reference pixel to determine the intra prediction mode of the prediction unit PU It can be read from 180.
  • the intra prediction unit 150 determines whether a reference pixel is generated by examining whether there is an unavailable reference pixel, and the reference pixels can be used to determine an intra prediction mode of the current block.
  • pixels adjacent to the upper side of the current block are not defined, and when the current block is located at the left boundary of the current picture, pixels adjacent to the left of the current block are not defined, It may be determined that the pixels are not available pixels.
  • the current block is located at the slice boundary and pixels adjacent to the upper or left side of the slice are not available pixels even if the pixels are not encoded and reconstructed first.
  • the intra prediction mode of the current block may be determined using only available pixels.
  • a reference pixel at a location that is not available may be generated using the available reference pixels of the current block. For example, when the pixels of the upper block are not available, the upper pixel may be used using some or all of the left pixels. You can create them, and vice versa.
  • a reference pixel is generated by copying an available reference pixel at a location closest to a predetermined direction from a reference pixel at a location that is not available, or when there is no reference pixel available in a predetermined direction, the closest in the opposite direction
  • a reference pixel may be generated by copying the available reference pixel of the position.
  • the upper or left pixels of the current block may be determined as a reference pixel that is not available according to an encoding mode of a block to which the pixels belong.
  • the pixels may be determined as unavailable pixels.
  • available reference pixels may be generated using pixels belonging to a reconstructed block in which a block adjacent to the current block is intra coded, and information indicating that the encoding apparatus 10 determines available reference pixels according to an encoding mode. It is transmitted to the decoding device 20.
  • the intra prediction unit 150 determines the intra prediction mode of the current block using the reference pixels, and the number of intra prediction modes allowable for the current block may vary according to the size of the block.
  • 34 intra prediction modes may exist when the size of the current block is 8x8, 16x16, and 32x32, and 17 intra prediction modes may exist when the size of the current block is 4x4.
  • the 34 or 17 intra prediction modes may be composed of at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).
  • the one or more non-directional modes may be DC mode and / or planar mode.
  • 35 intra prediction modes may exist regardless of the size of the current block.
  • DC mode and planner mode two non-directional modes (DC mode and planner mode) and 33 directional modes may be included.
  • At least one pixel value (or a prediction value of the pixel value, hereinafter referred to as a first reference value) and a reference pixel positioned at a bottom-right of the current block is used to predict the prediction block of the current block. Is generated.
  • the configuration of the video decoding apparatus may be derived from the configuration of the video encoding apparatus 10 described with reference to FIGS. 1 to 6, for example, as described with reference to FIGS. 1 to 6.
  • An image can be decoded by inversely performing the processes of the same image encoding method.
  • the decoding apparatus 20 includes an entropy decoding unit 210, an inverse quantization / inverse transformation unit 220, an adder 270, It has a post-processing unit 250, a picture storage unit 260, an intra prediction unit 230, a motion compensation prediction unit 240, and an intra / inter switch 280.
  • the entropy decoding unit 210 receives and decodes the encoded bit stream from the image encoding apparatus 10, separates it into intra prediction mode indexes, motion information, and quantization coefficient sequences, and decodes the decoded motion information into a motion compensation prediction unit ( 240).
  • the entropy decoding unit 210 transmits the intra prediction mode index to the intra prediction unit 230 and the inverse quantization / inverse transformation unit 220 to transmit the inverse quantization coefficient sequence to the inverse quantization / inverse transformation unit 220.
  • the inverse quantization / inverse transform unit 220 converts the quantization coefficient sequence into an inverse quantization coefficient in a two-dimensional array, and can select one of a plurality of scanning patterns for the conversion, for example, a prediction mode (ie, a current block) (Intra prediction or inter prediction) and an intra prediction mode.
  • a prediction mode ie, a current block
  • Intra prediction or inter prediction Intra prediction mode
  • the inverse quantization / inverse transform unit 220 restores a quantization coefficient by applying a quantization matrix selected from a plurality of quantization matrices to an inverse quantization coefficient of a two-dimensional array.
  • a quantization matrix may be selected for a block having the same size based on at least one of the prediction mode and the intra prediction mode of the current block.
  • the inverse quantization / inverse transform unit 220 inversely transforms the reconstructed quantization coefficient to restore a residual block, and the inverse transform process may be performed using a transform unit (TU) as a basic unit.
  • TU transform unit
  • the adder 270 reconstructs the image block by combining the residual block reconstructed by the inverse quantization / inverse transform unit 220 and the prediction block generated by the intra prediction unit 230 or the motion compensation prediction unit 240.
  • the post-processing unit 250 may perform post-processing on the reconstructed image generated by the adder 270 to reduce deblocking artifacts and the like due to image loss due to quantization by filtering or the like.
  • the picture storage unit 260 is a frame memory for storing a local decoded image in which filter post-processing is performed by the post-processing unit 250.
  • the intra prediction unit 230 restores the intra prediction mode of the current block based on the intra prediction mode index received from the entropy decoding unit 210 and generates a prediction block according to the restored intra prediction mode.
  • the motion compensation prediction unit 240 generates a prediction block for a current block from a picture stored in the picture storage unit 260 based on the motion vector information, and applies a selected interpolation filter to apply the selected interpolation filter when motion compensation with a decimal precision is applied.
  • the intra / inter switch 280 may provide the adder 270 with a prediction block generated by any one of the intra prediction unit 230 and the motion compensation prediction unit 240 based on the encoding mode.
  • the inter prediction decoder is a demultiplexer 241, a motion information encoding mode determination unit 242, a merge mode motion
  • the de-multiplexer 241 demultiplexes the currently encoded motion information and the encoded residual signals from the received bitstream, and transmits the demultiplexed motion information to the motion information encoding mode determination unit 242 Then, the demultiplexed residual signal may be transmitted to the residual block decoder 246.
  • the motion information encoding mode determining unit 242 determines the motion information encoding mode of the current block, and when the skip_flag of the received bitstream has a value of 1, the motion information encoding mode of the current block is determined to be encoded in the skip encoding mode can do.
  • the motion information encoding mode determining unit 242 is a motion information encoding mode of the current block It can be determined that is encoded in the merge mode.
  • the motion information encoding mode determining unit 242 has a value of 0 for skip_flag of the received bitstream, and motion information received from the demultiplexer 241 has a reference picture index, a differential motion vector, and an AMVP index. In this case, it may be determined that the motion information encoding mode of the current block is encoded in the AMVP mode.
  • the merge mode motion information decoding unit 243 is activated when the motion information encoding mode determining unit 242 determines the current block motion information encoding mode as skip or merge mode, and the AMVP mode motion information decoding unit 244 moves It may be activated when the information encoding mode determining unit 242 determines the current block motion information encoding mode as the AMVP mode.
  • the selection mode motion information decoding unit 248 may decode motion information in a prediction mode selected from among other motion compensation prediction modes except for the above-described AMVP mode, merge mode, and skip mode.
  • the selective prediction mode may include a more precise motion prediction mode compared to the AMVP mode, and may be determined block-adaptively according to predetermined conditions (eg, block size and block segmentation information, signaling information existence, block position, etc.). .
  • Selective prediction mode is, for example, FRUC (FRAME RATE UP-CONVERSION) mode, BIO (BI-DIRECTIONAL OPTICAL FLOW) mode, AMP (AFFINE MOTION PREDICTION) mode, OBMC (OVERLAPPED BLOCK MOTION COMPENSATION) mode, DMVR (DECODER-SIDE) It may include at least one of a MOTION VECTOR REFINEMENT mode, an ATMVP (Alternative temporal motion vector prediction) mode, a STMVP (Spatial-temporal motion vector prediction) mode, and a LIC (Local Illumination Compensation) mode.
  • FRUC FRAME RATE UP-CONVERSION
  • BIO BIO
  • AMP AFFINE MOTION PREDICTION
  • OBMC OverLAPPED BLOCK MOTION COMPENSATION
  • DMVR DECODER-SIDE
  • the prediction block generator 245 generates a prediction block of the current block by using the motion information restored by the merge mode motion information decoder 243 or the AMVP mode motion information decoder 244.
  • a block corresponding to a position indicated by the motion vector in the picture indicated by the reference picture index may be copied to generate a prediction block of the current block.
  • pixels of a prediction block are generated from integer unit pixels in a picture indicated by a reference picture index.
  • an 8-tap interpolation filter is used for a luminance pixel and a color difference pixel Prediction pixels may be generated using a 4-tap interpolation filter.
  • the residual block decoder 246 entropy-decodes the residual signal and inversely scans the entropy-decoded coefficients to generate a two-dimensional quantized coefficient block, and the inverse scanning method may vary according to the entropy decoding method.
  • the inverse scanning method may be applied in a diagonal raster inverse scanning method when decoded based on CABAC or in a zigzag inverse scan method when decoded based on CAVLC.
  • the inverse scanning method may be differently determined according to the size of the prediction block.
  • the residual block decoding unit 246 may inverse quantize the coefficient block generated as described above using an inverse quantization matrix, and reconstruct a quantization parameter to derive the quantization matrix.
  • the quantization step size may be restored for each coding unit having a predetermined size or more.
  • the residual block decoding unit 260 inversely transforms the inverse-quantized coefficient block to restore the residual block.
  • the reconstructed block generator 270 generates a reconstructed block by adding the predicted block generated by the predicted block generator 250 and the residual block generated by the residual block decoder 260.
  • the intra prediction mode of the current block is decoded from the received bitstream, and for that purpose, the entropy decoding unit 210 restores the first intra prediction mode index of the current block by referring to one of the plurality of intra prediction mode tables. You can.
  • any one table selected according to the distribution of intra prediction modes for multiple blocks adjacent to the current block may be applied.
  • the first intra prediction mode table is applied to restore the index of the first intra prediction mode of the current block, and is not the same. Otherwise, the first intra prediction mode index of the current block may be restored by applying the second intra prediction mode table.
  • the intra prediction mode of the upper block and the left block of the current block are both directional intra prediction modes
  • the direction of the intra prediction mode of the upper block and the intra prediction mode of the left block If it is within a predetermined angle, the first intra prediction mode index of the current block is restored by applying the first intra prediction mode table, and if it is outside the predetermined angle, the second intra prediction mode table is applied to the first intra prediction mode index of the current block. Can also be restored.
  • the entropy decoding unit 210 transmits the first intra prediction mode index of the restored current block to the intra prediction unit 230.
  • the intra prediction unit 230 receiving the index of the first intra prediction mode may determine the maximum possible mode of the current block as the intra prediction mode of the current block. .
  • the intra prediction unit 230 compares the index indicated by the maximum possible mode of the current block with the index of the first intra prediction mode, and as a result of the comparison, the first intra prediction mode If the index is not smaller than the index indicated by the maximum possible mode of the current block, the intra prediction mode corresponding to the second intra prediction mode index obtained by adding 1 to the first intra prediction mode index is determined as the intra prediction mode of the current block. Otherwise, the intra prediction mode corresponding to the first intra prediction mode index may be determined as the intra prediction mode of the current block.
  • the intra prediction mode allowable for the current block may include at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).
  • the one or more non-directional modes may be DC mode and / or planar mode.
  • either the DC mode or the planner mode may be adaptively included in the allowable intra prediction mode set.
  • information specifying a non-directional mode included in the allowable intra prediction mode set may be included in a picture header or a slice header.
  • the intra prediction unit 230 reads the reference pixels from the picture storage unit 260 to generate an intra prediction block, and determines whether there is an unavailable reference pixel.
  • the determination may be made according to the presence or absence of reference pixels used to generate an intra prediction block by applying the decoded intra prediction mode of the current block.
  • the intra prediction unit 230 may generate reference pixels at a location that is not available using previously reconstructed available reference pixels.
  • the definition of a reference pixel that is not available and the method of generating the reference pixel may be the same as the operation of the intra prediction unit 150 according to FIG. 1, but an intra prediction block is generated according to the decoded intra prediction mode of the current block.
  • the reference pixels used for this may be selectively restored.
  • the intra prediction unit 230 determines whether to apply a filter to reference pixels to generate a prediction block, that is, whether to apply filtering to reference pixels to generate an intra prediction block of the current block. It can be determined based on the decoded intra prediction mode and the size of the current prediction block.
  • the problem of blocking artifacts increases as the size of the block increases, so as the size of the block increases, the number of prediction modes for filtering the reference pixel can be increased, but if the block becomes larger than a predetermined size, it can be seen as a flat area, reducing complexity. For example, the reference pixel may not be filtered.
  • the intra prediction unit 230 filters the reference pixels using a filter.
  • At least two or more filters may be adaptively applied according to the difference in the level difference between the reference pixels.
  • the filter coefficient of the filter is preferably symmetrical.
  • the above two or more filters may be adaptively applied according to the size of the current block.
  • a filter having a narrow bandwidth for a small block and a filter having a wide bandwidth for a large block May be applied.
  • whether or not filtering is applied is also related to the intra prediction mode of the current block, so that the reference pixel can be filtered adaptively based on the intra prediction mode of the current block and the size of the prediction block.
  • the intra prediction unit 230 generates a prediction block using reference pixels or filtered reference pixels according to the restored intra prediction mode, and generation of the prediction block is the same as that of the operation of the encoding apparatus 10. Since it can be, a detailed description thereof will be omitted.
  • the intra prediction unit 230 determines whether to filter the generated prediction block, and the filtering may be determined by using information included in a slice header or a coding unit header or according to an intra prediction mode of the current block.
  • the intra prediction unit 230 may generate a new pixel by filtering a pixel at a specific position of the generated prediction block using available reference pixels adjacent to the current block. .
  • a prediction pixel contacting the reference pixels among the prediction pixels may be filtered using a reference pixel contacting the prediction pixel.
  • the prediction pixel is filtered using one or two reference pixels according to the position of the prediction pixel, and filtering of the prediction pixel in DC mode can be applied to prediction blocks of all sizes.
  • prediction pixels in contact with the left reference pixel among the prediction pixels of the prediction block may be changed using reference pixels other than the upper pixel used to generate the prediction block.
  • prediction pixels that come into contact with the upper reference pixel among the generated prediction pixels may be changed using reference pixels other than the left pixel used to generate the prediction block.
  • the current block may be reconstructed using the prediction block of the current block reconstructed and the residual block of the decoded current block.
  • FIG. 9 is a diagram for describing a second embodiment of a method of dividing and processing an image in block units.
  • a coding tree unit (CTU) having a maximum size of 256x256 pixels is first divided into a quad tree structure, and can be divided into four coding units (CUs) having a square shape.
  • At least one of the coding units divided into the quad tree structure is divided into a binary tree structure, and may be re-divided into two coding units (CUs) having a rectangular shape.
  • At least one of the coding units divided into the quad tree structure may be divided into a quad tree structure and re-divided into four coding units (CUs) having a square shape.
  • CUs coding units
  • At least one of the coding units re-divided into the binary tree structure may be divided into a binary tree structure and divided into two coding units (CUs) having a square or rectangular shape.
  • At least one of the coding units re-divided into the quad tree structure may be divided into a quad tree structure or a binary cree structure, and may be divided into coding units (CUs) having a square or rectangular shape.
  • CUs coding units
  • the binary partitioned CU may include a coding block (CB) that is a block unit that performs actual sub / decoding and syntax corresponding to the corresponding coding block. That is, the size of the prediction unit PU and the transformation unit TU belonging to the coding block CB as shown in FIG. 9 may be the same as the size of the corresponding coding block CB.
  • CB coding block
  • the coding unit split into the quad tree structure may be divided into one or more prediction units (PUs) using the method as described with reference to FIGS. 3 and 4.
  • the coding unit divided into a quad tree structure may be divided into one or two or more transform units (TUs) using the method as described with reference to FIG. 5, and the split transform unit (TU) Can have a maximum size of 64x64 pixels.
  • FIG. 10 illustrates an embodiment of a syntax structure used to process an image by dividing it into blocks.
  • a block structure according to an embodiment of the present invention may be determined through split_cu_flag indicating whether to split a quad tree and binary_split_flag indicating whether to split a binary tree.
  • whether to split the coding unit (CU) as described above may be indicated using split_cu_flag.
  • binary_split_flag indicating whether to split or not and syntax indicating the split direction may be determined in correspondence to a binary partitioned CU after quad tree splitting.
  • a method of indicating the directionality of binary splitting a method of determining a splitting direction based on this by decoding a plurality of syntaxes such as binary_split_hor and binary_split_ver, or decoding a single syntax and signal values according to it, such as binary_split_mode, and Horizontal (0)
  • a method of processing division in the vertical (1) direction may be exemplified.
  • the depth of a coding unit (CU) split using a binary tree may be represented using binary_depth.
  • coding unit e.g., a coding unit (CU), a prediction unit (PU), and a transform unit (TU)
  • PU prediction unit
  • TU transform unit
  • the coding unit may be divided into a binary tree structure and divided into transform units (TUs), which are basic units for transforming residual blocks.
  • TUs transform units
  • At least one of rectangular coding blocks CU 0 and Cu 1 divided into a binary tree structure and having a size of Nx2N or 2NxN is divided into a binary tree structure, and the size of NxN It can be divided into square transform units (TU 0 , TU 1 ).
  • the block-based image encoding method may perform prediction, transform, quantization, and entropy encoding steps.
  • a prediction signal is generated by referring to a block performing current encoding and an existing coded image or a surrounding image, and through this, a difference signal from the current block can be calculated.
  • the difference signal is used as input to perform conversion using various conversion functions, and the converted signal is classified into DC coefficients and AC coefficients to be energy compacted to improve encoding efficiency. You can.
  • quantization is performed with transform coefficients as an input, and then entropy encoding is performed on the quantized signal, so that an image may be encoded.
  • the image decoding method proceeds in the reverse order of the encoding process as described above, and an image quality distortion phenomenon may occur in the quantization step.
  • the size or shape of the transform unit (TU) and the type of transform function applied can be varied according to the distribution of the difference signal input to the input in the transform step and the characteristics of the image. have.
  • a difference is measured using a cost measurement method such as SAD (Sum of Absolute Difference) or MSE (Mean Square error).
  • SAD Sum of Absolute Difference
  • MSE Mel Square error
  • efficient encoding can be performed by selectively performing the transformation by determining the size or shape of the transformation unit CU based on the distribution of various difference signals.
  • the DC value generally represents the average value of the input signal
  • two coding units CUx
  • TUs conversion units
  • a square coding unit (CU 0 ) having a size of 2Nx2N is divided into a binary tree structure, and can be divided into rectangular transform units (TU 0 and TU 1 ) having a size of Nx2N or 2NxN. .
  • the step of dividing the coding unit (CU) into a binary tree structure may be performed repeatedly two or more times, and divided into a plurality of transform units (TUs).
  • a rectangular coding block (CB 1 ) having a size of Nx2N is divided into a binary tree structure, and a block having the size of the divided NxN is further divided into a binary tree structure to N / 2xN or NxN /
  • the block having the size of N / 2xN or NxN / 2 is divided into a binary tree structure, and square transform units having a size of N / 2xN / 2 (TU 1 , TU 2 , TU 4 , TU 5 ).
  • a square coding unit (CU 0 ) having a size of 2Nx2N is divided into a binary tree structure, and a block having the size of the divided Nx2N is divided into a binary tree structure, and a square having a size of NxN is obtained.
  • the block having the size of NxN may be further divided into a binary tree structure and divided into rectangular transform units (TU 1 and TU 2 ) having the size of N / 2xN.
  • a rectangular coding unit (CU 0 ) having a size of 2NxN is divided into a binary tree structure, and a block having the size of the divided NxN is divided into a quad tree structure to generate N / 2xN / 2. It can be divided into square transform units having a size (TU 1 , TU 2 , TU 3 , TU 4 ).
  • blocks eg, a coding unit (CU), a prediction unit (PU), and a transform unit (TU)
  • CU coding unit
  • PU prediction unit
  • TU transform unit
  • the picture dividing unit 110 provided in the image encoding apparatus 10 performs rate distortion optimization (RDO) according to a preset order, and as described above, the dividable coding unit (CU), prediction unit (PU), and transform
  • RDO rate distortion optimization
  • CU dividable coding unit
  • PU prediction unit
  • transform The division structure of the unit TU can be determined.
  • the picture division unit 110 determines an optimal block division structure in terms of bitrate and distortion while performing rate distortion optimization-quantization (RDO-Q). You can.
  • RDO-Q rate distortion optimization-quantization
  • RD may be performed in the order of 2NxN pixel sized conversion unit (PU) split structure shown in (d) to determine the optimal split structure of the transform unit (PU).
  • PU 2NxN pixel sized conversion unit
  • the coding unit CU has a form of Nx2N or 2NxN pixel size
  • the pixel size of Nx2N (or 2NxN) shown in (a) the pixel size of NxN shown in (b), Pixel size of N / 2xN (or NxN / 2) and NxN shown in (c), N / 2xN / 2, N / 2xN and pixel size of NxN shown in (d), N shown in (e)
  • It is possible to determine the optimal division structure of the conversion unit PU by performing RDO in the order of the division structure of the conversion unit (PU) having a pixel size of / 2xN.
  • the block division method of the present invention has been described as an example in which a block division structure is determined by performing rate distortion optimization (RDO), but the picture division unit 110 has a Sum of Absolute Difference (SAD) or Mean Square Error (MSE). ), It is possible to maintain a proper efficiency while reducing the complexity by determining the block division structure.
  • RDO rate distortion optimization
  • MSE Mean Square Error
  • ALF adaptive loop filtering
  • whether to apply the adaptive loop filter (ALF) may be determined on a coding unit (CU) basis, and the size or coefficient of a loop filter to be applied may vary according to the coding unit (CU).
  • information indicating whether to apply the adaptive loop filter (ALF) for each coding unit (CU) may be included in each slice header.
  • a chrominance signal it may be determined whether to apply an adaptive loop filter (ALF) on a picture-by-picture basis, and the shape of the loop filter may have a rectangular shape unlike luminance.
  • ALF adaptive loop filter
  • adaptive loop filtering may determine whether to apply for each slice. Accordingly, information indicating whether adaptive loop filtering (ALF) is applied to the current slice may be included in a slice header or a picture header.
  • the slice header or picture header may additionally include information indicating the filter length in the horizontal and / or vertical direction of the luminance component used in the adaptive loop filtering process.
  • the slice header or the picture header may include information indicating the number of filter sets, and when the number of filter sets is 2 or more, filter coefficients may be encoded using a prediction method.
  • the slice header or the picture header may include information indicating whether filter coefficients are encoded by a prediction method, and when the prediction method is used, may include predicted filter coefficients.
  • information indicating whether each chrominance component is filtered may be included in a slice header or a picture header, and for Cr and Cb to reduce the number of bits.
  • Joint coding ie, multiplexing coding
  • information indicating whether to filter may be included in a slice header or a picture header, and for Cr and Cb to reduce the number of bits.
  • Joint coding ie, multiplexing coding
  • entropy coding may be performed by assigning the largest index.
  • 19 is a view for explaining a composite partition structure according to another embodiment of the present invention.
  • the coding unit CU is divided into a binary tree structure, a rectangle having a shape in which the horizontal length W as shown in FIG. 19 (A) is longer than the vertical length H, and a vertical length as shown in FIG. 19 (B).
  • the shape of the coding unit (CU) in which H is divided into a rectangle having a shape longer than the width W may appear.
  • it is highly likely that the encoding information is concentrated in the left and right or upper and lower edge regions compared to the middle region.
  • the encoding apparatus 10 can facilitate the edge area of a coding unit, which has a long specific direction, by splitting quad trees and binary trees.
  • the coding unit can be split into a ternary tree or triple tree structure that can be split.
  • FIG. 19 (A) shows a first area of the left edge having a horizontal W / 8 and a vertical H / 4 length, and a horizontal W / 8 * 6, vertical when the coding unit to be divided is a horizontally divided coding unit.
  • H / 4 length it is shown that the second region, which is the middle region, and the third region of the right edge of the horizontal W / 8 and the vertical H / 4 length can be struck.
  • the encoding apparatus 10 may process the splitting of the ternary tree structure through the picture splitter 110.
  • the picture divider 110 may not only determine the division into the above-described quad-tree and binary-tree structures according to encoding efficiency, but may also fine-tune the segmentation scheme by considering the ternary tree structure together.
  • the division of the ternary tree structure may be processed for all coding units without limitation. However, considering the encoding and decoding efficiency as described above, it may be desirable to allow a ternary tree structure only for coding units having specific conditions.
  • the ternary tree structure may require ternary division of various methods for the coding tree unit, but it may be desirable to allow only an optimized predetermined form in consideration of encoding and decoding complexity and transmission bandwidth by signaling.
  • the picture division unit 110 may determine and determine whether to divide the current coding unit into a ternary tree structure of a specific type only when the preset coding condition is met.
  • the split ratio of the binary tree can be extended and varied to 3: 1, 1: 3, etc., not only 1: 1.
  • the splitting structure of the coding unit according to the embodiment of the present invention may include a composite tree structure that is subdivided into quad trees, binary trees, or ternary trees according to ratios.
  • the picture division unit 110 may determine a complex division structure of the coding unit to be divided based on the division table.
  • the picture dividing unit 110 processes quad-tree splitting and quad-tree-divided in response to a maximum size of a block (eg, pixel-based 128 x 128, 256 x 256, etc.) It is possible to perform a complex partitioning process that processes at least one of a dual tree structure and a triple tree structure partition corresponding to the terminal node.
  • a maximum size of a block eg, pixel-based 128 x 128, 256 x 256, etc.
  • the picture partitioning unit 110 may include a first binary partition (BINARY 1) and a second binary partition (BINARY 2) that are binary tree partitions corresponding to characteristics and sizes of a current block according to a partition table. ) And a first ternary partition (TRI 1) or a second ternary partition (TRI 2), which is a ternary tree partition, may be determined.
  • the first binary division may correspond to a vertical or horizontal division having a ratio of N: N
  • the second binary division may correspond to a vertical or horizontal division having a ratio of 3N: N or N: 3N
  • each The binary partitioned root CU may be divided into CU0 and CU1 of each size specified in the partition table.
  • the first ternary division may correspond to a vertical or horizontal division having a ratio of N: 2N: N
  • the second ternary division may correspond to a vertical or horizontal division having a ratio of N: 6N: N
  • each The ternary partitioned root CU may be divided into CU0, CU1, and CU2 of each size specified in the partition table.
  • a partition table indicating each processable partitioning structure and a coding unit size in the case of partitioning may be determined.
  • the picture division unit 110 according to an embodiment of the present invention, the maximum coding unit size and the minimum coding unit size for applying the first binary division, the second binary division, the first ternary division or the second ternary division Can be set respectively.
  • the allowable splitting structure for each size of each coding unit may be predefined.
  • the picture dividing unit 110 may prevent a case in which the horizontal or vertical pixel size is divided into 2 as a minimum size, for example, a size of less than 4, and for this purpose, the size of the block to be divided Determine whether the first binary partition, the second binary partition, the first ternary partition or the second ternary partition is allowed, and compare the RDO performance operation corresponding to the allowable partitioning structure to determine the optimal partitioning structure. You can.
  • the binary dividing structure may be divided into CU0 and CU1 constituting any one of 1: 1, 3: 1, or 1: 3 vertical partitioning.
  • the ternary division structure may be divided into CU0, CU1, and CU2 constituting either one of 1: 2: 1 or 1: 6: 1 vertical division.
  • an allowable vertical division structure may be limitedly determined depending on the size of the coding unit to be divided.
  • the vertical division structure of the 64X64 coding unit and the 32X32 coding unit may allow all of the first binary division, the second binary division, the first ternary division and the second ternary division, but among the vertical division structures of the 16X16 coding unit.
  • the second strikeout division may be limited to impossible.
  • only the first binary division may be limitedly allowed. Thus, partitioning into blocks below the minimum size that causes complexity can be prevented in advance.
  • the binary-divided structure may be divided into CU0 and CU1 constituting any one of 1: 1, 3: 1, or 1: 3 horizontal divisions
  • the ternary division structure may be divided into CU0, CU1 and CU2 constituting either one of 1: 2: 1 or 1: 6: 1 horizontal division.
  • an allowable horizontal division structure may be limitedly determined.
  • the horizontal division structure of the 64X64 coding unit and the 32X32 coding unit may allow all of the first binary division, the second binary division, the first ternary division and the second ternary division, but among the horizontal division structures of the 16X16 coding unit
  • the second strikeout division may be limited to impossible.
  • only the first binary division may be limitedly permitted.
  • the picture dividing unit 110 horizontally processes the coding unit vertically divided into the first binary division or the second binary division, or horizontally divides the first ternary division or the second ternary division according to the division table. You can.
  • the picture division unit 110 divides into CU0 and CU1 of 32X32 according to the first binary division, or CX and CU1 of 32X48 and 32X16 according to the second binary division.
  • the first ternary division 32X32, 32X16, 32X16 CU0, CU1, CU2, or according to the second ternary division 32X8, 64X48, 32X8 CU0, CU1, CU2.
  • the picture splitter 110 may vertically process the horizontally divided coding unit as the first binary split or the second binary split, or vertically split the first split or the second ternary split.
  • the picture division unit 110 may be divided into CU0 and CU1 of 16X16 according to the first binary division, or C0 and CU1 of 24X16 8X16 according to the second binary division.
  • the first ternary division it may be divided into CU0, CU1, CU2 of 8X16, 16X16, 8X16, or divided into CU0, CU1, CU2 of 4X16, 24X16, 4X16 according to the second ternary division.
  • the partitioning allowable structure may be conditionally determined differently for each CTU size, CTU group unit, and slice unit, and vertical and horizontal directions, such that the first binary partition, the second binary partition, the first ternary partition, and the second ternary partition
  • each CU partition ratio and decision size information may be defined by a partition table, or condition information may be set in advance.
  • division processing by allowing conditional division using a binary tree and a ternary tree, division of an appropriate ratio according to characteristics of the coding unit is possible, and thus encoding efficiency can be improved.
  • 20 is a block diagram illustrating a quantization unit according to an embodiment of the present invention in more detail.
  • the encoding and decoding process according to an embodiment of the present invention also corresponds to a coding tree unit (CTU) -based quantization group unit block for accurate and efficient prediction And a selective adaptive quantization parameter determination and quantization process.
  • CTU coding tree unit
  • the quantization parameter determination and quantization process may be performed by a specific configuration of the quantization unit 130 as illustrated in FIG. 20, and the operation of the quantization unit 130 in both the encoding and decoding processes according to an embodiment of the present invention And processing can be performed.
  • the quantization unit 130 may selectively adaptively determine a quantization parameter according to the quantization process of the target block, based on the quantization group unit information of the quantization target block.
  • the quantization unit 130 includes an initial quantization parameter derivation unit 1310, a characteristic adaptive quantization parameter determination unit 1320, a group unit difference quantization parameter derivation unit 1330, and coding block unit adaptation. And an enemy quantization parameter application unit 1340.
  • the initial quantization parameter deriving unit 1310 derives an initial quantization parameter (QUANTIZATION PARAMETER, QP) corresponding to each unit block.
  • the initial quantization parameter may be determined for each quantization group unit block corresponding to a CTU unit in a slice, and the initial quantization parameter derivation unit 1310 may include an initial quantization parameter determined for the quantization group unit block.
  • a list of quantization parameter (INITIAL QP) may be set in advance, and the initial quantization parameter list may be updated based on the determined difference quantization parameter (DELTA QP) value.
  • the quantization unit block may be determined corresponding to blocks grouped within a CTU unit in one slice, and the initial quantization parameter derivation unit 1310 may determine detailed initial quantization parameters for each unit. Accordingly, according to an embodiment of the present invention, an initial quantization parameter corresponding to at least one of a temporal spatial region (REGION) of a picture such as a VR image, or a subpicture, tile, slice in a picture, or the CTU-based quantization group unit Can be determined.
  • the initial quantization parameter derivation unit 1310 may construct an initial quantization parameter list based on the determined initial quantization parameter, and perform update processing corresponding thereto.
  • 21 is a flowchart illustrating an operation of the initial quantization parameter derivation unit 1310 according to an embodiment of the present invention.
  • the initial quantization parameter derivation unit 1310 first determines a division unit and a CTU quantization group region (S1001).
  • the initial quantization parameter derivation unit 1310 may determine a division unit and a CTU quantization group region based on image characteristics and picture division information obtained from the division unit 110.
  • the initial quantization parameter derivation unit 1310 configures an initial QP list based on the determined division unit and quantization group unit region (S1003).
  • the initial quantization parameter deriving unit 1310 derives an offset value based on the initial QP index of the configured initial QP list (S1005).
  • the initial quantization parameter derivation unit 1310 sets an initial QP corresponding to the determined division unit and CTU-based quantization group unit region, respectively (S1007).
  • an initial quantization parameter for each slice unit may be determined, and initial quantization for each CTU group unit included in the slice unit
  • the parameter may be determined as a value corresponding to the index.
  • the differential quantization parameter may also determine the differential quantization parameter for each slice unit and the differential quantization parameter for each CTU quantization group unit, and the differential quantization parameter for each CTU quantization group unit may be identified by a separate CTU QP index value. Can be processed.
  • Figure 23 shows the operation of the initial quantization parameter derivation unit 1310 for calculating the subdivided initial QP value for each CTU quantization unit block.
  • the initial quantization parameter derivation unit 1310 uses PPS (PICTURE) from the header information of the image. PARAMETER SET) is parsed (S1011), and an initial QP value corresponding to the picture division unit is derived (S1013).
  • the initial quantization parameter deriving unit 1310 obtains slice header information from the header information, derives a slice delta QP that is a difference value, and calculates an initial slice QP value for each slice unit based on the slice delta QP (S1017). .
  • the initial quantization parameter derivation unit 1310 determines a CTU quantization unit division, derives a CTU delta QP list for updating a difference value (S1019), and each CTU quantization unit corresponding thereto A CTU delta QP index for each block is determined (S1021).
  • the initial quantization parameter derivation unit 1310 may generate and update an initial quantization parameter list for each CTU quantization unit block by deriving an initial CTU QP value based on the CTU delta QP index for each CTU quantization unit block (S1023). have.
  • the characteristic adaptive quantization parameter determiner 1320 selectively updates and adapts a process for quantization parameter values corresponding to each CTU quantization group unit block and processes according to the image characteristics. It is possible to determine the quantization parameter accordingly, and it can be determined for each CTU quantization group unit block.
  • the characteristic adaptive quantization parameter determiner 1320 may set a quantization step size (QP STEP SIZE) and a quantization range (QP RANGE) for each CTU quantization group unit according to the determined process.
  • QP STEP SIZE quantization step size
  • QP RANGE quantization range
  • the characteristic adaptive quantization parameter determining unit 1320 sets the quantization step size (QP STEP SIZE) and quantization range (QP RANGE) for each CTU quantization group unit corresponding to the initial quantization parameter determined by the initial quantization parameter derivation unit 1310. By setting, it is possible to determine the final quantization parameter corresponding to this.
  • the determined quantization parameter value may be transmitted to the group-level difference quantization parameter derivation unit 1330.
  • the characteristic adaptive quantization parameter determiner 1320 illustrates the characteristic adaptive quantization parameter determiner 1320 in more detail.
  • the characteristic adaptive quantization parameter determiner 1320, an image case-based quantization parameter derivation unit 1321, slices A selection adaptive quantization parameter determination process may be processed by including at least one of a type-based quantization parameter derivation unit 1323, a color sample-based quantization parameter derivation unit 1325, and a transform coefficient-based quantization parameter derivation unit 1327.
  • the image case-based quantization parameter derivation unit 1321 may determine a process for deriving quantization parameters for each case according to input image characteristics. Depending on the derivation process, the quantization step size and QP range can be determined for each case.
  • CASE 2 When the input image is a multi-view image or multiple images for each synchronized area are merged into one input image; EX) VR video such as Cube Map / ERP
  • CASE 3 When a single picture is divided into a plurality of input video segmentation units by slice / tile / region, etc .;
  • the image case-based quantization parameter derivation unit 1321 determines different quantization step sizes and QP ranges for Case 2 and Case 3 for each subregion REGION corresponding to a user's viewpoint in each division unit or a picture division unit. It can be processed to apply the Initial QP and update accordingly.
  • the image case-based quantization parameter derivation unit 1321 in applying the initial QP, determines different quantization step sizes and QP ranges for each CTU quantization group unit in the slice initial QP value, and thus delta QP By processing to be applied, various offset values corresponding to regions divided in one input image may be processed.
  • the image case-based quantization parameter derivation unit 1321 may store delta QPs corresponding to each split unit image as a list corresponding to CTU quantization group units. And, the delta QPs can be matched to the offset in the list, and index values are assigned for each offset, which can be used to distinguish each other.
  • the image case-based quantization parameter derivation unit 1321 derives the offset value as a delta value for adjusting the initial QP in the slice, or the CTU quantization group unit.
  • the corresponding delta QP (CTU_dQP) value can be derived directly. Accordingly, the image case-based quantization parameter derivation unit 1321 may process different quantization step sizes and QP ranges between units of each CTU quantization group, even if they are within one picture, and calculate an Initial QP value.
  • the image case-based quantization parameter derivation unit 1321 determines the CTU quantization group unit for adjusting the initial quantization parameter value based on the image case, and the quantization step size and QP range are determined for each group, and separately The delta QP value can be processed to be updated as an offset value.
  • the video case-based quantization parameter deriving unit 1321 configures an initial QP list for each CTU quantization group unit that can be applied for each CTU based on each video characteristic case, and each delta QP corresponding to the index of the list. You can handle the process of deriving the offset value.
  • the image case-based quantization parameter derivation unit 1321 may set in advance the quantization step size and QP range and the number and size of QPs of the corresponding CTU quantization group unit or CTU_Initial_QP_List using slice header information or picture header information. .
  • the video case-based quantization parameter derivation unit 1321 may include video header information of a video parameter set (VPS), sequence parameter set (SPS), or a type of video encoded from a separate SEI message and a projection format. Alternatively, it is possible to derive flag information or the like that can identify it.
  • the encoded image may be determined using the corresponding encoding information, and an initial quantization parameter according to a picture division unit (eg, a sub picture or a CTU group unit) may be set.
  • a picture division unit eg, a sub picture or a CTU group unit
  • the initial QP information of the header information set as described above may be inserted into, for example, a PPS or a sequence parameter set (SPS), and the characteristic adaptive quantization parameter determination unit 1320 is included in the PPS or SPS as described above.
  • SPS sequence parameter set
  • the encoding parameter may include information on an encoding region (picture division region, picture division boundary, tile boundary, slice boundary, lower picture region and boundary) divided within one picture.
  • the image case-based quantization parameter derivation unit 1321 corresponds to the Initial QP List set in the initial quantization parameter derivation unit 1310 to adjust the initial QP between the divided regions, and divides it by adjusting the offset value for each index of the list Initial QP for each CTU quantization group unit in a region may be adjusted.
  • the image case-based quantization parameter derivation unit 1321 sets the maximum size of the Initial QP List, or according to a predefined Default value or Defined value between the encoding device 100 and the decoding device 200, to the initial QP list. Corresponding QP list update processing can be performed.
  • the slice type-based quantization parameter deriving unit 1323 may apply the quantization parameter differently according to the slice type to which the CTU quantization group unit belongs.
  • the slice type may be an I / B / P type slice, and prediction and transform results may be differently generated according to the slice type.
  • the slice type-based quantization parameter derivation unit 1323 adaptively determines different quantization parameters corresponding to slice type-based prediction and transformation results for each CTU quantization unit block, from a cognitive quality point of view. Can improve the coding efficiency.
  • the slice type-based quantization parameter derivation unit 1323 first determines the set value (quantization step size and QP range) of the quantization parameter and performs the quantization process accordingly, firstly slices the CTU quantization unit block. Judge.
  • the slice type-based quantization parameter derivation unit 1323 in the case of I Slice, the allowable prediction in the slice may be limited to intra-prediction, so a predefined quantization step size and QP range corresponding to the intra-prediction process Can decide.
  • uni-directional prediction may be allowed in intra-screen prediction and inter-screen prediction, and a predefined quantization step size and QP range are determined in response to the intra-screen prediction process and uni-directional prediction of inter-screen prediction. You can.
  • a prediction method may be limited, and this may affect an RDO process for determining a block partitioning structure. Therefore, the slice type based quantization parameter derivation unit 1323 according to an embodiment of the present invention, Coding and decoding efficiency can be improved by subdividing and determining an efficient quantization step size and QP range corresponding to each CTU quantization unit block.
  • the residual signal by the prediction is the input of the transform (Transform) process, it may affect the transform function mode and transform coefficients.
  • FIG. 25 shows a QP range table for determining the quantization step size and QP range for each slice type.
  • the slice type based quantization parameter derivation unit 1323 may selectively adaptively adjust the QP Range corresponding to the slice type based on the QP range table.
  • the selective adaptive QP adjustment information may be indicated in header information as a separate flag, or may be transmitted from the encoding device 100 to the decoding device 200 through separate signaling information.
  • the QP adjustment flag or signaling information may be 0 (False), and in this case, the quantization unit 130 processes a quantization process according to a fixed QP range, based on a fixed base QP It is possible to determine the applied QP for the picture and slice.
  • the slice type-based quantization parameter derivation unit 1323 determines a selective adaptive QP range based on a table and determines a corresponding base QP. By determining, the QP range corresponding to each slice and picture may be selectively adaptively determined for each CTU quantization block.
  • the slice type-based quantization parameter derivation unit 1323 may determine the minimum QP and the maximum QP corresponding to the slice or picture type in order to selectively adaptively calculate the QP value. Then, the slice type-based quantization parameter derivation unit 1323 calculates detailed QP values for each CTU quantization group unit based on the obtained CTU quantization group unit offset value and the minimum base QP value determined for each slice. You can.
  • the slice type-based quantization parameter derivation unit 1323 may perform a hierarchical QP range adjustment of a temporal layer of a picture to which the slice belongs.
  • a slice according to an embodiment of the present invention may be classified into hierarchical slices (B Slice0 to B Slice4) corresponding to a temporal layer of each picture.
  • the slice type-based quantization parameter derivation unit 1323 may adjust each detailed QP Range according to the classified layer information corresponding to each temporal layer, and may determine the BASE QP and QP values accordingly.
  • the slice type-based quantization parameter derivation unit 1323 may adaptively and efficiently determine a quantization parameter value according to characteristic information of an image corresponding to the slice type.
  • the slice type-based quantization parameter derivation unit 1323 may adaptively determine at least one of the range, step size, and QP value of the quantization parameter according to the slice type and image resolution. From the perspective of cognitive quality, as a region that can be recognized by a person varies according to the resolution (HD / FHD / 2K / 4K / 8K, etc.) of the image, the slice type-based quantization parameter derivation unit 1323 determines the specific image resolution. This is to allow the QP value to be adjusted by using as a reference point.
  • the slice type-based quantization parameter derivation unit 1323 may define a QP Resolution operation so that it can be applied according to a resolution, which may include an operation in which QP parameters corresponding to an arbitrary resolution are calculated.
  • the slice type-based quantization parameter derivation unit 1323 may selectively adaptively determine the QP of the slice to be encoded or decoded according to the resolution-based QP parameter calculated by the QP resolution operation.
  • the slice type based quantization parameter derivation unit 1323 may derive an individual QP value corresponding to an HDR image or a 360 VR image among image characteristics corresponding to the slice type. have.
  • the slice type-based quantization parameter derivation unit 1323 when the block to be determined QP is a block of an HDR image, the slice type-based quantization parameter derivation unit 1323 corresponds to a block of an image region in which the luminance range is significantly changed, and a relatively high QP Can be applied.
  • the slice type-based quantization parameter derivation unit 1323 obtains a separately designated QP offset value for the illumination adaptive deblocking filter for the QP determination target block from header information, and the QP determination target block is applied by HDR or the like.
  • the luminance range level is greater than a predetermined value
  • illuminance adaptive deblocking filtering based on the separately designated QP offset value may be performed. Accordingly, adaptive deblocking filtering for a block of an image region having a large change in the luminance range is selectively adaptively performed, thereby improving subjective image quality.
  • the slice type-based quantization parameter derivation unit 1323 identifies a region and a region of interest (ROI) of the image when the QP determination target block is a block of a VR 360 image for virtual reality, and the region of interest is identified in the identified region of interest. It is also possible to set a relatively high QP for the corresponding block.
  • ROI region of interest
  • the color sample-based quantization parameter deriving unit 1325 may derive a selective adaptive QP value according to the color sample.
  • the color sample-based quantization parameter derivation unit 1325 may calculate the QP value of a chroma color component sample using the QP value of the luminance color component sample.
  • the color sample-based quantization parameter derivation unit 1325 may conditionally determine the QP value of the saturation color sample based on the QP value of the illuminance color sample.
  • the determination condition i) when a separate block structure is formed between the illuminance sample and the color difference sample matching it, ii) a separate block tree is constructed between the illuminance sample and the color difference sample matching it, and has different block partitioning trees.
  • various conditions such as the case where the block division characteristics (division direction, division depth, block size, etc.) of the illuminance color sample have a predetermined value may be exemplified.
  • the color sample-based quantization parameter derivation unit 1325 is configured when i) a separate block structure between an illuminance sample and a color difference sample matching it, ii) a separate block tree between an illuminance sample and a color difference sample matching it. Therefore, in case of having different block partitioning trees, iii) for each color condition of the saturation color sample, such as the case where the block partitioning characteristics (division direction, division depth, block size, etc.) of the illuminance color sample have a predetermined value. Whether to derive QP values and quantization offset parameters for deriving QP values may be selectively adaptively determined.
  • the color sample-based quantization parameter derivation unit 1325 determines the QP value derivation function of the saturation color sample differently according to the block division characteristics (division direction, division depth, block size, block tree structure, etc.) of the illuminance color sample.
  • a selective adaptive QP decision can be processed.
  • the adaptive QP decision process according to an embodiment of the present invention can be illustrated as follows.
  • Chroma_QP Luma_QP-Diff_Offset
  • Diff_Offset min (MaxDiff_ChromaQP, LumaQP-Chroma_QP); To deal with.
  • the QP parameter applied to the Chroma Sample can be determined using an offset (Diff_Offset) value defined separately from Luma QP.
  • the Luma QP is a difference value corresponding to an initial QP calculated in advance from header information and block information, and can be calculated from delta QP obtained from header information or block information. Accordingly, when a separate block structure or block tree between the illuminance sample and a color difference sample matching it is configured in dual, Luma QP may be obtained from header information or block information of a block structure or block tree corresponding to illuminance.
  • the offset may be derived from QP Offset (MaxDiff_ChromaQP) transmitted from the encoding device 100.
  • the offset may be a smaller or larger value among the difference values between LumaQP and ChromaQP derived in advance.
  • the QP offset may be transmitted as header information signaled from the encoding apparatus 100 according to each condition or block information corresponding to a coding unit or a transformation unit.
  • the color sample-based quantization parameter derivation unit 1325 is composed of a separate block structure between the i) illuminance sample and the color difference sample matching it, ii) a separate block tree between the illuminance sample and the color difference sample matching it, In a DUAL Tree case or the like having a different block partitioning tree, a QP offset value as a Chroma QP parameter for each unit for individually deriving QPs corresponding to quantization units of a color difference sample tree from LumaQPs from signaling information or decoding target block information Accordingly, the color sample-based quantization parameter derivation unit 1325 may derive the QP of the color difference sample from the Luma QP using the determined Chroma QP parameter.
  • QP offset information for Chroma QP parameter calculation may be derived from high-level syntax of header information such as PPS or SPS, or may be obtained from block information such as a coding unit and a transformation unit.
  • the Chroma QP parameter calculation The QP offset information for can be configured in a list form corresponding to a quantization unit, and QP values for each color difference sample corresponding to each coding unit or transformation unit are pre-computed Luma QP values and QPs obtained from the list It can be calculated and determined based on the offset information.
  • the list may be derived from high level syntax of header information such as PPS or SPS as described above, or may be obtained from block information such as a coding unit and a transformation unit.
  • the transform coefficient-based quantization parameter derivation unit 1327 may process an adaptive quantization parameter determination process in consideration of transform coefficients and encoding characteristics in response to a CTU quantization unit block.
  • the transform coefficient may be leveled, and the quantization unit 130 may include a plurality of quantizers for each level (not shown). Such a plurality of quantizers may be mapped to each level, and accordingly, quantization processing for each transform level may be performed in each quantizer. Pre-defined step sizes and coefficient levels for each quantizer may be mapped, and accordingly, a scalar-based scaled quantization process may be processed for each transform coefficient level.
  • the transform coefficient-based quantization parameter deriving unit 1327 corresponds to at least one of the level of each transform coefficient, encoding characteristic information, and image characteristic information. Selective adaptive quantization parameter derivation can be handled.
  • the encoding characteristic information may include spatial layer information of a picture, slice type information, encoding layer information including an enhancement layer or a base layer, and the like.
  • HDR image information As the characteristic information of the input image, HDR image information, 360 VR image information, image information requesting high image quality of a partial region, or image resolution information may be exemplified.
  • two or three quantizers that provide level-based quantization processing of transform coefficients may be exemplified.
  • the first quantizer (BaseQ) that processes the base quantization parameter and a second quantizer (EnhancementQ) that processes the enhanced quantization parameter
  • the base may be composed of a first quantizer corresponding to (BASEQ), a second quantizer corresponding to uplink (HIGHQ) and a third quantizer corresponding to downlink (LOWQ).
  • the transform coefficient-based quantization parameter derivation unit 1327 may select information of each quantizer, quantization step size, and size according to characteristics of encoding information and image information.
  • the QP range can be determined.
  • the transform coefficient-based quantization parameter derivation unit 1327 may determine separate quantization step sizes and ranges using only an uplink quantizer (HighQ) and a base quantizer (baseQ), and correspond to the same. Can induce QP.
  • HighQ uplink quantizer
  • baseQ base quantizer
  • the transform coefficient-based quantization parameter derivation unit 1327 selects a base quantizer (BaseQ) or a down-quantizer (LowQ) corresponding to the resolution information in the case of a general image, and processes the QP value corresponding thereto to be derived. You can.
  • BaseQ base quantizer
  • LowQ down-quantizer
  • quantization processing using each quantizer includes a group-by-group difference quantization parameter derivation unit 1330 and a coding block unit adaptive quantization parameter application unit ( 1340).
  • the quantization unit 130 may perform level classification processing based on a transform coefficient value for a transform block.
  • the transform coefficient-based quantization parameter derivation unit 1327 may determine at least one of a plurality of quantizers to process quantization based on the processed level classification, image characteristic information, and encoding characteristic information.
  • the encoding characteristic information may include at least one of block structure information, block division information, division depth information, division direction information, or color difference component information.
  • at least one of the plurality of quantizers may be at least one of the above-described two or three quantizers.
  • the transform coefficient-based quantization parameter derivation unit 1327 may process QP values to be selectively adaptively determined by applying different quantization step sizes according to configuration and selection information of each quantizer.
  • 26 is a view showing in more detail the operation of the quantization unit 130 thus processed.
  • the quantization unit 130 derives an initial quantization parameter and an initial quantization parameter list through the initial quantization parameter derivation unit 1310 (S2001).
  • the quantization unit 130 determines a characteristic adaptive quantization parameter through the characteristic adaptive quantization parameter determination unit 1320 (S2003).
  • the characteristic adaptive quantization parameter determining unit 1320 includes the above-described image case-based quantization parameter derivation unit 1321, slice type-based quantization parameter derivation unit 1323, color sample-based quantization parameter derivation unit 1325, and transform coefficient-based quantization.
  • a quantization parameter corresponding to the initial quantization parameter list may be selectively adaptively determined for each CTU quantization unit block by using at least one of the parameter derivation units 1327.
  • the quantization unit 130 through the group unit difference quantization parameter derivation unit 1330, performs quantization processing, and then corresponds to the quantization parameters configured and updated by the list, and sets the difference quantization parameters for each CTU quantization unit block.
  • Derivation (S2005), and adaptive quantization parameters in units of coding blocks are applied accordingly (S2007), so that selective adaptive quantization for each block is processed (S2009).
  • the decoding device 200 can also process in the reverse order. That is, the inverse quantization unit 220 may selectively adaptively determine an inverse quantization parameter for inverse quantization processing of the target block, based on the quantization group unit information of the target block, and subsequent processing is described above. The same or similar treatment as the bar.
  • FIGS. 27 to 29 are diagrams illustrating high-level syntax information signaled for the operation of the quantization unit 130 according to an embodiment of the present invention.
  • FIG. 27 illustrates PPS syntax.
  • DIFF_CU_QP_DELTA_DEPTH information and QP offset information for each color difference are signaled as differential information for adaptively determining QP for each CTU quantization unit block through PPS. Can be illustrated.
  • FIG. 28 illustrates a range extension syntax of PPS.
  • Adaptive_Group_QP_list_flag indicating whether adaptive quantization group unit list processing is applied through the range extension syntax of PPS may be signaled, and each corresponding thereto It may be exemplified that offset and index information of the list and offset depth information (DIFF_CU_CHROMA_QP_OFFSET_DEPTH) corresponding to the differential color difference QP are signaled.
  • FIG. 29 illustrates slice header syntax according to an embodiment of the present invention.
  • color difference QP offset information and GROUP_QP_LIST [i] for processing each adaptive quantization group unit list are defined, and the slice QP value It may be indicated that this can be applied adaptively.
  • syntax information is exemplified for an embodiment of the present invention, and it is obvious that the expressions or terms may be modified.
  • FIGS. 30 to 33 are diagrams for explaining a case in which a QP parameter is adaptively determined when a picture according to an embodiment of the present invention is divided into a plurality of input image segmentation units.
  • the image division unit may not only be determined corresponding to each coding unit, but also a sub-picture (Sub) that divides one picture separately from the detailed coding unit (Sub picture) unit.
  • the sub-picture may correspond to a region corresponding to a specific viewpoint in a VR image, etc., and one picture may be composed of a plurality of sub-pictures.
  • separate signaling information corresponding to the sub-pictures may be transmitted from the encoding device 100 to the decoding device 200.
  • 30 is an exemplary syntax diagram of subpicture signaling information inserted into header information as such signaling information.
  • sub-picture signaling information includes the total number of sub-pictures (Total_subpics_nums), sub-picture width information (SubPic_Width), and sub-picture height information (SubPic_Height), It may include at least one of sub-picture horizontal split information (Subpic_Hor_split_flag) and sub-picture identification information (Subpic_ID), and further, whether to use information as a picture flag of the sub-picture and whether to allow loop filtering between sub-pictures may be further included. .
  • the entire picture may be divided into a plurality of sub-pictures, and sub-picture indices (SubPic [0] [0] to SubPic [2] [2) corresponding to each sub-picture. ]).
  • subpicture IDs (0 to N) may be assigned to each subpicture.
  • the signaling information may include picture width and height information corresponding to each sub-picture identifier and sub-picture index.
  • sub picture identifier may be included in header information such as SPS or PPS or an SEI message.
  • the sub picture ID may include initial sub picture ID information of the first coded picture in the sequence.
  • the decoding apparatus 200 may identify a specific picture and identify and update IDs of subpictures in the specific picture based on the subpicture ID information received through the SEI message. Accordingly, the subpicture ID in the sequence can be stored and updated.
  • the SEI message may be replaced with encoded header information such as PPS or slice header. Accordingly, the decoding apparatus 200 may update the identifier information of the subpicture in the picture using the PPS or slice header.
  • sub-picture identifier may be variably changed.
  • separate sub-picture Grid information, Index information, or Offset information for indicating the variable may be signaled as sub-picture identifier variable information.
  • the decoding apparatus 200 may determine the location and size of the sub-pictures in the picture based on the signaled sub-picture identifier variable information. IDs of sub-pictures corresponding to sub-pictures having the same size may be flexibly allocated and updated according to the progress of a picture sequence.
  • a sub-picture split flag indicating this may be designated and included in signaling information.
  • width information and height information of the subpicture are respectively designated and may be included in signaling information.
  • the sub-picture signaling information may be transmitted by being included in header information of a video bitstream such as Sequence Parameter Set (SPS) and Picture Parameter Set (PPS).
  • SPS Sequence Parameter Set
  • PPS Picture Parameter Set
  • Total_subpics_nums may be sub-picture signaling information transmitted on the SPS, and may indicate the total number of sub-pictures (or sub-pictures) divided within each picture.
  • Subpic_ID may indicate a unique ID assigned to each subpicture.
  • unique identification information may be allocated and signaled to the sub-picture, and basically, a sub-picture ID may be allocated in Z-Scan order from the top-left to the right bottom of the picture.
  • sub-picture identification information may be transmitted in the form of a list or a table.
  • the decoding apparatus 200 may set an index based on the total number of sub-pictures, and sequentially allocate IDs of sub-pictures to the index according to a predefined order.
  • the sub-picture ID may indicate attributes of a sub-picture, and may be used as identification information (ID) for selecting a parameter set to be applied in response to a sub-picture, unlike index information corresponding to each position.
  • ID identification information
  • the sub-picture ID allocated to the sub-picture may be mapped or corresponded to the slice ID of the slice header.
  • the index or grid information corresponding to the sub picture ID may indicate the structure and size information of each sub picture, or may be used as fixed location information for slice decoding.
  • the decoding apparatus 200 may selectively adaptively derive and determine various encoding information (reference picture list information, selective adaptive QP allocation, NAL UNIT TYPE, and SLICE TYPE) using ID information of the subpicture.
  • various encoding information reference picture list information, selective adaptive QP allocation, NAL UNIT TYPE, and SLICE TYPE
  • the decoding apparatus 200 determines the QP offset corresponding to the sub-picture ID, changes the Scanning Order, or refers to a separate optional adaptive parameter set to initialize parameters for each sub-picture.
  • Subpicture id information may be used to determine the QP value.
  • a separate parameter set corresponding to the sub picture ID may be referenced.
  • the parameter set may be a picture parameter set or a separately selected adaptive parameter set, and the sub picture id may be configured to be included in the sequence parameter set and mapped to each other parameter set.
  • the decoding apparatus 200 may decode the size and structure information of each subpicture using Total_subpics_nums and Subpic_Width and Subpic_Height from the subpicture signaling information.
  • the decoding apparatus 200 may determine the structure and size information of the subpicture by using the width and height information of the pre-decoded picture and the subpic_width and subpic_height obtained previously. .
  • the decoding apparatus 200 may determine size information of a subpicture located in a right boundary of a picture through a separate operation.
  • the decoding apparatus 200 divides the entire picture width (Picture width) by the sub-picture width (Subpic width) or performs a difference operation, but when the remaining value within a certain size occurs, the sub located at the right boundary of the picture
  • the width of the picture (Subpicture_Width) may be derived from the remaining values.
  • Subpic_Hor_split_flag may be signaled when the height or width of the current sub-picture is smaller than the sub-picture of the upper left (Left-Top), and may include segmentation information including a segmentation ratio, segmentation direction, or segmentation size.
  • sub-pictures obtained by subpic_Hor_split_flag are sub-pictures whose height is horizontally divided according to a height corresponding to a sub-picture of a left-top.
  • the divided subpicture list may be included in the signaling information and transmitted, for example, the entire substructure information of the subpicture may be derived according to index and offset information for each subpicture.
  • the decoding apparatus 200 uses the remaining information (total number of sub pictures, split flag information, etc.) to more accurate sub picture for each index. You can derive the size and position of the.
  • FIG. 33 is a diagram for explaining a process of changing a subpicture id that is variable according to an embodiment of the present invention.
  • a unique ID may be assigned to each sub-picture, and for example, a sub-picture ID within a specific sequence unit may be variable.
  • the Subpic_ID corresponding to the subpicture may be variably determined based on separate signaling information.
  • the sub picture identification information may be mapped to a parameter set to be referenced in order to determine a QP value for the sub picture.
  • a parameter set to be referred to for determining the QP value for each sub picture may also be changed. For example, when the main view to be viewed by the user changes in the VR image, for example, the decoding apparatus 200 changes the current sub-picture identification information to sub-picture identification information for the main view, thereby referring to a parameter for reference to determine the QP value You can change the set.
  • the method according to the present invention described above is produced as a program to be executed on a computer and can be stored in a computer-readable recording medium.
  • the computer-readable recording medium include ROM, RAM, CD-ROM, and magnetic tape. , Floppy disks, optical data storage devices, and the like, and also implemented in the form of a carrier wave (for example, transmission via the Internet).
  • the computer-readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
  • functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention pertains.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Un mode de réalisation de la présente invention concerne un procédé de traitement d'image comprenant les étapes consistant à : diviser une photographie d'une image en une pluralité d'unités de codage dont chacune est une unité de base pour laquelle une prédiction inter ou une prédiction intra est effectuée, puis diviser la photographie ou chacune des unités de codage divisées en un arbre quaternaire, en un arbre binaire ou en une structure d'arbre ternaire afin de déterminer un bloc cible pour le décodage de l'unité de codage; et déterminer de manière sélective et adaptative un paramètre de quantification destiné au traitement d'une quantification du bloc cible sur la base d'informations d'unité de groupe de quantification basées sur une unité d'arbre de codage comprenant le bloc cible.
PCT/KR2019/011651 2018-09-07 2019-09-09 Procédé de décodage et de codage d'image destiné au traitement d'un paramètre de quantification d'unité de groupe WO2020050705A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR20180107200 2018-09-07
KR10-2018-0107200 2018-09-07

Publications (1)

Publication Number Publication Date
WO2020050705A1 true WO2020050705A1 (fr) 2020-03-12

Family

ID=69723172

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/011651 WO2020050705A1 (fr) 2018-09-07 2019-09-09 Procédé de décodage et de codage d'image destiné au traitement d'un paramètre de quantification d'unité de groupe

Country Status (1)

Country Link
WO (1) WO2020050705A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210337226A1 (en) * 2019-01-09 2021-10-28 Huawei Technologies Co., Ltd. Sub-picture Level Indicator Signaling In Video Coding
CN113766227A (zh) * 2020-06-06 2021-12-07 华为技术有限公司 用于图像编码和解码的量化和反量化方法及装置
US20220094909A1 (en) * 2019-01-02 2022-03-24 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
CN114697656A (zh) * 2020-12-31 2022-07-01 浙江宇视科技有限公司 一种编码方法、装置、电子设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120106628A1 (en) * 2009-07-02 2012-05-03 Joel Sole Methods and apparatus for video encoding and decoding binary sets using adaptive tree selection
KR20140120253A (ko) * 2011-12-15 2014-10-13 타지반 투 엘엘씨 화상 부호화 방법, 화상 복호 방법, 화상 부호화 장치, 화상 복호 장치, 화상 부호화 복호 장치
WO2016090568A1 (fr) * 2014-12-10 2016-06-16 Mediatek Singapore Pte. Ltd. Structure de partitionnement de bloc d'arbre binaire
KR20180033030A (ko) * 2016-09-23 2018-04-02 세종대학교산학협력단 적응적 블록 분할에 기반한 비디오 신호 처리 방법 및 장치
KR20180085526A (ko) * 2017-01-19 2018-07-27 가온미디어 주식회사 효율적 변환을 처리하는 영상 복호화 및 부호화 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120106628A1 (en) * 2009-07-02 2012-05-03 Joel Sole Methods and apparatus for video encoding and decoding binary sets using adaptive tree selection
KR20140120253A (ko) * 2011-12-15 2014-10-13 타지반 투 엘엘씨 화상 부호화 방법, 화상 복호 방법, 화상 부호화 장치, 화상 복호 장치, 화상 부호화 복호 장치
WO2016090568A1 (fr) * 2014-12-10 2016-06-16 Mediatek Singapore Pte. Ltd. Structure de partitionnement de bloc d'arbre binaire
KR20180033030A (ko) * 2016-09-23 2018-04-02 세종대학교산학협력단 적응적 블록 분할에 기반한 비디오 신호 처리 방법 및 장치
KR20180085526A (ko) * 2017-01-19 2018-07-27 가온미디어 주식회사 효율적 변환을 처리하는 영상 복호화 및 부호화 방법

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220094909A1 (en) * 2019-01-02 2022-03-24 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US11778171B2 (en) * 2019-01-02 2023-10-03 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US20210337226A1 (en) * 2019-01-09 2021-10-28 Huawei Technologies Co., Ltd. Sub-picture Level Indicator Signaling In Video Coding
US11949893B2 (en) * 2019-01-09 2024-04-02 Huawei Technologies Co., Ltd. Sub-picture level indicator signaling in video coding
CN113766227A (zh) * 2020-06-06 2021-12-07 华为技术有限公司 用于图像编码和解码的量化和反量化方法及装置
CN114697656A (zh) * 2020-12-31 2022-07-01 浙江宇视科技有限公司 一种编码方法、装置、电子设备及介质
CN114697656B (zh) * 2020-12-31 2024-02-13 浙江宇视科技有限公司 一种编码方法、装置、电子设备及介质

Similar Documents

Publication Publication Date Title
WO2019112394A1 (fr) Procédé et appareil de codage et décodage utilisant un partage d'informations sélectif entre des canaux
WO2018030773A1 (fr) Procédé et appareil destinés au codage/décodage d'image
WO2018012886A1 (fr) Procédé de codage/décodage d'images et support d'enregistrement correspondant
WO2018070809A1 (fr) Procédé de traitement d'image, et procédé de codage et de décodage d'image associé
WO2019172705A1 (fr) Procédé et appareil de codage/décodage d'image utilisant un filtrage d'échantillon
WO2019022568A1 (fr) Procédé de traitement d'image, et procédé et dispositif de codage/décodage d'image en utilisant celui-ci
WO2017204427A1 (fr) Procédé de traitement d'image, et procédé d'encodage et de décodage d'image utilisant celui-ci
WO2018026148A1 (fr) Procédé et dispositif de codage/décodage d'images, et support d'enregistrement stockant un flux binaire
WO2018135885A1 (fr) Procédé de décodage et de codage d'image pour fournir un traitement de transformation
WO2019083334A1 (fr) Procédé et dispositif de codage/décodage d'image sur la base d'un sous-bloc asymétrique
WO2020060184A1 (fr) Procédé et appareil de codage/décodage d'images, et support d'enregistrement stockant un flux binaire
WO2020005035A1 (fr) Appareil et procédé de décodage/codage d'image destiné à l'amélioration du débit de traitement, et support d'enregistrement stockant un train de bits
WO2019182292A1 (fr) Procédé et appareil de traitement du signal vidéo
WO2012044104A2 (fr) Procédé de codage vidéo pour coder des symboles de structure hiérarchique et dispositif correspondant, et procédé de décodage vidéo pour décoder des symboles de structure hiérarchique et dispositif correspondant
WO2021015581A1 (fr) Procédé, appareil et support d'enregistrement pour coder/décoder une image à l'aide d'un partitionnement géométrique
WO2020050705A1 (fr) Procédé de décodage et de codage d'image destiné au traitement d'un paramètre de quantification d'unité de groupe
WO2019194653A1 (fr) Procédé de traitement d'image de fourniture de processus de mode de fusion complexe d'informations de mouvement, procédé de codage et de décodage d'image l'utilisant, et appareil associé
WO2020050600A1 (fr) Procédé et dispositif de codage/décodage vidéo, et support d'enregistrement pour stockage de flux binaire
WO2020139006A2 (fr) Procédé et appareil pour déduction de mode d'intra-prédiction
WO2020004978A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2019194647A1 (fr) Procédé de filtrage adaptatif de boucle basé sur des informations de filtre et procédé de codage et de décodage d'image l'utilisant
WO2020141904A1 (fr) Procédé de codage/décodage de signal d'image et dispositif associé
WO2020130714A1 (fr) Procédé de codage/décodage de signal vidéo et dispositif associé
WO2020171658A1 (fr) Procédé et dispositif de codage/décodage vidéo, et support d'enregistrement permettant de stocker un flux binaire
WO2020256522A1 (fr) Procédé et appareil de codage d'image et de décodage d'image à l'aide d'une segmentation de zone

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19856711

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19856711

Country of ref document: EP

Kind code of ref document: A1