WO2012043678A1 - Dispositif de décodage d'image, dispositif de codage d'image et structure de données - Google Patents

Dispositif de décodage d'image, dispositif de codage d'image et structure de données Download PDF

Info

Publication number
WO2012043678A1
WO2012043678A1 PCT/JP2011/072283 JP2011072283W WO2012043678A1 WO 2012043678 A1 WO2012043678 A1 WO 2012043678A1 JP 2011072283 W JP2011072283 W JP 2011072283W WO 2012043678 A1 WO2012043678 A1 WO 2012043678A1
Authority
WO
WIPO (PCT)
Prior art keywords
prediction
unit
prediction mode
target
partition
Prior art date
Application number
PCT/JP2011/072283
Other languages
English (en)
Japanese (ja)
Inventor
山本　智幸
将伸 八杉
知宏 猪飼
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2012043678A1 publication Critical patent/WO2012043678A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding

Definitions

  • the present invention relates to an image decoding device that decodes encoded data, and an image encoding device that generates encoded data.
  • the present invention also relates to a data structure of encoded data generated by the image encoding device and referenced by the image decoding device.
  • a moving image encoding device that generates encoded data by encoding the moving image, and decoding the encoded data
  • a video decoding device image decoding device that generates a decoded image
  • As a specific moving picture encoding method for example, H.264 is used. H.264 / MPEG-4.
  • Adopted in the KTA software which is a codec for joint development in AVC (Non-Patent Document 1) and VCEG (Video Coding Expert Group), and TMuC (Test Model Under Consideration) software, the successor codec The method that has been used.
  • an image (picture) constituting a moving image includes a slice obtained by dividing the image, a coding unit obtained by dividing the slice (macroblock or maximum coding unit (Largest Coding) And a hierarchical structure composed of blocks and partitions obtained by dividing the coding unit.
  • a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is subtracted from the input image (original image).
  • the prediction residual (which may be referred to as “difference image” or “residual image”) is encoded.
  • examples of the method for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).
  • a predicted image in a frame being decoded is generated for each prediction unit by applying motion compensation using a motion vector with the decoded frame as a reference frame.
  • intra prediction a predicted image in a frame being decoded is generated for each prediction unit based on a decoded area of the frame being decoded.
  • a predetermined prediction mode group also referred to as “basic prediction mode group”
  • a method for generating a pixel value on the prediction unit by selecting a prediction mode and (2) extrapolating a pixel value of a decoded area in an extrapolation direction (prediction direction) corresponding to the selected prediction mode (“ Sometimes referred to as “basic prediction”).
  • Non-Patent Document 2 for each prediction unit, the edge direction in the prediction unit is estimated based on the pixel values of pixels located around the prediction unit, and the pixel value of the decoded area is extrapolated to the estimated edge direction.
  • a method for generating a pixel value on the prediction unit a method called Differential Coding of Mode Intra Modes (DCIM), sometimes called “edge prediction” or “edge based prediction” is disclosed.
  • DCIM Differential Coding of Mode Intra Modes
  • FIG. 24A is a diagram schematically illustrating a prediction unit (CurrentBlock) to be processed and pixels located around the prediction unit.
  • an edge vector a i 1 to N, N is the total number of surrounding pixels to be referred to).
  • a Sobel operator also referred to as a Sobel filter
  • an argument ⁇ * argmaxS ( ⁇ ) that maximizes the function S ( ⁇ ) is calculated, and the direction represented by ⁇ * is set as the prediction direction.
  • the pixel value on the prediction unit is generated by extrapolating the pixel value of the decoded area in the prediction direction.
  • the argument ⁇ * that maximizes the function S ( ⁇ ) represents an estimated value of the edge direction in the prediction unit to be processed, and is also referred to as “neighbors' suggested prediction direction”. Note that the calculation of the argument ⁇ * is performed in both the encoding device and the decoding device, and thus the argument ⁇ * itself is not encoded.
  • the direction represented by ⁇ * + ⁇ can be used as the prediction direction.
  • ⁇ indicates direction adjustment, and ⁇ used in the encoding device needs to be encoded and transmitted to the decoding device.
  • the decoding apparatus sets ⁇ * + k ⁇ ⁇ as the prediction direction.
  • FIG. 24C shows a parameter set including the syntax adjust_neighb_dir for specifying the quantization index k. In the example shown in FIG.
  • Non-Patent Document 2 H. H.264 / MPEG-4.
  • the number of selectable prediction modes increases, so that the prediction accuracy can be improved.
  • Non-Patent Document 2 requires a syntax for designating one of the one or more prediction modes added to the basic prediction mode group, the encoded data There is also an aspect that the amount of codes increases. For this reason, when the additional prediction mode is frequently used, there has been a problem that the encoding efficiency may not be improved, or the encoding efficiency may not be improved as expected.
  • the present invention has been made in view of the above-described problems, and the object of the present invention is to provide a code amount of encoded data while maintaining high prediction accuracy even when a prediction mode is added to the basic prediction mode group. It is to realize an image encoding device and an image decoding device capable of improving the encoding efficiency by suppressing the increase in the image quality.
  • an image decoding apparatus is a prediction mode belonging to a prediction mode group in a prediction residual decoded from encoded data together with prediction mode designation information and other side information,
  • the image decoding apparatus that generates a decoded image by adding the prediction images generated according to the prediction mode specified by the prediction mode specification information, the number N of prediction modes to be added to the prediction mode group related to the target prediction unit is used as the side information.
  • a setting unit that is an encoding parameter that is included and is set with reference to an encoding unit that includes the target prediction unit or an encoding parameter that is decoded prior to the encoding unit; Prediction parameters corresponding to each of the N prediction modes to be added to the prediction mode group related to the target prediction unit are decoded. It is characterized in that it comprises a derivation means for deriving from the decoded region of the image, a.
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually encoded that is decoded before the prediction unit or the encoding unit. It can be determined from the coding parameters for the unit.
  • the coding parameter is a parameter that is referred to in order to generate a decoded image, a locally decoded image, or a predicted image, and is referred to in a motion vector or intra prediction that is referred to in inter-screen prediction.
  • the prediction parameters such as the prediction mode, the size and shape of the partition, the size and shape of the block, the size and shape of the encoding unit, and residual data between the original image and the predicted image are included.
  • the side information refers to a set of all information except for the residual data among the information included in the encoding parameter.
  • the encoding parameter related to the encoding unit including the target prediction unit is related to the encoding parameter associated with the encoding unit and one or a plurality of prediction units included in the encoding unit. Encoding parameters are included.
  • the prediction unit may be a PU (Prediction Unit) described in the embodiment, or may be a partition obtained by dividing the PU.
  • the coding unit may be a maximum coding unit (LCU: Largegest Coding Unit) described in the embodiment, or a coding unit (CU: Coding Unit) obtained by dividing an LCU. Also good.
  • the image coding apparatus designates the selected prediction mode based on the prediction residual obtained by subtracting the predicted image generated according to the prediction mode selected from the prediction mode group for each prediction unit from the original image.
  • the number N of prediction modes to be added to the prediction mode group related to the target prediction unit is encoded in the side information.
  • a parameter setting unit that refers to a coding unit that includes the target prediction unit or a coding parameter that is encoded prior to the coding unit; and the target prediction Deriving prediction parameters corresponding to each of the N prediction modes to be added to the prediction mode group related to the unit from the decoded region of the locally decoded image It is characterized in that comprises a derivation means that, the.
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually a code encoded before the prediction unit or the encoding unit. It can be determined from the encoding parameters for the encoding unit.
  • the encoding efficiency is increased by referring to the encoding parameter including the target prediction unit or the encoding parameter related to the encoding unit encoded before the encoding unit.
  • the number of prediction modes to be added to the prediction mode group related to the target prediction unit can be set. Therefore, according to said structure, encoding efficiency can be improved by suppressing the increase in the code amount of coding data, maintaining high prediction accuracy.
  • the coding parameter is a parameter that is referred to in order to generate a decoded image, a locally decoded image, or a predicted image, and is referred to in a motion vector or intra prediction that is referred to in inter-screen prediction.
  • the prediction parameters such as the prediction mode, the size and shape of the partition, the size and shape of the block, the size and shape of the encoding unit, and residual data between the original image and the predicted image are included.
  • the side information refers to a set of all information except for the residual data among the information included in the encoding parameter.
  • the encoding parameter related to the encoding unit including the target prediction unit is related to the encoding parameter associated with the encoding unit and one or a plurality of prediction units included in the encoding unit. Encoding parameters are included.
  • the prediction unit may be a PU (Prediction Unit) described in the embodiment, or may be a partition obtained by dividing the PU.
  • the coding unit may be a maximum coding unit (LCU: Largegest Coding Unit) described in the embodiment, or a coding unit (CU: Coding Unit) obtained by dividing an LCU. Also good.
  • the data structure of the encoded data according to the present invention includes a prediction residual obtained by subtracting, from the original image, a prediction image generated according to a prediction mode selected from a prediction mode group for each prediction unit.
  • the side information includes the number of prediction modes to be added to the prediction mode group related to the target prediction unit, which is a data structure of encoded data generated by encoding together with the prediction mode specifying information to be specified and other side information. Coding parameters that are expressed implicitly by a coding unit that includes the target prediction unit or a coding parameter that is coded before the coding unit. It is characterized by.
  • the image decoding apparatus includes the prediction mode belonging to the prediction mode group in the prediction residual decoded from the encoded data together with the prediction mode specifying information and other side information, and the prediction mode specifying information.
  • the image decoding device that generates a decoded image by adding the prediction images generated according to the prediction mode specified by the encoding
  • the number N of prediction modes to be added to the prediction mode group related to the target prediction unit is encoded in the side information
  • a parameter setting unit that refers to a coding unit that includes the target prediction unit or a coding parameter that is decoded prior to the coding unit; and the target prediction unit
  • Has a a derivation means for deriving from.
  • the above image decoding apparatus it is possible to improve the encoding efficiency by suppressing an increase in the code amount of the encoded data while maintaining high prediction accuracy.
  • FIG. 4 is a diagram illustrating a configuration of a picture layer of encoded data
  • (b) is a diagram illustrating a configuration of a slice layer included in the picture layer
  • (c) is a configuration of an LCU layer included in the slice layer.
  • (D) is a figure which shows the structure of the leaf CU contained in an LCU layer
  • (e) is a figure which shows the structure of the inter prediction information about leaf CU
  • (f) is It is a figure which shows the structure of the intra prediction information about leaf CU.
  • FIG. 1 It is a flowchart which shows the flow of the prediction image generation process by the moving image decoding apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the prediction image production
  • FIG. 1 It is a block diagram which shows the structure of the leaf CU encoding part with which the LCU encoding part which concerns on the 1st Embodiment of this invention is provided. It is a flowchart which shows the flow of the encoding process by the moving image encoder which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the prediction image generation process by the moving image decoding apparatus which concerns on the 1st Embodiment of this invention, and a moving image encoding apparatus, Comprising: A mode that an edge exists ranging over a target partition and an adjacent partition.
  • the conventional moving image decoding apparatus it is a figure for demonstrating the production
  • FIG. 2B is a block diagram showing a configuration of a receiving device equipped with a video decoding device.
  • BRIEF DESCRIPTION OF THE DRAWINGS It is a figure for demonstrating that the moving image decoding apparatus and moving image encoding apparatus which concern on embodiment of this invention can be utilized for recording and reproduction
  • Embodiments of an image decoding apparatus and an image encoding apparatus according to the present invention will be described below with reference to the drawings.
  • the image decoding apparatus according to the present embodiment decodes a moving image from encoded data. Therefore, hereinafter, this is referred to as “moving image decoding apparatus”.
  • the image encoding device according to the present embodiment generates encoded data by encoding a moving image. Therefore, in the following, this is referred to as a “video encoding device”.
  • the scope of application of the present invention is not limited to this. That is, as will be apparent from the following description, the feature of the present invention lies in intra prediction, and is established without assuming a plurality of frames. That is, the present invention can be applied to general image decoding apparatuses and general image encoding apparatuses regardless of whether the target is a moving image or a still image.
  • the encoded data # 1 has a hierarchical structure including a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, and a maximum coding unit (LCU: Large Coding Unit) layer.
  • GOP Group Of Pictures
  • LCU Large Coding Unit
  • FIG. 2 shows the hierarchical structure below the picture layer in the encoded data # 1.
  • (A) to (f) in FIG. 2 are a picture layer P, a slice layer S, an LCU layer LCU, a leaf CU included in the LCU (denoted as CUL in FIG. 2 (d)), and inter prediction (inter-screen)
  • PI_Inter which is the prediction information PI about (prediction) partition
  • PI_Intra intra prediction information
  • the picture layer P is a set of data that is referenced by the video decoding device 1 in order to decode a target picture that is a processing target picture. As shown in FIG. 2A, the picture layer P includes a picture header PH and slice layers S1 to SNs (Ns is the total number of slice layers included in the picture layer P).
  • the picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture.
  • the encoding mode information (entoropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.
  • entorpy_coding_mode_flag When entorpy_coding_mode_flag is 0, the picture is encoded by CAVLC (Context-based Adaptive Variable Length Coding). It has become.
  • Each slice layer S included in the picture layer P is a set of data referred to by the video decoding device 1 in order to decode a target slice that is a slice to be processed.
  • the slice layer S includes a slice header SH and LCU layers LCU1 to LCUn (Nc is the total number of LCUs included in the slice S).
  • the slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice.
  • Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.
  • the slice header SH includes a filter parameter FP that is referred to by a loop filter included in the video decoding device 1.
  • slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, and (2) P using unidirectional prediction or intra prediction at the time of encoding. Slice, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding.
  • Each LCU layer LCU included in the slice layer S is a set of data that the video decoding device 1 refers to in order to decode the target LCU that is the processing target LCU.
  • the LCU layer LCU includes an LCU header LCUH and a plurality of coding units (CU: Coding Units) obtained by dividing the LCU by quadtree division.
  • the size that each CU can take depends on the LCU size and hierarchical depth included in the sequence parameter set SPS of the encoded data # 1. For example, when the size of the LCU is 128 ⁇ 128 pixels and the maximum hierarchical depth is 5, the CU included in the LCU has five types of sizes, that is, 128 ⁇ 128 pixels, 64 ⁇ 64 pixels, Any of 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, and 8 ⁇ 8 pixels can be taken. A CU that is not further divided is called a leaf CU.
  • the LCU header LCUH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target LCU. Specifically, as shown in FIG. 2 (c), CU partition information SP_CU that specifies a partition pattern of each target LCU into each leaf CU, and a quantization parameter difference ⁇ qp that specifies the size of the quantization step. (Mb_qp_delta) is included.
  • CU division information SP_CU is information that specifies the shape and size of each CU (and leaf CU) included in the target LCU, and the position in the target LCU.
  • the CU partition information SP_CU does not necessarily need to explicitly include the shape and size of the leaf CU.
  • the CU partition information SP_CU may be a set of flags (split_coding_unit_flag) indicating whether or not the entire LCU or a partial region of the LCU is divided into four. In that case, the shape and size of each leaf CU can be specified by using the shape and size of the LCU together.
  • the quantization parameter difference ⁇ qp is a difference qp ⁇ qp ′ between the quantization parameter qp in the target LCU and the quantization parameter qp ′ in the LCU encoded immediately before the LCU.
  • CU leaf A CU (CU leaf) that cannot be further divided is handled as a prediction unit (PU: Prediction Unit) and a transform unit (TU: Transform Unit).
  • PU Prediction Unit
  • TU Transform Unit
  • the leaf CU (denoted as CUL in (d) of FIG. 2) is (1) PU information PUI that is referred to when the moving image decoding apparatus 1 generates a predicted image. And (2) the TU information TUI that is referred to when the moving image decoding apparatus 1 decodes the residual data.
  • the PU information PUI may include a skip flag SKIP. When the value of the skip flag SKIP is 1, the TU information is omitted.
  • the PU information PUI includes prediction type information PT and prediction information PI, as shown in FIG.
  • the prediction type information PT is information that specifies whether intra prediction or inter prediction is used as a predicted image generation method for the target leaf CU (target PU).
  • the prediction information PI includes intra prediction information PI_Intra or inter prediction information PI_Inter depending on which prediction method is specified by the prediction type information PT.
  • a PU to which intra prediction is applied is also referred to as an intra PU
  • a PU to which inter prediction is applied is also referred to as an inter PU.
  • the PU information PUI includes information specifying the shape and size of each partition included in the target PU and the position in the target PU.
  • the partition is one or a plurality of non-overlapping areas constituting the target leaf CU, and the generation of the predicted image is performed in units of partitions.
  • the TU information TUI includes TU partition information SP_TU that specifies a partition pattern for each block of the target leaf CU (target TU), and quantized prediction residuals QD1 to QDNT (NT Includes the total number of blocks included in the target TU).
  • TU partition information SP_TU is information that specifies the shape and size of each block included in the target TU and the position in the target TU.
  • Each TU can be, for example, a size from 64 ⁇ 64 pixels to 2 ⁇ 2 pixels.
  • the block is one or a plurality of non-overlapping areas constituting the target leaf CU, and prediction residual encoding / decoding is performed in units of TUs or blocks obtained by dividing TUs.
  • Each quantized prediction residual QD is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on a target block that is a processing target block.
  • Process 1 DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image.
  • Process 2 The DCT coefficient obtained in Process 1 is quantized.
  • Process 3 The DCT coefficient quantized in Process 2 is variable length encoded.
  • the inter prediction information PI_Inter includes a coding parameter that is referred to when the video decoding device 1 generates an inter prediction image by inter prediction. As shown in FIG. 2 (e), the inter prediction information PI_Inter includes inter PU partition information SP_Inter that specifies a partition pattern of the target PU into each partition, and inter prediction parameters PP_Inter1 to PP_InterNe (Ne for each partition). , The total number of inter prediction partitions included in the target PU).
  • the inter-PU partition information SP_Inter is information for designating the shape and size of each inter prediction partition included in the target PU (inter PU) and the position in the target PU.
  • the inter PU is composed of four symmetric splittings of 2N ⁇ 2N pixels, 2N ⁇ N pixels, N ⁇ 2N pixels, and N ⁇ N pixels, and 2N ⁇ nU pixels, 2N ⁇ nD pixels, and nL ⁇ 2N. It is possible to divide into 8 types of partitions in total by four asymmetric splits of pixels and nR ⁇ 2N pixels.
  • the specific value of N is defined by the size of the CU to which the PU belongs, and the specific values of nU, nD, nL, and nR are determined according to the value of N.
  • an inter PU of 128 ⁇ 128 pixels is 128 ⁇ 128 pixels, 128 ⁇ 64 pixels, 64 ⁇ 128 pixels, 64 ⁇ 64 pixels, 128 ⁇ 32 pixels, 128 ⁇ 96 pixels, 32 ⁇ 128 pixels, and 96 ⁇ It is possible to divide into 128-pixel inter prediction partitions.
  • the inter prediction parameter PP_Inter includes a reference image index RI, an estimated motion vector index PMVI, and a motion vector residual MVD.
  • the intra prediction information PI_Intra includes an encoding parameter that is referred to when the video decoding device 1 generates an intra predicted image by intra prediction.
  • the intra prediction information PI_Intra includes intra PU partition information SP_Intra that specifies a partition pattern of the target PU (intra PU) into each partition, and intra prediction parameters PP_Intra1 to PP_IntraNa (Na is the total number of intra prediction partitions included in the target PU).
  • the intra-PU partition information SP_Intra is information that specifies the shape and size of each intra-predicted partition included in the target PU, and the position in the target PU.
  • the intra PU split information SP_Intra includes an intra split flag (intra_split_flag) that specifies whether or not the target PU is split into partitions. If the intra partition flag is 1, the target PU is divided symmetrically into four partitions. If the intra partition flag is 0, the target PU is not divided and the target PU itself is one partition.
  • N 2 n , n is an arbitrary integer of 1 or more.
  • a 128 ⁇ 128 pixel intra PU can be divided into 128 ⁇ 128 pixel and 64 ⁇ 64 pixel intra prediction partitions.
  • the intra prediction parameter PP_Intra includes an edge-based prediction flag EF, a sub-direction index AI, an estimation flag MPM, and a residual prediction mode index RIPM as shown in (f) of FIG.
  • the intra prediction parameter PP_Intra is a parameter for designating an intra prediction method (prediction mode) for each partition.
  • the edge-based prediction flag EF is a flag that specifies whether or not the edge-based prediction mode is applied to the target partition that is the processing target partition. In the following description, it is assumed that the edge-based prediction mode is applied to the target partition when the edge-based prediction flag is 1.
  • the correction direction index (also referred to as an additional index) AI is an index included in the prediction parameter PP_Intra when the edge-based prediction mode is applied to the target partition, and is the main direction derived in the edge-based prediction mode. This is an index for designating whether or not to add the correction direction, and for specifying the correction direction to be added when adding the correction direction.
  • the total number of correction directions that can be selected by the additional index AI differs depending on the size of the target partition. For example, when the size of the target partition is 4 ⁇ 4 pixels or 8 ⁇ 8 pixels, the total number of correction directions selectable by the additional index AI is 8, and when the size of the target partition is 16 ⁇ 16 pixels, the additional index When the total number of correction directions that can be selected by AI is 2, and the size of the target partition is 32 ⁇ 32 pixels or 64 ⁇ 64 pixels, the total number of correction directions that can be selected by the additional index AI is 0 (that is, the correction direction is not used). The main direction is always used).
  • the estimation flag MPM is a flag that is included in the prediction parameter PP_Intra when the edge-based prediction mode is not applied to the target partition, and is an estimated prediction mode that is estimated based on the prediction mode assigned to the surrounding partitions of the target partition. It is a flag indicating whether or not the prediction mode for the target partition is the same.
  • the residual prediction mode index RIPM specifies the prediction mode assigned to the target partition when the edge-based prediction mode is not applied to the target partition and the estimated prediction mode and the prediction mode for the target partition are different. It is an index to do.
  • the moving picture decoding apparatus 1 includes H.264 as a part thereof.
  • KTA software which is a codec for joint development in the Video Coding Expert Group (VCEG)
  • TMuC Traffic Model under Consideration
  • the video decoding device 1 generates a prediction image for each prediction unit, generates a decoded image # 2 by adding the generated prediction image and a prediction residual decoded from the encoded data # 1, The generated decoded image # 2 is output to the outside.
  • the generation of the predicted image is performed with reference to the encoding parameter obtained by decoding the encoded data # 1.
  • the encoding parameter is a parameter referred to in order to generate a prediction image, and in addition to a prediction parameter such as a motion vector referred to in inter-screen prediction and a prediction mode referred to in intra-screen prediction. Partition size and shape, block size and shape, and residual data between the original image and the predicted image.
  • side information a set of all information excluding the residual data among the information included in the encoding parameter is referred to as side information.
  • the prediction unit is a partition constituting the LCU
  • the present embodiment is not limited to this, and the prediction unit is a unit larger than the partition.
  • the present invention can also be applied to the case where the prediction unit is a unit smaller than the partition.
  • a frame, a slice, an LCU, a block, and a partition to be decoded are referred to as a target frame, a target slice, a target LCU, a target block, and a target partition, respectively.
  • the LCU size is, for example, 64 ⁇ 64 pixels
  • the partition size is, for example, 64 ⁇ 64 pixels, 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, 8 ⁇ 8 pixels, 4 ⁇ 4 pixels, or the like. These sizes do not limit the present embodiment, and the size and partition of the LCU may be other sizes.
  • FIG. 3 is a block diagram showing a configuration of the moving picture decoding apparatus 1.
  • the video decoding device 1 includes a variable length code demultiplexing unit 11, a header information decoding unit 12, an LCU setting unit 13, an LCU decoding unit 14, and a frame memory 15.
  • the encoded data # 1 input to the video decoding device 1 is input to the variable length code demultiplexing unit 11.
  • the variable-length code demultiplexing unit 11 demultiplexes the input encoded data # 1, thereby converting the encoded data # 1 into header encoded data # 11a that is encoded data related to header information, and a slice.
  • the encoded data # 11a is output to the header information decoding unit 12 and the encoded data # 11b is output to the LCU setting unit 13, respectively.
  • the header information decoding unit 12 decodes the header information # 12 from the encoded header data # 11a.
  • the header information # 12 is (1) information about the size, shape and position of the LCU belonging to the target slice, and (2) the size, shape and shape of the leaf CU belonging to each LCU. And (3) information about the size and shape of the partition belonging to each leaf CU and the position within the target leaf CU.
  • the LCU setting unit 13 separates the encoded data # 11b into encoded data # 13 corresponding to each LCU based on the input header information # 12 and sequentially outputs the encoded data # 11b to the LCU decoding unit 14.
  • the LCU decoding unit 14 generates and outputs a decoded image # 2 corresponding to each LCU by sequentially decoding the encoded data # 13 corresponding to each input LCU.
  • the decoded image # 2 is stored in the frame memory 15. The configuration of the LCU decoding unit 14 will be described later and will not be described here.
  • the decoded image # 2 is recorded in the frame memory 15.
  • decoded images corresponding to all the LCUs decoded before the target LCU are recorded. .
  • the decoded image # corresponding to the encoded data for one frame input to the video decoding device 1 2 generation processing is completed.
  • LCU decoding unit 14 LCU decoding unit 14
  • the LCU decoding unit 14 will be described more specifically with reference to different drawings.
  • FIG. 4 is a block diagram showing a configuration of the LCU decoding unit 14.
  • the LCU decoding unit 14 includes an LCU division structure setting unit 141, a leaf CU scanning unit 142, and a leaf CU decoding unit 143.
  • the LCU division structure setting unit 141 sets the division structure of the target LCU into leaf CUs with reference to the header information # 12.
  • the division structure information # 141 that specifies the division structure of the target LCU into leaf CUs is supplied to the leaf CU scanning unit 142 together with the encoded data # 13 about the target LCU.
  • FIG. 5 is a diagram illustrating an example of a division structure into which the target LCU is divided into leaf CUs and specified by the division structure information # 141.
  • the target LCU is divided into CUL1 to CUL7 that are leaf CUs.
  • the leaf CU scan unit 142 scans the leaf CUs included in the target LCU in a predetermined order (for example, raster scan order), and encodes data # 142a and CU information for the target leaf CU that is the leaf CU to be processed.
  • # 142b is supplied to the leaf CU decoding unit 143.
  • CU information # 142b includes (1) size, shape, and position in the target LCU for the target leaf CU, (2) PU information about the target leaf CU, and (3) TU information about the target leaf CU. , Including.
  • FIG. 5 is a diagram illustrating a case where CUL4 is set as the target leaf CU by the leaf CU scanning unit 142 among the CUL1 to CUL7 which are the leaf CUs included in the target LCU.
  • the leaf CU scanning unit 142 supplies the encoded data for CUL4, which is the leaf CU, and the CU information for CUL4 to the leaf CU decoding unit 143.
  • the leaf CU decoding unit 143 refers to the encoded data and CU information about the target leaf CU, and the decoded pixel values stored in the frame memory 15, and generates a predicted image for the target leaf CU.
  • the prediction residual for the target leaf CU is decoded.
  • the leaf CU decoding unit 143 generates a decoded image for the target leaf CU by adding the generated predicted image and the decoded prediction residual.
  • the leaf CU decoding unit 143 integrates the decoded images for each leaf CU belonging to the target LCU, and generates a decoded image # 2 for the target LCU.
  • FIG. 6 is a block diagram showing a configuration of the leaf CU decoding unit 143.
  • the leaf CU decoding unit 143 includes a PU decoding unit 431, a predicted image generation unit 432, a TU decoding unit 433, a prediction residual restoration unit 434, and a decoded image generation unit 435.
  • the PU decoding unit 431 determines the division pattern for each partition of the target leaf CU by decoding the PU information about the target leaf CU, determines the prediction mode for each partition, and determines each determined prediction mode # 431 is assigned to each partition.
  • the prediction mode # 431 assigned to each partition is supplied to the predicted image generation unit 432.
  • the prediction mode designates a generation method for generating a prediction image by intra-screen prediction (intra prediction).
  • intra prediction intra-screen prediction
  • a basic prediction mode set or edge-based prediction is used for each partition. Any prediction mode is selected from the mode set.
  • the basic prediction mode set is (1) a direction prediction mode for generating a prediction image for the target partition by extrapolating a decoded image around the target partition along a predetermined prediction direction, and (2) A DC prediction mode for generating a prediction image for the target partition by taking an average value of decoded pixel values around the target partition.
  • FIG. 7 is a diagram illustrating prediction modes included in the basic prediction mode set according to the present embodiment, together with a prediction mode index. Moreover, in FIG. 7, the prediction direction of each direction prediction mode is shown. As shown in FIG. 7, the basic prediction mode set in this embodiment includes a direction prediction mode specified by indexes 0, 1, 3 to 8 and a DC prediction mode specified by index 2, respectively.
  • the basic prediction mode set is exemplified as a case where a prediction mode that specifies any of eight different direction predictions is included.
  • the present embodiment is not limited to this. Absent.
  • a set including a prediction mode that specifies any of nine or more different directions may be used as the basic prediction mode set.
  • a set including a prediction mode for designating any of 16 different directions and a prediction mode for designating any of 32 different directions can be given.
  • the edge-based prediction mode set is (1) deriving the main direction for the target partition based on the decoded pixel values around the target partition, and surrounding the target partition along the main direction.
  • An edge-based prediction mode (hereinafter also referred to as a main direction edge-based prediction mode) that generates a predicted image in the target partition by extrapolating the decoded pixel values of (2), and (2)
  • Edge-based prediction mode for deriving a sub-direction by adding direction correction and generating a predicted image in the target partition by extrapolating decoded pixel values around the target partition along the sub-direction (Hereinafter also referred to as a sub-direction edge-based prediction mode).
  • FIG. 1 is a flowchart showing a flow of PU information decoding processing by the PU decoding unit 431.
  • FIG. 8A is a table showing the value of the edge-based prediction flag EF, the value of the additional index AI, the value of the estimation flag MPM, and the value of the residual prediction mode index RIPM that can be taken for the target partition.
  • FIG. 8B is a diagram illustrating the prediction direction corresponding to each value of the decoded additional index AI.
  • the PU decoding unit 431 refers to an intra partition flag (intra_split_flag) that specifies whether or not to divide the target leaf CU into partitions among the PU information about the target leaf CU, and converts the target leaf CU into a plurality of partitions. It is determined whether to divide or handle the target leaf CU as one partition. Specifically, the PU decoding unit 431 divides the target leaf CU symmetrically into four partitions if the intra split flag is 1, and splits the target leaf CU if the intra split flag is 0. It handles as one partition without doing.
  • intra_split_flag an intra partition flag
  • Step S102 the PU decoding unit 431 initializes the value of the loop variable part_id to 0, and starts a loop process that sets the increment value of the loop variable part_id for each loop to 1 for part_id that satisfies part_id ⁇ Npart-1.
  • the loop variable part_id is an index for mutually identifying partitions included in the target leaf CU
  • Npart is the total number of partitions included in the target leaf CU.
  • Step S103 Subsequently, the PU decoding unit 431 decodes the edge-based prediction flag EF for the target partition specified by the loop variable part_id.
  • Step S104 the PU decoding unit 431 determines whether or not the edge-based prediction flag EF decoded in step S103 indicates that the edge-based prediction mode is applied to the target partition.
  • Step S105 When the edge-based prediction mode is not applied to the target partition (No in step S104), the PU decoding unit 431 decodes the estimation flag MPM for the target partition. If the estimation flag MPM is 0, the PU decoding unit 431 continues to decode the residual prediction mode index RIPM for the target partition.
  • the residual prediction mode index RIPM is the prediction mode specified by the estimation flag MPM from the prediction mode included in the basic prediction mode set. Any of a total of eight types of prediction modes excluding the above is designated. Therefore, in the present embodiment, the residual prediction mode index RIPM is expressed by a 3-bit bit string.
  • Step S106 the PU decoding unit 431 selects a prediction mode for the target partition from the basic prediction mode set, and assigns the prediction mode to the target partition. Specifically, if the estimated MPM flag decoded in step S105 is 1, the estimated prediction mode obtained by referring to the prediction modes assigned to the decoded partitions around the target partition is set for the target partition. Set to the prediction mode. If the estimation flag MPM decoded in step S105 is 0, the PU decoding unit 431 sets the prediction mode specified by the residual prediction mode index RIPM as the prediction mode for the target partition.
  • Step S107 On the other hand, when the edge-based prediction mode is applied to the target partition (Yes in step S104), the PU decoding unit 431 determines the total number of correction directions that can be selected for the target partition according to the size of the target partition. . More specifically, the PU decoding unit 431 determines the total number of correction directions that can be selected for the target partition so as to have a negative correlation with the size of the target partition.
  • the total number of correction directions that can be selected for the target partition is set to 8
  • the target when the total number of correction directions that can be selected for a partition is set to 2 and the size of the target partition is 32 ⁇ 32 pixels or 64 ⁇ 64 pixels, the total number of correction directions that can be selected for the target partition is set to zero.
  • the PU decoding unit 431 sets the total number of correction directions that can be selected for the target partition to Nad1
  • the total number of pixels belonging to the target partition is Npix2.
  • the total number of correction directions that can be selected for the target partition is set to Nad2 (Nad1 ⁇ Nad2).
  • the PU decoding unit 431 is the division information included in the encoded data # 1, and includes CU division information SP_CU that specifies a division pattern for each leaf CU of the target LCU including the target partition, and the target PU (target The size of the target partition can be identified by referring to the intra PU partition information SP_Intra that specifies the partition pattern of each leaf CU).
  • Step S108 the PU decoding unit 431 decodes the additional index AI for the target partition.
  • the most significant bit of the additional index AI functions as a flag indicating whether the main direction is used as it is or as shown in FIG. 8A.
  • the bit string other than the most significant bit of the additional index AI functions as information for designating any correction direction from selectable correction directions.
  • Step S110 When the most significant bit of the additional index AI indicates that the main direction is used as it is (YES in step S109), the PU decoding unit 431 sets the prediction mode for the target partition to the main direction edge-based prediction mode. To do.
  • Step S111 When the most significant bit of the additional index AI indicates that the sub-direction is used (NO in step S109), the PU decoding unit 431 decodes a bit string (binary representation) other than the most significant bit of the additional index AI. Then, the sub-direction index k (decimal expression) designated by the bit string is determined. Also, the PU decoding unit 431 sets the prediction mode for the target partition to the sub-direction edge-based prediction mode specified by the sub-direction index k.
  • the possible values of the sub-direction index k are, for example, any one of ⁇ 1 and +1 It is.
  • the total number of correction directions that can be selected is 8, and thus the possible values of the sub-direction index k are, for example, ⁇ 4, -3, -2, -1, +1, +2, +3, and +4.
  • the total number of correction directions that can be selected is 0, and the additional index AI is not decoded.
  • the bit string other than the most significant bit of the additional index AI corresponding to the sub-direction index k is composed of a bit string corresponding to abs (k) and a bit string corresponding to sign (k). it can.
  • the bit string corresponding to sign (k) is 0 when k is a negative value and 1 when k is a positive value.
  • FIG. 8B is a diagram illustrating an example of the main direction and the sub-direction set by the PU decoding unit 431 when the size of the target partition is 4 ⁇ 4 pixels.
  • the angle ⁇ sk that designates the sub direction (sub direction vector) includes the sub direction index k and the correction.
  • ⁇ sk ⁇ m + k ⁇ ⁇ (1-1) It is expressed.
  • the parameter ⁇ is derived by the predicted image generation unit 432 according to a method described later.
  • Step S112 This step is the end of the loop.
  • the above is the flow of PU information decoding processing by the PU decoding unit 431.
  • the prediction mode set for each partition included in the target leaf CU in step S106, step S110, and step S111 is supplied to the predicted image generation unit 432 as a prediction mode # 431.
  • the prediction image generation unit 432 refers to the prediction image for each partition included in the target leaf CU with reference to the prediction mode # 431 supplied from the PU decoding unit 431 and the decoded pixel values around the partition. Generate.
  • FIG. 9 is a flowchart showing the flow of predicted image generation processing for the target leaf CU by the predicted image generation unit 432.
  • the predicted image generation unit 432 initializes the value of the loop variable part_id to 0, and starts loop processing that sets the increment value of the loop variable part_id for each loop to 1 for part_id that satisfies part_id ⁇ Npart-1.
  • the loop variable part_id is the same loop variable as described above. That is, the loop variable part_id is an index for mutually identifying partitions included in the target leaf CU, and Npart is the total number of partitions included in the target leaf CU.
  • the predicted image generation unit 432 refers to the prediction mode # 431 supplied from the PU decoding unit 431, refers to the prediction mode for the target partition specified by the loop variable part_id, and performs edge-based prediction for the target partition. Determine whether the mode is applied.
  • Step S203 When the edge-based prediction mode is not applied to the target partition (No in step S202), the predicted image generation unit 432 uses the direction prediction mode as the prediction mode for the target partition, that is, a prediction mode that uses a predetermined prediction direction. It is determined whether or not.
  • Step S204 When the prediction mode for the target partition is not the direction prediction mode (No in step S203), the predicted image generation unit 432 averages the decoded pixel values around the target partition for the predicted image for the target partition. Generate by.
  • FIG. 10 is a diagram illustrating each pixel (prediction target pixel) of the target partition, which is 4 ⁇ 4 pixels, and pixels (reference pixels) around the target partition.
  • the prediction target pixels are denoted by symbols a to p
  • the reference pixels are denoted by symbols A to M
  • the pixel value of the pixel X (X is any of a to p or A to M) It will be expressed as Further, it is assumed that the reference pixels A to M have all been decoded.
  • ave Indicates that an element included in parentheses is averaged.
  • Step S205 When the prediction mode for the target partition is the direction prediction mode (Yes in step S203), the predicted image generation unit 432 sets the prediction direction corresponding to the direction prediction mode as the prediction direction for the target partition.
  • Step S206 On the other hand, when the prediction mode for the target partition is the edge-based prediction mode (Yes in step S202), the predicted image generation unit 432 is based on decoded pixel values (pixel values of reference pixels) around the target partition. Thus, the main direction assigned to the target partition is derived.
  • the predicted image generation unit 432 performs, for example, the following sub-step S206-1 to sub-step S206-4.
  • FIG. 11 is a diagram showing the target partition OP together with adjacent partitions NP2 and NP3 adjacent to the target partition OP, and a partition NP1 sharing the top left vertex of the target partition.
  • the pixel values of the pixels included in the adjacent partitions NP1 to NP3 shown in FIG. 11 have been decoded.
  • pixels included in partitions NP1 to NP3 around the target partition OP can be used as reference pixels.
  • FIG. 11 shows the case where the target partition OP and the adjacent partitions NP1 to NP3 are all 4 ⁇ 4 pixels, but the size of these partitions is 8 ⁇ 8 pixels or other sizes. The same applies to the case of.
  • the predicted image generation unit 432 initializes the value of the loop variable i to 1, and starts a loop process that sets the increment value of the loop variable i for each loop to 1 for j that satisfies i ⁇ M.
  • M is the number of reference pixels that are referred to in order to derive the main direction for the target partition.
  • the prediction image generation unit 432 calculates the edge vectors b i.
  • the calculation of the edge vectors b i is Sobel operator (Sobel operators, also referred to as a Sobel filter) Gx, and may be used to Gy.
  • the Sobel filters Gx and Gy are filter matrices used for calculating the image gradient along the x direction and the image gradient along the y direction, respectively. For example, as a 3 ⁇ 3 matrix, ,
  • Prediction image generating unit 432 Given by. Prediction image generating unit 432, image gradient for the calculated x-direction, and calculates an edge vector b i that is perpendicular to the image gradients represented by the image gradient along the y-direction.
  • the predicted image generation unit 432 has a function T ( ⁇ ) shown below.
  • T ( ⁇ ) ⁇ ⁇ e, b i > 2
  • e represents a unit vector whose angle between its own direction and the horizontal direction (x direction) is ⁇
  • ⁇ represents the inner product of both vectors.
  • the symbol ⁇ indicates that the subscript i is to be summed from 1 to M.
  • the predicted image generation unit 432 calculates the main direction
  • the partition adjacent to the upper side of the target partition, the partition adjacent to the left side of the target partition, and the upper left vertex of the target partition are shared.
  • the case where the pixel values of the pixels belonging to the partition are referred to has been taken as an example, but the present embodiment is not limited to this, and the predicted image generation unit 432 is more generally a reference set around the target partition. With reference to the decoded pixel values belonging to the region, the main direction for the target partition can be calculated.
  • the method of deriving the main direction by the predicted image generation unit 432 is not limited to the above example.
  • the predicted image generation unit 432 may be configured to calculate, as the main direction, the edge vector having the largest magnitude among the edge vectors b i calculated for the reference pixel, that is, the direction indicated by the edge vector having the largest norm. Good.
  • Step S207 the predicted image generation unit 432 determines whether or not the prediction mode for the target partition is the main direction prediction mode.
  • Step S208 When the prediction mode for the target partition is the main direction prediction mode (Yes in step S207), the predicted image generation unit 432 sets the main direction calculated in step S206 as the prediction direction for the target partition.
  • Step S209 On the other hand, when the prediction mode for the target partition is not the main direction prediction mode (No in step S207), the predicted image generation unit 432 performs a quantization step for specifying the roughness of quantization when the correction angle is quantized. A parameter ⁇ representing the size is calculated. In the present embodiment, the predicted image generation unit 432 has confidence measures s shown below.
  • the parameter ⁇ is calculated as a function of Note that the parameter ⁇ derived by the predicted image generation unit 432 has a property of being a decreasing function of the certainty factor s.
  • the predicted image generation unit 432 may be configured to use a predetermined parameter ⁇ , for example, instead of the configuration that calculates the parameter ⁇ individually for each partition.
  • the parameter ⁇ is encoded for each slice or LCU, the parameter ⁇ is included in the encoded data # 1, and the predicted image generation unit 432 A configuration using the parameter ⁇ decoded from the encoded data # 1 may be adopted.
  • the predicted image generation unit 432 calculates an angle ⁇ sk indicating the sub-direction for the target partition using Equation (1-1) using the sub-direction index k obtained by decoding the additional index AI.
  • Step S210 Subsequently, the predicted image generation unit 432 sets the direction indicated by the angle ⁇ sk calculated in step S209 as the predicted direction for the target partition.
  • Step S211 the predicted image generation unit 432 extrapolates the decoded pixel values around the target partition along the prediction direction for the target partition set in step S205, step S208, and step S210. Then, a predicted image for the target partition is generated.
  • the predicted image generation unit 432 uses the pixel position of the prediction target pixel in the target partition as a starting point, and among the decoded pixels located on the virtual line segment facing the reverse direction of the prediction direction, By setting the pixel value of the pixel closest to the pixel (hereinafter also referred to as the closest pixel) to the pixel value of the prediction target pixel, a predicted image in the target partition is generated. Further, the pixel value of the prediction target pixel may be a value calculated using the pixel value of the nearest pixel and the pixel values of the pixels around the nearest pixel.
  • the predicted image generation unit 432 includes the pixel value a in the example illustrated in FIG. 10.
  • “>>” represents a right shift operation, and for any positive integer x, s, the value of x >> s is equal to the value obtained by rounding down the decimal part of x
  • the predicted image generation unit 432 can calculate the pixel values a to p by the same method for the basic prediction modes other than the above prediction modes. Also, the predicted image generation unit 432 can calculate the pixel values a to p by the same method for the edge-based prediction mode.
  • Step S212 This step is the end of the loop whose loop variable is id_part.
  • the predicted image generation unit 432 generates a predicted image for the target leaf CU by performing the above processing. Further, the generated predicted image for the target leaf CU is supplied to the decoded image generation unit 435 as a predicted image # 432.
  • the TU decoding unit 433 decodes TU information for the target leaf CU. Also, the TU decoding unit 433 divides the target leaf CU into one or a plurality of blocks with reference to the TU partition information SP_TU included in the TU information for the target leaf CU. Also, the quantization prediction residual for each block is decoded from the encoded data # 142a for the target leaf CU, and the decoded quantization prediction residual is supplied to the prediction residual restoration unit 434.
  • the prediction residual restoration unit 434 performs inverse quantization and inverse DCT transform (Inverse Discrete Cosine Transform) on the quantized prediction residual supplied from the TU decoding unit 433, so that the prediction residual for each partition is obtained. Restore the difference.
  • the restored prediction residual # 434 is supplied to the decoded image generation unit 435.
  • the decoded image generation unit 435 adds the prediction image # 432 supplied from the prediction image generation unit 432 and the prediction residual supplied from the prediction residual restoration unit 434, thereby obtaining a decoded image for the target leaf CU. Generate.
  • the decoded image generation unit 435 generates a decoded image # 2 for the target LCU by integrating the decoded images for each leaf CU included in the target LCU.
  • FIG. 12A is a flowchart showing an overall flow of decoding processing by the leaf CU decoding unit 143.
  • Step S301 the leaf CU decoding unit 143 uses the TU decoding unit 433 to decode the TU information for the target leaf CU. Further, the target leaf CU is divided into one or a plurality of blocks with reference to the TU partition information SP_TU included in the TU information regarding the target leaf CU. Also, the quantization prediction residual for each block is decoded from the encoded data # 142a for the target leaf CU, and the decoded quantization prediction residual is supplied to the prediction residual restoration unit 434.
  • Step S302 the leaf CU decoding unit 143 performs inverse quantization and inverse DCT transform on the quantized prediction residual decoded in step S301 by the prediction residual restoration unit 434, so that each pixel for the target leaf CU is processed. Restore the prediction residual of.
  • Step S303 the leaf CU decoding unit 143 determines the division pattern for each partition of the target leaf CU by decoding the PU information about the target leaf CU in the PU decoding unit 431, and also predicts each partition. A mode is determined, and each determined prediction mode is assigned to each partition. Since specific PU information decoding processing has already been described, description thereof will be omitted.
  • Step S304 the leaf CU decoding unit 143 causes the predicted image generation unit 432 to initialize the value of the loop variable part_id to 0, and for the part_id satisfying part_id ⁇ Npart ⁇ 1, the increment value of the loop variable part_id for each loop is set to 1.
  • the loop processing is started.
  • the loop variable part_id is the same loop variable as described above. That is, the loop variable part_id is an index for mutually identifying partitions included in the target leaf CU, and Npart is the total number of partitions included in the target leaf CU.
  • Step S305 the leaf CU decoding unit 143 generates a predicted image for the partition specified by part_id in the predicted image generation unit 432. Since specific prediction image generation processing has already been described, description thereof will be omitted.
  • Step S306 the leaf CU decoding unit 143 adds the prediction image # 432 supplied from the prediction image generation unit 432 and the prediction residual supplied from the prediction residual restoration unit 434, thereby adding a prediction for the target leaf CU.
  • a decoded image generation unit 435 generates a decoded image.
  • Step S307 This step is the end of the loop.
  • the video decoding device 1 includes the prediction mode belonging to the prediction mode group in the prediction residual decoded from the encoded data together with the prediction mode designation information and other side information, and the prediction In an image decoding apparatus that generates a decoded image by adding predicted images generated according to a prediction mode specified by mode specifying information, the number of prediction modes N (N is an arbitrary natural number) added to the prediction mode group related to the target prediction unit Referring to a coding parameter included in the side information and including a coding unit including the target prediction unit or a coding unit decoded prior to the coding unit.
  • PU decoding unit 431 to set and each of the N prediction modes to be added to the prediction mode group related to the target prediction unit
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually encoded that is decoded before the prediction unit or the encoding unit. It can be determined from the coding parameters for the unit.
  • the amount of code of side information necessary for encoding an encoding target image for a certain encoding unit has a positive correlation with the total number of prediction units included in the encoding unit. That is, if the total number of prediction units is large, the code amount of the side information increases.
  • the total number of prediction units included in a certain coding unit has a negative correlation with the size of each prediction unit.
  • the PU decoding unit 431 determines the total number of correction directions that can be selected for the target partition so as to have a negative correlation with the size of the target partition. Specifically, when the size of the target partition is 4 ⁇ 4 pixels or 8 ⁇ 8 pixels, the total number of correction directions that can be selected for the target partition is set to 8, and the size of the target partition is 16 ⁇ 16 pixels. In this case, when the total number of correction directions that can be selected for the target partition is set to 2 and the size of the target partition is 32 ⁇ 32 pixels or 64 ⁇ 64 pixels, the total number of correction directions that can be selected for the target partition is set to 0. .
  • FIG. 12B shows the result of measuring the relationship between the number of correction directions and the effect of introducing edge-based prediction for each partition size.
  • the horizontal axis indicates the number of correction directions
  • sd0, sd1, sd2, sd3, and sd4 indicate the number of correction directions of 0, 2, 4, 6, and 8, respectively.
  • the vertical axis indicates the effect of introducing edge-based prediction. More precisely, the rate distortion cost reduction achieved by making it possible to select the additional prediction mode in addition to the basic prediction mode is reduced based on the rate distortion cost when only the basic prediction mode can be selected. Expressed as a percentage.
  • FIG. 12B shows that when the partition size is 4 ⁇ 4 pixels or 8 ⁇ 8 pixels, the rate distortion cost can be reduced most by setting sd4, that is, eight correction directions.
  • sd4 that is, eight correction directions.
  • 16 ⁇ 16 pixels it can be seen that if sd1, that is, two correction directions are used, the rate distortion cost can be sufficiently reduced, and the effect is not recognized even if more correction directions are added.
  • sd0 that is, when the number of correction directions is 0 (only the main direction is used)
  • the reduction rate of the rate distortion cost is the maximum. It turns out that it is.
  • the addition of the correction direction is related to both the increase of the code amount of the side information and the improvement of the prediction accuracy.
  • the influence of the side information increase per unit area is larger than in the case of the large size.
  • the improvement range of the prediction accuracy is large as compared with the case of the large size. This is because there is a high possibility that a straight line exists in a block at a small size. From the experimental results shown in FIG. 12 (b), it can be said that the influence of the latter prediction accuracy improvement exceeds the influence of the former side information increase.
  • Moving picture encoding apparatus 2 (Moving picture encoding device 2)
  • the moving picture encoding apparatus (encoding apparatus) 2 according to the present embodiment will be described with reference to FIGS.
  • the same part as the part already demonstrated is attached
  • FIG. 13 is a block diagram showing a configuration of the moving picture encoding apparatus 2.
  • the moving image encoding device 2 includes an LCU header information determination unit 21, a header information encoding unit 22, an LCU setting unit 23, an LCU encoding unit 24, a variable length code multiplexing unit 25, and an LCU decoding.
  • a unit 26 and a frame memory 27 are provided.
  • the video encoding device 2 is a device that generates and outputs encoded data # 1 by encoding the input image # 100, in brief.
  • the moving image encoding apparatus 2 includes, as part thereof, H.264. H.264 / MPEG-4 AVC standard technology, VCEG (Video Coding Expert Group) technology used in KTA software, which is a joint development codec, and successor codec TMuC (Test Model Under Consideration) )
  • KTA software which is a joint development codec, and successor codec TMuC (Test Model Under Consideration)
  • the LCU header information determination unit 21 determines LCU header information based on the input image # 100.
  • the determined LCU header information is output as LCU header information # 21.
  • the LCU header information # 21 includes (1) information about the size and shape of the LCU belonging to the target slice and the position in the target slice, and (2) the size, shape and target of the leaf CU belonging to each LCU. Contains information about the location within the LCU.
  • the LCU header information # 21 is input to the LCU setting unit 23 and also supplied to the header information encoding unit 22.
  • the header information encoding unit 22 encodes the LCU header information # 21 supplied from the LCU header information determination unit 21 and the leaf CU header information # 53 supplied from the LCU encoding unit 24, and has encoded header information. # 22 is output. The encoded header information # 22 is supplied to the variable length code multiplexer 25.
  • the LCU setting unit 23 divides the input image (encoding target image) # 100 into a plurality of LCUs based on the LCU header information # 21, and outputs an LCU image # 23 related to each LCU.
  • the LCU image # 23 is sequentially supplied to the LCU encoding unit 24.
  • the LCU encoding unit 24 encodes sequentially input LCU image # 23 to generate LCU encoded data # 24.
  • the generated LCU encoded data # 24 is supplied to the variable length code multiplexing unit 25 and the LCU decoding unit 26. Since the configuration of the LCU encoding unit 24 will be described later, description thereof is omitted here.
  • variable length code multiplexing unit 25 generates encoded data # 1 by multiplexing encoded header information # 22 and LCU encoded data # 24, and outputs the encoded data # 1.
  • the LCU decoding unit 26 sequentially decodes the LCU encoded data # 24 corresponding to each input LCU, thereby generating and outputting a decoded image # 26 corresponding to each LCU.
  • the decoded image # 26 is supplied to the frame memory 27. Since the configuration of the LCU decoding unit 26 is the same as the configuration of the LCU decoding unit 14 included in the video decoding device 1, detailed description thereof is omitted here.
  • the input decoded image # 26 is recorded in the frame memory 27.
  • decoded images corresponding to all the LCUs preceding the target LCU in the raster scan order are recorded in the frame memory 27.
  • the decoded image recorded in the frame memory is called decoded image # 27. That is, the decoded image # 27 corresponds to the decoded image # 26 recorded in the frame memory 27.
  • LCU encoder 24 LCU encoder 24
  • the LCU encoding unit 24 will be described more specifically with reference to different drawings.
  • FIG. 14 is a block diagram showing a configuration of the LCU encoding unit 24.
  • the LCU encoding unit 24 includes an LCU division structure determining unit 241, a leaf CU scanning unit 242, and a leaf CU encoding unit 243.
  • the LCU partition structure determining unit 241 refers to the LCU header information # 21 to determine the partition structure of the target LCU into leaf CUs.
  • Division structure information # 241 for designating the division structure of the target LCU into leaf CUs is supplied to the leaf CU scanning unit 242 together with the LCU image # 23 for the target LCU.
  • An example of the divided structure determined by the LCU divided structure determining unit 241 is the one shown in FIG.
  • the leaf CU scanning unit 242 scans the leaf CUs included in the target LCU in a predetermined order (for example, raster scan order), and the leaf CU image # 242a and CU information about the target leaf CU that is the leaf CU to be processed.
  • # 242b is supplied to the leaf CU encoding unit 243.
  • the CU information # 242b includes the size and shape of the target leaf CU and the position in the target LCU.
  • the leaf CU encoding unit 243 refers to the leaf CU image and CU information for the target leaf CU, and the decoded pixel value stored in the frame memory 15, and generates a predicted image for the target leaf CU. At the same time, the prediction residual between the leaf CU image and the prediction image is encoded. Also, the leaf CU encoding unit 243 integrates the encoded prediction residuals for each leaf CU belonging to the target LCU, and generates LCU encoded data # 24 for the target LCU.
  • FIG. 15 is a block diagram showing the configuration of the leaf CU encoding unit 243.
  • the leaf CU encoding unit 243 includes a PU determination unit 531, a predicted image generation unit 532, a decoded image generation unit 533, a prediction residual calculation unit 534, a TU determination unit 535, and a prediction residual restoration unit 536.
  • the PU determination unit 531 refers to the leaf CU image # 242a and the CU information # 242b, and determines (1) a division pattern for each partition of the target leaf CU and (2) a prediction mode to be allocated to each partition. To do.
  • a division pattern into the partitions of the target leaf CU for example, a division pattern that is handled as one partition without dividing the target leaf CU, or the target leaf CU is divided into four partitions symmetrically. One of the division patterns can be selected.
  • the PU determination unit 531 determines the total number of correction directions that can be selected for the target partition according to the size of the target partition. More specifically, the PU determination unit 531 determines the total number of correction directions that can be selected for the target partition so as to have a positive correlation with the size of the target partition.
  • the total number of correction directions that can be selected for the target partition is set to 8
  • the target when the total number of correction directions that can be selected for the partition is set to 2 and the size of the target partition is 32 ⁇ 32 or 64 ⁇ 64, the total number of correction directions that can be selected for the target partition is set to 0.
  • the PU determination unit 531 determines the total number of correction directions that can be selected for the target partition as Nad1, and the total number of pixels belonging to the target partition is Npix2 ( When Npix1 ⁇ Npix2), the total number of correction directions that can be selected for the target partition is determined to be Nad2 (Nad1 ⁇ Nad2).
  • PU information including (1) an intra split flag (intra_split_flag) that specifies whether or not the target leaf CU is split into four partitions, and (2) prediction mode information that specifies a prediction mode assigned to each partition.
  • # 531 is supplied to the predicted image generation unit 532 and the LCU encoded data generation unit 538.
  • the PU information # 531 includes an additional index AI including a 1-bit flag that specifies whether to use the main direction or the sub-direction for the target partition.
  • the additional index AI includes a plurality of sub-directions and an index that specifies which of the plurality of sub-directions whose total number is determined according to the size of the target partition is used. It is.
  • the PU determination unit 531 applies to each of all combinations of (1) a possible division pattern for each partition of the target leaf CU and (2) a prediction mode that can be assigned to each partition. Corresponding PU information is generated. In addition, generation of a prediction image, a prediction residual, and a decoded image, which will be described later, is performed for each piece of PU information.
  • the PU determination unit 531 supplies PU information that optimizes rate distortion supplied from the rate distortion evaluation unit 537 described later to the header information encoding unit 22 as part of the leaf CU header information # 53. Then, the PU information for optimizing the rate distortion is supplied to the predicted image generation unit 532 and the LCU encoded data generation unit 538 as the final PU information for the target leaf CU.
  • the predicted image generation unit 532 generates a predicted image # 532 for the target leaf CU with reference to the decoded image # 27 stored in the frame memory 27 and the PU information # 531 supplied from the PU determination unit 531. To do. Since the predicted image generation process by the predicted image generation unit 532 is the same as the predicted image generation unit 432 included in the video decoding device 1, description thereof is omitted here.
  • the prediction residual calculation unit 534 generates a prediction residual # 534 by subtracting the prediction image # 532 from the leaf CU image # 242a.
  • the generated prediction residual # 534 is supplied to the TU determination unit 535.
  • the TU determination unit 535 refers to the leaf CU image # 242a and the CU information # 242b, and determines a division pattern of the target leaf CU into one or a plurality of blocks. Also, according to the determined division pattern, the prediction residual # 534 for the target leaf CU supplied from the prediction residual calculation unit 534 is divided into prediction residuals for each block.
  • the TU determination unit 535 generates a prediction residual in the frequency domain by performing DCT transform (DiscretecreCosine Transform) on the prediction residual for each block, and then quantizes the prediction residual in the frequency domain. Generate a quantized prediction residual for each block.
  • the generated quantization prediction residual for each block is transmitted to the prediction residual restoration unit 536 and the LCU encoded data generation unit 538 as TU information # 535 together with TU partition information SP_TU that specifies the partition pattern of the target leaf CU. Supplied.
  • the TU determination unit 535 generates TU information # 535 corresponding to all possible division patterns for each block of the target leaf CU. Further, the generation of the quantized prediction residual is performed for all of the divided patterns.
  • the TU determination unit 535 supplies the TU information for optimizing the rate distortion supplied from the rate distortion evaluation unit 537 described later to the header information encoding unit 22 as leaf CU header information # 53, and the rate.
  • TU information for optimizing distortion is supplied to the prediction residual restoration unit 536 and the LCU encoded data generation unit 538 as final TU information for the target leaf CU.
  • the prediction residual restoration unit 536 performs prediction for each block by performing inverse quantization and inverse DCT transform (Inverse Discrete Cosine Transform) on the quantization prediction residual for each block supplied from the TU determination unit 535. Restore the residual. Further, the prediction residuals for each block are integrated according to the division pattern determined by the TU determination unit 535, and the prediction residual # 536 for the target leaf CU is generated. The prediction residual # 536 for the generated target leaf CU is supplied to the decoded image generation unit 533.
  • inverse quantization and inverse DCT transform Inverse Discrete Cosine Transform
  • the decoded image generation unit 533 adds the prediction image # 532 for the target leaf CU supplied from the prediction image generation unit 532 and the prediction residual # 536 for the target leaf CU supplied from the prediction residual restoration unit 536. As a result, a decoded image # 533 for the target leaf CU is generated. The generated decoded image # 533 for the target leaf CU is supplied to the rate distortion evaluation unit 537.
  • the rate distortion evaluation unit 537 refers to the decoded image # 533 for the target leaf CU and the leaf CU image # 242a for the target leaf CU supplied from the decoded image generation unit 533, and calculates the rate distortion for the target leaf CU. evaluate.
  • the rate distortion evaluation unit 537 includes (1) a possible division pattern for each partition of the target leaf CU, (2) a prediction mode that can be assigned to each partition, and (3) each block of the target leaf CU. Among possible combinations of divided patterns, a combination that optimizes the rate distortion is determined.
  • the PU information and TU information specified by the determined combination are supplied to the PU determination unit 531 and the TU determination unit 535, respectively.
  • the LCU encoded data generation unit 538 performs variable length encoding on the final PU information supplied from the PU determination unit 531 and the final TU information supplied from the TU determination unit 535, thereby performing the target leaf CU.
  • the encoded data for is generated.
  • the LCU encoded data generation unit 538 generates encoded data # 24 for the target LCU by multiplexing the encoded data for each leaf CU belonging to the target LCU.
  • FIG. 16 is a flowchart showing the flow of the encoding process performed by the leaf CU encoder 243.
  • Step S401 the leaf CU encoding unit 243 initializes the value of the loop variable pu_param_id to 0, and for the pu_param_id that satisfies pu_param_id ⁇ Npu ⁇ 1, performs a first loop process in which the increment value of the loop variable pu_param_id for each loop is 1.
  • the loop variable pu_param_id is an index for mutually identifying combinations of possible division patterns for each partition of the target leaf CU and possible prediction modes to be assigned to each partition.
  • Npu represents the total number of combinations of possible division patterns for each partition of the target leaf CU and prediction modes that can be assigned to each partition.
  • Npu 1 ⁇ Nmode + 4 ⁇ Nmode
  • pu_param_id functions as an index for identifying these combinations of 1 ⁇ Nmode + 4 ⁇ Nmode from each other.
  • Step S402 the leaf CU encoding unit 243 initializes the value of the loop variable tu_param_id to 0, and for the tu_param_id that satisfies tu_param_id ⁇ Ntu ⁇ 1, the second loop processing sets the increment value of the loop variable tu_param_id to 1 for each loop.
  • the loop variable tu_param_id is an index for mutually identifying possible division patterns for each block of the target leaf CU.
  • Ntu represents the total number of possible division patterns for each block of the target leaf CU.
  • the size of the target leaf CU is 8 ⁇ 8 pixels
  • the division pattern in which the target leaf CU is handled as it is without being divided and the target leaf CU is symmetrical to four 4 ⁇ 4 pixel blocks. If the division pattern to be divided is possible, the total number Ntu of possible division patterns for each block of the target leaf CU is 2, and the loop variable tu_param_id identifies these two division patterns from each other. Function as an index for
  • Step S403 the leaf CU encoding unit 243 initializes the value of the loop variable part_id to 0, and performs a third loop process in which the increment value of the loop variable part_id for each loop is set to 1 for part_id that satisfies part_id ⁇ Npart ⁇ 1.
  • the loop variable part_id is an index for identifying each partition
  • Npart is the total number of the partitions when the target leaf CU is divided into partitions according to the division pattern corresponding to the loop variable pu_param_id.
  • Step S404 the leaf CU encoding unit 243 generates a predicted image for the target partition specified by the set of loop variables (pu_param_id, tu_param_id, part_id) in the predicted image generation unit 532.
  • Step S405 the leaf CU encoding unit 243 calculates the prediction residual for the partition specified by the set of loop variables (pu_param_id, part_id) by the prediction residual calculation unit 534.
  • Step S406 the leaf CU encoding unit 243 generates a quantized prediction residual by DCT transforming and quantizing the prediction residual for the target block belonging to the target leaf CU in the TU determination unit 535.
  • Step S407 the leaf CU encoding unit 243 restores the prediction residual by performing inverse quantization and inverse DCT transform on the quantized prediction residual generated in step S406.
  • Step S408 the leaf CU encoding unit 243 generates a decoded image for the target partition by adding the prediction image generated in step S404 and the prediction residual restored in step S407.
  • Step S409 This step is the end of the third loop. Generation of the decoded image for the target leaf CU is completed by the third loop processing.
  • the rate distortion evaluation unit 537 refers to the code amount and the distortion for the target leaf CU and evaluates the rate distortion for the target leaf CU.
  • Step S411 This step is the end of the second loop.
  • Step S412 This step is the end of the first loop. Note that the first and second loop processes complete the rate distortion evaluation for all possible combinations of the loop variable pu_param_id and the loop variable tu_param_id.
  • Step S413 Subsequently, the leaf CU encoding unit 243, in the rate distortion evaluation unit 537, out of the rate distortions for all possible combinations of the loop variable pu_param_id and the loop variable tu_param_id, the loop variable pu_param_id and the loop corresponding to the optimum rate distortion The combination of the variable tu_param_id is determined.
  • Step S414 The leaf CU encoding unit 243 generates encoded data for the target leaf CU using a combination of the loop variable pu_param_id and the loop variable tu_param_id corresponding to the optimum rate distortion determined in step S413. That is, the leaf CU encoding unit 243 generates encoded data for the target leaf CU using a division pattern for dividing the target leaf CU into partitions and blocks and optimizing rate distortion.
  • the video encoding device 2 has selected the prediction residual obtained by subtracting the prediction image generated according to the prediction mode selected from the prediction mode group for each prediction unit from the original image.
  • An image encoding apparatus that generates encoded data by encoding together with prediction mode specifying information for specifying a prediction mode and other side information, and the number N of prediction modes to be added to the prediction mode group related to the target prediction unit,
  • the encoding parameter included in the side information is set with reference to an encoding unit including the target prediction unit or an encoding parameter related to an encoding unit encoded before the encoding unit.
  • Prediction parameters corresponding to each of the N prediction modes to be added to the setting unit (PU determination unit 531) and the prediction mode group related to the target prediction unit Includes a deriving means for deriving from the decoded region of the local decoded image (predicted image generation unit 532), the.
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually encoded that is decoded before the prediction unit or the encoding unit. It can be determined from the coding parameters for the unit.
  • the total number of basic prediction modes that can be selected for the target partition is a specified value
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 are Although the case where the total number of sub-directions that can be selected for a partition is changed according to the size of the target partition has been described, the present embodiment is not limited to this.
  • the total number of basic prediction modes that can be selected for each partition may be changed according to the size of the partition. Specifically, the total number of basic prediction modes that can be selected for a partition of 32 ⁇ 32 pixels is set to 9, and the total number of basic prediction modes that can be selected for a partition of 8 ⁇ 8 pixels is set to 33. .
  • a flag indicating the total number of basic prediction modes may be included in the encoded data, and the video decoding device 1 may be configured to decode the basic prediction mode with reference to the flags.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 determine the total number of sub-directions that can be selected for the target partition.
  • the configuration may be changed in accordance with the total number of basic prediction modes that can be selected for the target partition. For example, when the number of basic prediction modes that can be selected for the target partition is 9, the total number of sub-directions that can be selected for the target partition is 0, and the number of basic prediction modes that can be selected for the target partition is 33 A configuration is also possible in which the total number of sub-directions that can be selected for the target partition is 8.
  • the total number of basic prediction modes that can be selected for the target partition is Nba1
  • the total number of sub-directions that can be selected for the target partition is determined as Nad1
  • the target partition can be selected.
  • the total number of basic prediction modes is Nba2 (Nba1 ⁇ Nba2)
  • the total number of correction directions that can be selected for the target partition is determined as Nad2 (Nad1 ⁇ Nad2).
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 are the partitions adjacent to the target partition, and the edge-based prediction mode for the decoded partition Depending on whether or not is applied, the total number of sub-directions that can be selected for the target partition may be changed.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 have an edge-based prediction mode for at least one of the decoded partitions adjacent to the target partition. If applied, the total number of sub-directions that can be selected for the target partition is 2. If the edge-based prediction mode is not applied to any of the decoded partitions adjacent to the target partition, the target partition can be selected. The total number of sub-directions may be four.
  • FIG. 17 is a diagram illustrating a case where the edge-based prediction mode is applied to an adjacent partition adjacent to the left side of the target partition. As shown in FIG. 17, when the edge-based prediction mode is applied to the adjacent partition, the prediction direction derived using the edge-based prediction mode for the adjacent partition is also an appropriate prediction direction for the target partition. There is a tendency.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 determine the number of sub-directions that can be selected for the target partition.
  • the configuration can be reduced compared to the case where the edge-based prediction mode is not applied to any partition adjacent to the partition.
  • the prediction direction derived using the edge-based prediction mode for the adjacent partition adjacent to the target partition tends to be an appropriate prediction direction for the target partition.
  • the main direction derived for the target partition tends to be close to the prediction direction derived using the edge-based prediction mode for the adjacent partition.
  • the number of sub-directions for the sub-direction edge-based prediction mode that is unlikely to be selected can be reduced, so that the code amount of the encoded data # 1 can be reduced while maintaining high prediction accuracy. Since it can be reduced, encoding efficiency can be improved.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 are applied with the edge-based prediction mode for the decoded adjacent partitions adjacent to the target partition.
  • the value of the parameter ⁇ may be set smaller than the value of the parameter ⁇ when the edge-based prediction mode is not applied to the adjacent partition.
  • the prediction direction derived using the edge-based prediction mode for the adjacent partition adjacent to the target partition tends to be close to an appropriate prediction direction for the target partition.
  • the main direction derived for the target partition tends to be close to the optimum prediction direction for the target partition.
  • the above configuration increases the possibility that the sub-direction derived for the target partition matches the optimal prediction direction for the target partition, so that the prediction accuracy is improved and the encoding efficiency is improved.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 determine the number of correction directions that can be selected in the target partition based on the partition size of the target partition. Although determined, the number of correction directions may be determined based on the size of the leaf CU including the target partition. In that case, the number of executions of the determination process of the number of correction directions can be reduced, and the processing amount of the decoding process or the encoding process is reduced.
  • the PU decoding unit 431 included in the video decoding device 1 and the PU determination unit 531 included in the video encoding device 2 use the partition size of the target partition and the size of the leaf CU including the target partition in combination.
  • the number of correction directions may be determined. For example, even when the partition size is the same, if the size of the leaf CU to which the partition belongs is different, a different number of correction directions may be selected. In this case, even if the regions have the same partition size, a different number of correction directions can be added, so that an appropriate number of sub-directions can be set for each region, and the coding efficiency is improved.
  • ⁇ Modification 5 of Embodiment 1> The total number of selectable basic prediction modes for a 4 ⁇ 4 pixel partition is 18 (direction prediction mode: 16, DC prediction mode: 1, Planar prediction mode: 1), and 8 ⁇ 8 pixels, 16 The total number of selectable basic prediction modes for a partition of ⁇ 16 pixels and 32 ⁇ 32 pixels is 35 (direction prediction mode: 33, DC prediction mode: 1, Planar prediction mode: 1). Good.
  • the number of sub-directions for edge-based prediction is set to 0 for 4 ⁇ 4 pixel and 8 ⁇ 8 pixel partitions, In the partition of 16 ⁇ 16 pixels and 32 ⁇ 32 pixels, it is preferable to set the number of sub-directions to four.
  • decoding of the additional index AI can be omitted in the PU information decoding process by the PU decoding unit 431. That is, the decoding process of the additional index in step S108 in FIG. 1 is omitted, and the main direction prediction mode is set as the prediction mode for the target partition.
  • the Planar prediction mode (also referred to as plane prediction mode or plane prediction mode) refers to a mode in which a prediction image in the target partition is predicted two-dimensionally from pixel values of surrounding partitions. Indicates a mode in which the pixel value predSamples [x, y] (where [x, y] represents the coordinates of the pixel) of the predicted image in the target partition is calculated by, for example, the following equation.
  • the number of main directions derived in the edge-based prediction mode is one for each partition. However, depending on the image characteristics of the encoding target image, a plurality of main directions are derived for each partition. It is good also as a possible structure. Also, the total number of main directions that can be derived is changed according to the size of the target partition, and when a plurality of main directions can be derived, one of the main directions is selected and used. It is good.
  • the total number of main directions that can be derived is changed according to the size of the target partition, and when a plurality of main directions can be derived, the plurality of main directions are used.
  • a configuration for selecting and using any of the main directions will be described. In the following, the same parts as those already described are denoted by the same reference numerals, and the description thereof is omitted.
  • the encoded data generated by the moving image encoding device 4 according to the present embodiment and decoded by the moving image decoding device 3 according to the present embodiment has substantially the same configuration as the encoded data # 1 in the first embodiment. However, the following points are different.
  • the intra prediction parameter PP_Intra for a partition of a predetermined size or larger includes a main direction designation index MDI that designates one of a plurality of main directions.
  • the encoded data according to the present embodiment may include the additional index AI described in the first embodiment, or may not include the additional index AI.
  • FIG. 18 illustrates a configuration of a leaf CU decoding unit included in the video decoding device according to the present embodiment.
  • Other configurations of the video decoding device according to the present embodiment are the same as the configurations of the video decoding device 1 according to the first embodiment.
  • the PU decoding unit 431 ′ decodes the PU information about the target leaf CU, thereby determining a division pattern for each partition of the target leaf CU and determining a prediction mode for each partition.
  • Mode # 431 is assigned to each partition.
  • the prediction mode # 431 assigned to each partition is supplied to the predicted image generation unit 432 ′.
  • the prediction mode designates a generation method for generating a prediction image by intra prediction (intra prediction).
  • intra prediction intra prediction
  • Any prediction mode is selected from the basic prediction mode set or the edge-based prediction mode set.
  • the PU decoding unit 431 ' refers to the size of the target partition and determines the number of main directions that can be derived for the target partition. In addition, when a plurality of main directions can be derived for the target partition, the PU decoding unit 431 ′ decodes a main direction designation index that designates any of the plurality of main directions. Also, the PU decoding unit 431 'sets the main direction specified by the main direction index as the prediction direction for the target partition.
  • FIG. 19 is a flowchart showing a flow of PU information decoding processing by the PU decoding unit 431 '.
  • FIG. 20 is a table showing the value of the edge-based prediction flag EF, the value of the main direction designation index MDI, the value of the estimation flag MPM, and the value of the residual prediction mode index RIPM that can be taken for the target partition.
  • Step S501 to (Step S506) and (Step S510) Steps S501 to S506 and S510 shown in FIG. 19 are the same as steps S101 to S106 and S112 of the decoding process performed by the PU decoding unit 431 described in the first embodiment, and thus description thereof is omitted.
  • Step S507 When the edge-based prediction mode is applied to the target partition, the PU decoding unit 431 ′ determines the total number of main directions derived for the target partition according to the size of the target partition.
  • the total number of main directions derived for the target partition preferably has a negative correlation with the size of the target partition.
  • the PU decoding unit 431 ′ sets the total number of main directions derived for the target partition to 2, and when the size of the target partition is 8 ⁇ 8 pixels, The total number of main directions derived for the target partition is set to 1. More generally, when the total number of pixels belonging to the target partition is Npix1, the PU decoding unit 431 ′ sets the total number of main directions that can be selected for the target partition to Nmain1, and the total number of pixels belonging to the target partition is When Npix2 (Npix1 ⁇ Npix2), the total number of main directions that can be selected for the target partition is set to Nmain2 (Nmain1 ⁇ Nmain2).
  • the PU decoding unit 431 ′ is the division information included in the encoded data, and includes CU division information SP_CU that specifies a division pattern for each leaf CU of the target LCU including the target partition, and the target PU (target leaf).
  • the size of the target partition can be identified by referring to the intra PU partition information SP_Intra that specifies the partition pattern of each partition of (CU).
  • the PU decoding unit 431 ′ decodes the main direction designation index MDI for the target partition.
  • Step S509 the PU decoding unit 431 ′ sets the prediction mode for the target partition to the edge-based prediction mode in which the main direction specified by the main direction specifying index decoded in step S508 is the prediction direction.
  • the above is the flow of PU information decoding processing by the PU decoding unit 431 '.
  • the prediction mode set for each partition included in the target leaf CU in step S506 and step S509 is supplied to the predicted image generation unit 432 'as a prediction mode # 431'.
  • the PU decoding unit 431 ′ decodes the additional index AI and sets the prediction mode for the target partition in the main direction specified in step S508. Is set to the edge-based prediction mode in which the sub-direction obtained by adding the correction angle specified by the additional index AI is the prediction direction.
  • the prediction image generation unit 432 ′ refers to the prediction image for each partition included in the target leaf CU with reference to the prediction mode # 431 ′ supplied from the PU decoding unit 431 ′ and the decoded pixel values around the partition And generate.
  • FIG. 21 is a flowchart showing the flow of predicted image generation processing for the target leaf CU by the predicted image generation unit 432 '.
  • Step S601 to (Step S605), and (Step S609) Steps S601 to S605 and S609 shown in FIG. 21 are the same as steps S201 to S205 and S212 of the predicted image generation processing by the predicted image generation unit 432 described in the first embodiment, and thus description thereof is omitted. To do.
  • Step S607 the predicted image generation unit 432 ′ sets the derived main direction as the predicted direction for the target partition.
  • Step S608 The predicted image generation unit 432 ′ generates a predicted image for the target partition by extrapolating the decoded pixel values around the target partition along the prediction direction set for the target partition.
  • the processing in this step is the same as that in step S211 described in the first embodiment.
  • the predicted image generation unit 432 ' generates a predicted image for the target leaf CU by performing the above processing. Also, the generated predicted image for the target leaf CU is supplied to the decoded image generation unit 435 as a predicted image # 432 '.
  • the predicted image generation unit 432 ′ adds the correction angle specified by the additional index AI to the main direction specified by the PU decoding unit 431 ′.
  • the predicted image for the target partition is generated by extrapolating the decoded pixel values around the target partition along the prediction direction obtained as described above.
  • the predicted image generation unit 432 ′ sets, as the main direction 1, the direction indicated by the edge vector having the maximum norm among the edge vectors calculated for the pixels belonging to the reference area set in the decoded area around the target partition.
  • the edge vector for each pixel belonging to the reference region can be calculated, for example, by applying a Sobel filter.
  • the predicted image generation unit 432 ′ derives the main direction 2 with reference to the predicted direction assigned to the decoded partitions around the target partition.
  • the predicted image generation unit 432 ′ determines the main direction (or main direction) derived for the plurality of adjacent partitions.
  • the main direction 2 for the target partition is derived by taking the average of the sub-directions obtained by adding the correction directions.
  • the predicted image generation unit 432 ′ determines the main direction derived by the edge-based prediction mode.
  • the main direction 2 for the target partition is set.
  • the prediction mode having a smaller prediction mode index among the basic prediction modes applied to the plurality of adjacent partitions is indicated.
  • the direction is set to the main direction 2 for the target partition.
  • the predicted image generation unit 432 ′ derives the main direction 2 for the target partition by taking the average of the edge vectors calculated for the pixels belonging to the reference region set in the decoded region around the target partition.
  • the predicted image generation unit 432 ′ sets the direction indicated by the edge vector having the maximum norm among the edge vectors calculated for each pixel belonging to the adjacent partition adjacent to the left side of the target partition as the main direction 1 for the target partition. To do.
  • the predicted image generation unit 432 ′ sets the direction indicated by the edge vector having the maximum norm among the edge vectors calculated for each pixel belonging to the adjacent partition adjacent to the upper side of the target partition as the main direction 2 for the target partition. To do.
  • the predicted image generation unit 432 ′ sets, as the main direction 1, the direction indicated by the edge vector having the maximum norm among the edge vectors calculated for the pixels belonging to the reference area set in the decoded area around the target partition. To derive.
  • the predicted image generation unit 432 ′ determines the direction indicated by the edge vector having the second largest norm among the edge vectors calculated for the pixels belonging to the reference region set in the decoded region around the target partition as the main direction 2.
  • the predicted image generation unit 432 ′ classifies each reference pixel belonging to the reference region set around the target partition into two clusters (cluster A and cluster B), and each of the reference pixels belonging to the cluster A Of the edge vectors calculated for the reference pixel, the direction indicated by the edge vector having the maximum norm is set as the main direction 1, and the norm is the maximum among the edge vectors derived for each reference pixel belonging to the cluster B. The direction indicated by the edge vector is set as the main direction 2.
  • the predicted image generation unit 432 ′ calculates an edge vector for each reference pixel belonging to the reference region, and calculates an edge strength of the edge vector.
  • Step S702 Clustering processing is applied to the three-dimensional vector (dx, dy, es) defined for each reference pixel (or a partial vector of the three-dimensional vector), and each reference pixel is classified into either cluster A or cluster B To do.
  • a k-average method, SVM (Support Vector Machine), or the like can be used as a specific example of the clustering process.
  • Step S703 the predicted image generation unit 432 ′ sets the direction indicated by the edge vector having the maximum norm among the edge vectors for each reference pixel classified into cluster A as the main direction 1 for the target partition. Also, the predicted image generation unit 432 ′ sets the direction indicated by the edge vector having the maximum norm among the edge vectors for each reference pixel classified into the cluster B as the main direction 2 for the target partition.
  • condition for determining which of the two clusters is the cluster A is shared by both the video encoding device according to the present embodiment and the video decoding device according to the present embodiment. Preferably it is.
  • FIG. 22 shows a reference area composed of an adjacent partition adjacent to the target partition OP and a partition sharing the upper left vertex of the target partition, and a cluster A and a cluster B into which each reference pixel belonging to the reference area is classified.
  • weak edges in the substantially vertical direction exist in the target partition OP and all areas of the partitions NP1 to NP3. Further, a strong edge in the substantially horizontal direction exists across the target partition OP and the adjacent partition NP3 adjacent to the left side of the target partition.
  • the predicted image generation unit 432 ′ overlaps the reference area composed of the partitions NP1 to NP3 with the cluster A that includes the strong edge and the cluster A as described above.
  • the main direction is set for each cluster with reference to each cluster, so that the main direction in the substantially horizontal direction which is the optimum prediction direction for the target partition OP is divided.
  • a direction can be derived.
  • the reference area can be divided into Nreg cluster areas, and the main direction can be derived for each cluster area.
  • FIG. 23 illustrates a configuration of a leaf CU encoding unit included in the moving image encoding device 4 according to the present embodiment.
  • Other configurations of the video encoding device 4 according to the present embodiment are the same as the configurations of the video encoding device 2 according to the first embodiment.
  • the PU determination unit 531 ′ refers to the leaf CU image # 242a and the CU information # 242b, and (1) the division pattern of the target leaf CU into each partition, and (2) the prediction mode assigned to each partition. decide.
  • a division pattern into the partitions of the target leaf CU for example, a division pattern that is handled as one partition without dividing the target leaf CU, or the target leaf CU is divided into four partitions symmetrically. One of the division patterns can be selected.
  • the PU determination unit 531 determines the total number of main directions that can be selected for the target partition according to the size of the target partition. More specifically, the PU determination unit 531 determines the total number of main directions that can be selected for the target partition so as to have a positive correlation with the size of the target partition.
  • the total number of main directions that can be selected for the target partition is set to 2, and when the size of the target partition is 8 ⁇ 8 pixels, the target partition can be selected.
  • Set the total number of main directions to 1. More generally, when the total number of pixels belonging to the target partition is Npix1, the PU determining unit 531 ′ determines the total number of main directions that can be selected for the target partition as Nmain1, and the total number of pixels belonging to the target partition is When Npix2 (Npix1 ⁇ Npix2), the total number of main directions that can be selected for the target partition is determined as Nmain2 (Nmain1 ⁇ Nmain2).
  • PU information including (1) an intra split flag (intra_split_flag) that specifies whether or not the target leaf CU is split into four partitions, and (2) prediction mode information that specifies the prediction mode assigned to each partition.
  • # 531 ′ is supplied to the predicted image generation unit 532 ′ and the LCU encoded data generation unit 538.
  • the PU information # 531 ′ includes a main direction designation index MDI that designates which of a plurality of main directions is used, the total number of which is determined according to the size of the target partition. It is.
  • the PU information # 531 ' may include a 1-bit flag that specifies whether to use the main direction or the sub direction for the target partition.
  • the additional index AI may include an index that specifies which of the plurality of sub directions is used.
  • the PU determination unit 531 ′ performs each of all combinations of (1) a possible division pattern for each partition of the target leaf CU and (2) a prediction mode that can be assigned to each partition. PU information corresponding to is generated. In addition, generation of a prediction image, a prediction residual, and a decoded image, which will be described later, is performed for each piece of PU information.
  • the PU determination unit 531 ′ supplies the PU information for optimizing the rate distortion supplied from the rate distortion evaluation unit 537 described later to the header information encoding unit 22 as a part of the leaf CU header information # 53.
  • the PU information for optimizing the rate distortion is supplied to the predicted image generation unit 532 ′ and the LCU encoded data generation unit 538 as final PU information for the target leaf CU.
  • the predicted image generation unit 532 ′ refers to the decoded image # 27 stored in the frame memory 27 and the PU information # 531 ′ supplied from the PU determination unit 531 ′, thereby predicting the predicted image # for the target leaf CU. 532 ′ is generated.
  • the prediction image generation process by the prediction image generation unit 532 ′ is the same as that of the prediction image generation unit 432 ′ included in the video decoding device according to the present embodiment, and thus description thereof is omitted here.
  • the above-described moving image encoding device 2 and moving image decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images. The same applies to the moving image encoding device 4 and the moving image decoding device 3, but the following description will be made by taking the moving image encoding device 2 and the moving image decoding device 1 as examples.
  • the moving picture encoding apparatus 2 and the moving picture decoding apparatus 1 described above can be used for transmission and reception of moving pictures.
  • FIG. 25 is a block diagram illustrating a configuration of a transmission device PROD_A in which the moving image encoding device 2 is mounted.
  • the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_A1.
  • a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided.
  • the moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.
  • the transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, and an input terminal PROD_A6 for inputting the moving image from the outside as a supply source of the moving image input to the encoding unit PROD_A1. May be further provided.
  • FIG. 25A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but a part of the configuration may be omitted.
  • the recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.
  • FIG. 25 is a block diagram illustrating a configuration of the receiving device PROD_B in which the moving image decoding device 1 is mounted.
  • the reception device PROD_B includes a reception unit PROD_B1 that receives a modulation signal, a demodulation unit PROD_B2 that obtains encoded data by demodulating the modulation signal received by the reception unit PROD_B1, and a demodulation A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2.
  • the moving picture decoding apparatus 1 described above is used as the decoding unit PROD_B3.
  • the receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3.
  • PROD_B6 may be further provided.
  • FIG. 25B illustrates a configuration in which the reception apparatus PROD_B includes all of these, but a part of the configuration may be omitted.
  • the recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
  • the transmission medium for transmitting the modulation signal may be wireless or wired.
  • the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.
  • a terrestrial digital broadcast broadcasting station (broadcasting equipment or the like) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting.
  • a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.
  • a server workstation etc.
  • Client television receiver, personal computer, smart phone etc.
  • VOD Video On Demand
  • video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication.
  • PROD_A / reception device PROD_B usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN.
  • the personal computer includes a desktop PC, a laptop PC, and a tablet PC.
  • the smartphone also includes a multi-function mobile phone terminal.
  • the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.
  • moving image encoding device 2 and moving image decoding device 1 can be used for recording and reproduction of moving images.
  • FIG. 26 (a) is a block diagram showing a configuration of a recording apparatus PROD_C in which the above-described moving picture encoding apparatus 2 is mounted.
  • the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M.
  • the moving image encoding apparatus 2 described above is used as the encoding unit PROD_C1.
  • the recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disk) or BD (Blu-ray Disk: registration) Or a drive device (not shown) built in the recording device PROD_C.
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • SD memory such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disk) or BD (Blu-ray Disk: registration) Or a drive device (not shown) built in the recording device PROD_C.
  • the recording device PROD_C receives a moving image as a supply source of a moving image to be input to the encoding unit PROD_C1, a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for inputting a moving image from the outside, and a moving image.
  • the receiving unit PROD_C5 may be further provided. In FIG. 26A, a configuration in which all of these are provided in the recording apparatus PROD_C is illustrated, but a part may be omitted.
  • the receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.
  • Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HD (Hard Disk) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main source of moving images).
  • a camcorder in this case, the camera PROD_C3 is a main source of moving images
  • a personal computer in this case, the receiving unit PROD_C5 is a main source of moving images
  • a smartphone in this case, the camera PROD_C3 or The receiving unit PROD_C5 is a main source of moving images
  • the recording apparatus PROD_C is an example of such a recording apparatus PROD_C.
  • FIG. 26 is a block showing a configuration of a playback device PROD_D equipped with the above-described video decoding device 1.
  • the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written to the recording medium PROD_M and a coded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained.
  • the moving picture decoding apparatus 1 described above is used as the decoding unit PROD_D2.
  • the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.
  • the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2.
  • PROD_D5 may be further provided.
  • FIG. 26B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but a part may be omitted.
  • the transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.
  • Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images). .
  • a television receiver in this case, the display PROD_D3 is a main destination of moving images
  • a desktop PC in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is a main destination of moving images
  • Laptop type or tablet type PC in this case, display PROD_D3 or transmission unit PROD_D5 is the main supply destination of moving images
  • smartphone in this case, display PROD_D3 or transmission unit PROD_D5 is the main supply destination of moving images)
  • each block of the above-described moving picture decoding apparatuses 1 and 3 and moving picture encoding apparatuses 2 and 4 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip). Alternatively, it may be realized by software using a CPU (Central Processing Unit).
  • IC chip integrated circuit
  • CPU Central Processing Unit
  • each device includes a CPU that executes instructions of a program that realizes each function, a ROM (Read (Memory) that stores the program, a RAM (Random Memory) that expands the program, the program, and various types
  • a storage device such as a memory for storing data is provided.
  • An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of each of the above devices, which is software that realizes the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying to each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).
  • Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R.
  • IC cards including memory cards
  • semiconductor memories such as mask ROM / EPROM / EEPROM / flash ROM, or PLD (Programmable logic device) or FPGA (Field Programmable Gate Array) Logic circuits can be used.
  • each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
  • the communication network is not particularly limited as long as it can transmit the program code.
  • the Internet intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network (Virtual Private Network), telephone line network, mobile communication network, satellite communication network, etc. can be used.
  • the transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type.
  • wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by wireless such as High Data Rate, NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, and terrestrial digital network.
  • wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by wireless such as High Data Rate, NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, and terrestrial digital network.
  • wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line,
  • the image decoding apparatus includes a prediction mode belonging to a prediction mode group in a prediction residual decoded from encoded data together with prediction mode specification information and other side information, and the prediction mode specification information
  • the number N of prediction modes to be added to the prediction mode group related to the target prediction unit is encoded in the side information
  • a parameter setting unit that refers to a coding unit that includes the target prediction unit or a coding parameter that is decoded prior to the coding unit; and the target prediction unit A prediction parameter corresponding to each of the N prediction modes to be added to the prediction mode group relating to the decoded region of the decoded image. It is characterized in that it comprises a, and deriving means for deriving from.
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually encoded that is decoded before the prediction unit or the encoding unit. It can be determined from the coding parameters for the unit.
  • the coding parameter is a parameter that is referred to in order to generate a decoded image, a locally decoded image, or a predicted image, and is referred to in a motion vector or intra prediction that is referred to in inter-screen prediction.
  • the prediction parameters such as the prediction mode, the size and shape of the partition, the size and shape of the block, the size and shape of the encoding unit, and residual data between the original image and the predicted image are included.
  • the side information refers to a set of all information except for the residual data among the information included in the encoding parameter.
  • the encoding parameter related to the encoding unit including the target prediction unit is related to the encoding parameter associated with the encoding unit and one or a plurality of prediction units included in the encoding unit. Encoding parameters are included.
  • the prediction unit may be a PU (Prediction Unit) described in the embodiment, or may be a partition obtained by dividing the PU.
  • the coding unit may be a maximum coding unit (LCU: Largegest Coding Unit) described in the embodiment, or a coding unit (CU: Coding Unit) obtained by dividing an LCU. Also good.
  • the deriving unit derives a main prediction parameter corresponding to a main prediction mode included in the prediction mode group from the decoded region and adds the main prediction parameter to the prediction mode group. It is preferable that a sub prediction parameter corresponding to each of the N sub prediction modes and having a specific relationship with the main prediction parameter is derived from the main prediction parameter.
  • the prediction accuracy is improved by using the prediction mode group including the main prediction mode. Can be achieved.
  • the sub prediction parameters corresponding to each of the N sub prediction modes to be added to the prediction mode group, the sub prediction parameters having a specific relationship with the main prediction parameter are converted into the main prediction parameters. Since it derives from the prediction parameters, the prediction accuracy can be further improved.
  • the total number of sub-prediction modes can be determined with reference to a coding unit that includes the target prediction unit or a coding parameter related to a coding unit that is decoded prior to the coding unit. It is possible to prevent a decrease in coding efficiency due to an increase in the amount of code that can be caused by excessively adding the sub prediction mode.
  • the main prediction parameter represents the edge direction ⁇ m in the target prediction unit estimated from the pixel value of the decoded region
  • the main prediction mode is the value of the decoded region.
  • This is a prediction mode for generating a prediction image on the target prediction unit by extrapolating the pixel value from the extrapolation direction represented by the main prediction parameter, and the N sub-prediction parameters have ⁇ m as the edge direction.
  • the N sub-prediction modes exclude the pixel values of the decoded region from the extrapolation direction represented by each of the N sub-prediction parameters. It is preferable that it is a prediction mode which produces
  • the target prediction unit when the edge exists in the encoding target image can be generated with high prediction accuracy.
  • the sub prediction mode can be used, so high prediction accuracy can be achieved. Can keep.
  • the total number of sub-prediction modes can be determined with reference to a coding unit that includes the target prediction unit or a coding parameter related to a coding unit that is decoded prior to the coding unit. It is possible to prevent a decrease in coding efficiency due to an increase in the amount of code that can be caused by excessively adding the sub prediction mode.
  • the prediction mode group includes a basic prediction mode in which a prediction parameter is predetermined in addition to the main prediction mode and the sub prediction mode, and the derivation is performed.
  • the means is configured such that the prediction mode designated by the prediction mode designation information relating to the adjacent prediction unit adjacent to the target prediction unit, the interval ⁇ in the extrapolation direction represented by each of the main prediction parameter and the sub prediction parameter is the main prediction mode. Or it is preferable to adjust to the value according to whether it is the said sub prediction mode or the said basic prediction mode.
  • the prediction unit in which the main prediction mode is selected tends to have a relatively strong edge.
  • such strong edges often exist continuously across a plurality of prediction units. Therefore, the main prediction parameter derived using the main prediction mode for the adjacent prediction unit tends to be close to the optimal prediction parameter for the target prediction unit.
  • the extrapolation direction interval ⁇ represented by each of the main prediction parameter and the sub prediction parameter for the target prediction unit is the main prediction mode, the sub prediction mode, or the basic prediction mode.
  • the side information includes an encoding parameter that defines a size of the target prediction unit, and the setting unit adds the number N of prediction modes to be added to the prediction mode group. Is preferably set to a value according to the size of the target prediction unit defined by the encoding parameter.
  • the code amount of side information necessary for encoding an encoding target image for a certain encoding unit has a positive correlation with the total number of prediction units included in the encoding unit. That is, if the total number of prediction units is large, the code amount of the side information increases.
  • the total number of prediction units included in a certain coding unit has a negative correlation with the size of each prediction unit.
  • the said setting means is high by setting the prediction mode number N added to the said prediction mode group to the value according to the size of the said object prediction unit prescribed
  • the side information includes an encoding parameter that defines the number M of basic prediction modes
  • the prediction mode group includes M basic prediction modes.
  • the setting means preferably sets the number N of prediction modes to be added to the prediction mode group to a value corresponding to the number M of basic prediction modes defined by the encoding parameter.
  • the prediction unit around the prediction unit in which the total number of selectable basic prediction modes is set to be large is that the coding efficiency is improved by increasing the number of prediction modes belonging to the prediction mode group and increasing the prediction accuracy. Tend.
  • the prediction mode group includes a basic prediction mode in which a prediction parameter is predetermined in addition to the main prediction mode and the sub prediction mode, and the setting The means is that the prediction mode designated by the prediction mode designation information related to the adjacent prediction unit adjacent to the target prediction unit for the number N of prediction modes to be added to the prediction mode group is the main prediction mode or the sub prediction mode. It is preferable to set the value according to whether the basic prediction mode is selected.
  • the prediction unit in which the main prediction mode is selected tends to have a relatively strong edge.
  • such strong edges often exist continuously across a plurality of prediction units. Therefore, the main prediction parameter derived using the main prediction mode for the adjacent prediction unit tends to be close to the optimal prediction parameter for the target prediction unit.
  • designated information regarding the adjacent prediction unit adjacent to the said prediction unit for the prediction mode number N added to the said prediction mode group is the said main prediction mode or the said sub prediction mode.
  • the deriving unit derives a main prediction parameter corresponding to each of N main prediction modes to be added to the prediction mode group from the decoded region. .
  • the deriving unit divides the decoded region into N regions and derives the main prediction mode from each of the obtained N regions.
  • an encoding target image is often composed of a plurality of regions having different image directivities.
  • the derivation unit divides the decoded region into N regions, and derives the main prediction mode from each of the obtained N regions, so that the decoded images are mutually imaged. Even if it is comprised from several area
  • the image coding apparatus designates the selected prediction mode based on the prediction residual obtained by subtracting the predicted image generated according to the prediction mode selected from the prediction mode group for each prediction unit from the original image.
  • the number N of prediction modes to be added to the prediction mode group related to the target prediction unit is encoded in the side information.
  • a parameter setting unit that refers to a coding unit that includes the target prediction unit or a coding parameter that is encoded prior to the coding unit; and the target prediction Deriving prediction parameters corresponding to each of the N prediction modes to be added to the prediction mode group related to the unit from the decoded region of the locally decoded image It is characterized in that comprises a derivation means that, the.
  • the prediction unit is a type that increases the prediction accuracy by increasing the number of prediction modes belonging to the prediction mode group, and the coding efficiency is improved by reducing the number of prediction modes belonging to the prediction mode group and improving the coding efficiency. Some types improve efficiency. According to the knowledge obtained by the inventors, whether a certain prediction unit belongs to the former type or the latter type is usually a code encoded before the prediction unit or the encoding unit. It can be determined from the encoding parameters for the encoding unit.
  • the encoding efficiency is increased by referring to the encoding parameter including the target prediction unit or the encoding parameter related to the encoding unit encoded before the encoding unit.
  • the number of prediction modes to be added to the prediction mode group related to the target prediction unit can be set. Therefore, according to said structure, encoding efficiency can be improved by suppressing the increase in the code amount of coding data, maintaining high prediction accuracy.
  • the coding parameter is a parameter that is referred to in order to generate a decoded image, a locally decoded image, or a predicted image, and is referred to in a motion vector or intra prediction that is referred to in inter-screen prediction.
  • the prediction parameters such as the prediction mode, the size and shape of the partition, the size and shape of the block, the size and shape of the encoding unit, and residual data between the original image and the predicted image are included.
  • the side information refers to a set of all information except for the residual data among the information included in the encoding parameter.
  • the encoding parameter related to the encoding unit including the target prediction unit is related to the encoding parameter associated with the encoding unit and one or a plurality of prediction units included in the encoding unit. Encoding parameters are included.
  • the prediction unit may be a PU (Prediction Unit) described in the embodiment, or may be a partition obtained by dividing the PU.
  • the coding unit may be a maximum coding unit (LCU: Largegest Coding Unit) described in the embodiment, or a coding unit (CU: Coding Unit) obtained by dividing an LCU. Also good.
  • the data structure of the encoded data according to the present invention includes a prediction residual obtained by subtracting, from the original image, a prediction image generated according to a prediction mode selected from a prediction mode group for each prediction unit.
  • the side information includes the number of prediction modes to be added to the prediction mode group related to the target prediction unit, which is a data structure of encoded data generated by encoding together with the prediction mode specifying information to be specified and other side information. Coding parameters that are expressed implicitly by a coding unit that includes the target prediction unit or a coding parameter that is coded before the coding unit. It is characterized by.
  • the present invention can be suitably applied to a decoding device that decodes encoded data and an encoding device that generates encoded data. Further, the present invention can be suitably applied to the data structure of encoded data generated by the encoding device and referenced by the decoding device.
  • Video decoding device 11 Variable length code demultiplexing unit 12 Header information decoding unit 13 LCU setting unit 14 LCU decoding unit (setting means, deriving means) 15 Frame memory 2, 4 Video encoding device 21 LCU header information determination unit 22 Header information encoding unit 23 LCU setting unit 24 LCU encoding unit (setting unit, deriving unit) 25 Variable length code multiplexing unit 26 LCU decoding unit 27 Frame memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon l'invention, une unité de décodage LCU (14), dont est équipé un dispositif de décodage d'image animée (1), règle le nombre N de modes de prédiction qui sont ajoutés au groupe de modes de prédiction concernant une unité de prédiction cible par référence à une unité de codage qui contient l'unité de prédiction cible susmentionnée ou par référence à un paramètre de codage concernant une unité de codage qui a été décodée avant ladite unité de codage et, sur la base des régions de l'image décodée qui ont été précédemment décodées, obtient les paramètres de prédiction correspondant à chacun des N modes de prédiction qui ont été ajoutés au groupe de modes de prédiction concernant l'unité de prédiction cible.
PCT/JP2011/072283 2010-09-30 2011-09-28 Dispositif de décodage d'image, dispositif de codage d'image et structure de données WO2012043678A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010-222234 2010-09-30
JP2010222234 2010-09-30

Publications (1)

Publication Number Publication Date
WO2012043678A1 true WO2012043678A1 (fr) 2012-04-05

Family

ID=45893117

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/072283 WO2012043678A1 (fr) 2010-09-30 2011-09-28 Dispositif de décodage d'image, dispositif de codage d'image et structure de données

Country Status (1)

Country Link
WO (1) WO2012043678A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009111691A (ja) * 2007-10-30 2009-05-21 Hitachi Ltd 画像符号化装置及び符号化方法、画像復号化装置及び復号化方法
WO2009090884A1 (fr) * 2008-01-18 2009-07-23 Panasonic Corporation Procédé de codage d'image et procédé de décodage d'image
JP2010056701A (ja) * 2008-08-27 2010-03-11 Nippon Telegr & Teleph Corp <Ntt> 画面内予測符号化方法，画面内予測復号方法，これらの装置，およびそれらのプログラム並びにプログラムを記録した記録媒体

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009111691A (ja) * 2007-10-30 2009-05-21 Hitachi Ltd 画像符号化装置及び符号化方法、画像復号化装置及び復号化方法
WO2009090884A1 (fr) * 2008-01-18 2009-07-23 Panasonic Corporation Procédé de codage d'image et procédé de décodage d'image
JP2010056701A (ja) * 2008-08-27 2010-03-11 Nippon Telegr & Teleph Corp <Ntt> 画面内予測符号化方法，画面内予測復号方法，これらの装置，およびそれらのプログラム並びにプログラムを記録した記録媒体

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
EHSAN MAANI ET AL.: "Differential Coding of Intra Modes (DCIM)", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-B109, 2ND MEETING, July 2010 (2010-07-01), GENEVA, CH, pages 1 - 7 *
TEST MODEL UNDER CONSIDERATION: "Output Document (draft000)", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-B205, 2ND MEETING, July 2010 (2010-07-01), GENEVA, CH, pages 1 - 14,53-67,132-136 *
TOMOYUKI YAMAMOTO ET AL.: "Flexible Representation of Intra Prediction Modes", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-B063, 2ND MEETING, July 2010 (2010-07-01), GENEVA, CH, pages 1 - 7 *

Similar Documents

Publication Publication Date Title
US11627337B2 (en) Image decoding device
JP7200320B2 (ja) 画像フィルタ装置、フィルタ方法および動画像復号装置
US10547861B2 (en) Image decoding device
WO2017068856A1 (fr) Dispositif de génération d&#39;image prédictive, dispositif de décodage d&#39;image et dispositif de codage d&#39;image
JP5551274B2 (ja) 画像フィルタ装置
JP6114787B2 (ja) 復号装置及び符号化装置
WO2015199040A1 (fr) Unité de prédiction de dmm, dispositif de décodage d&#39;image et dispositif de codage d&#39;image
WO2017195532A1 (fr) Dispositif de décodage d&#39;image et dispositif de codage d&#39;image
JP5995448B2 (ja) 画像復号装置、および画像符号化装置
WO2012077719A1 (fr) Dispositif de décodage d&#39;images et dispositif de codage d&#39;images
TW202133613A (zh) 圖像解碼裝置及圖像編碼裝置
WO2013046990A1 (fr) Appareil de décodage de décalage, appareil de codage de décalage, appareil de filtre d&#39;image et structure de données
WO2018110281A1 (fr) Dispositif de prédiction de code, dispositif de décodage d&#39;image et dispositif de codage d&#39;image
US20210006836A1 (en) Image encoding apparatus, image encoding method, image decoding apparatus, and image decoding method
JP2013192118A (ja) 算術復号装置、画像復号装置、算術符号化装置、および画像符号化装置
WO2012077795A1 (fr) Dispositif de codage d&#39;images, dispositif de décodage d&#39;images et structure de données
WO2012090962A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image, structure de données des données codées, dispositif de décodage arithmétique et dispositif de codage arithmétique
WO2012043678A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image et structure de données
JP6162289B2 (ja) 画像復号装置および画像復号方法
WO2012043766A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image et structure de données pour données codées
WO2012043676A1 (fr) Dispositif de décodage, dispositif de codage et structure de données
JP2016143962A (ja) 領域分割画像生成装置、画像復号装置、および符号化装置。
JP2016072941A (ja) Ｄｍｍ予測装置、画像復号装置、および画像符号化装置
WO2012081706A1 (fr) Dispositif de filtre d&#39;image, dispositif de filtre, décodeur, codeur et structure de données
JP2013251827A (ja) 画像フィルタ装置、画像復号装置、画像符号化装置、およびデータ構造

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11829231

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11829231

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP