US20210243475A1 - Image processing apparatus and method - Google Patents

Image processing apparatus and method Download PDF

Info

Publication number
US20210243475A1
US20210243475A1 US16/468,451 US201716468451A US2021243475A1 US 20210243475 A1 US20210243475 A1 US 20210243475A1 US 201716468451 A US201716468451 A US 201716468451A US 2021243475 A1 US2021243475 A1 US 2021243475A1
Authority
US
United States
Prior art keywords
transform
flag
chrominance
primary
inverse
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/468,451
Other languages
English (en)
Inventor
Takeshi Tsukuba
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TSUKUBA, TAKESHI
Publication of US20210243475A1 publication Critical patent/US20210243475A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present disclosure relates to an image processing apparatus and method, and particularly to an image processing apparatus and method that can curb deterioration in coding efficiency.
  • an adaptive primary transform for adaptively selecting a primary transform from a plurality of different orthogonal transforms for each of a primary transform in a horizontal direction PThor (which is also referred to as a primary horizontal transform) and a primary transform in a vertical direction PTver (which is also referred to as a primary vertical transform) of each transform unit (TU) for luminance is described (e.g., refer to Non-Patent Literature 1).
  • AMT Adaptive Multiple Core Transforms
  • Non-Patent Literature 1 Jianle Chen, pupils Alshina, Gary J. Sullivan, Jens-Rainer, and Jill Boyce, “Algorithm Description of Joint Exploration Test Model 4” JVET-D1001_v3, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 4 th Meeting at Chengdu, China. Oct. 15 to 21, 2016
  • Non-Patent Literature 1 Although the adaptive primary transform can be applied only to luminance (Y), it is not applied to chrominances (Cb and Cr). Thus, there is concern of the coding efficiency of the primary transform for the chrominances deteriorating more in comparison to that for luminance.
  • the present disclosure takes the above circumstances into consideration and aims to curb deterioration in coding efficiency.
  • An image processing apparatus of a first aspect of the present technology is an image processing apparatus including: an inverse orthogonal transform unit configured to perform an inverse orthogonal transform of chrominance using information regarding the inverse orthogonal transform of chrominance derived on the basis of information regarding an inverse orthogonal transform of luminance.
  • the information regarding the inverse orthogonal transform can include an adaptive primary transform flag indicating whether an adaptive inverse primary transform of adaptively selecting one from a plurality of different inverse orthogonal transforms and using the selected transform as an inverse primary transform is to be applied.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance.
  • the information regarding the inverse orthogonal transform can include a primary transform identifier indicating which inverse primary transform is to be applied to inverse primary transforms in a vertical direction and a horizontal direction.
  • a value of the primary transform identifier of chrominance can be set to a value of the primary transform identifier of luminance in a case in which the adaptive primary transform flag of chrominance is true, and can be set to a predetermined value in a case in which the adaptive primary transform flag of chrominance is false.
  • the inverse orthogonal transform unit can perform the inverse primary transform as the inverse orthogonal transform.
  • the inverse orthogonal transform unit can perform an inverse primary horizontal transform which is the inverse primary transform in the horizontal direction and an inverse primary vertical transform which is the inverse primary transform in the vertical direction as the inverse orthogonal transform.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction or a case in which the prediction type is intra prediction of which a prediction mode is intra block copy.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction or a case in which the prediction type is intra prediction in which prediction modes for luminance and chrominance match each other.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance in a case in which a chrominance adaptive primary transform information inference flag indicating whether the adaptive primary transform flag of chrominance is to be inferred on the basis of the adaptive primary transform flag of luminance is true.
  • a value of the adaptive primary transform flag of chrominance can be set to a value of the adaptive primary transform flag of luminance in a case in which a size of a short side of a transform block to be processed of chrominance is greater than or equal to a predetermined threshold value.
  • the inverse orthogonal transform unit can be configured to perform an inverse primary horizontal transform which is an inverse primary transform in a horizontal direction and an inverse primary vertical transform which is an inverse primary transform in a vertical direction as the inverse orthogonal transform, a transform type of the inverse primary horizontal transform can be set on the basis of a horizontal transform set and a primary horizontal transform specifying flag in a case in which a size of a picture width of a transform block to be processed of chrominance is greater than a predetermined threshold value, and a transform type of the inverse primary vertical transform can be set on the basis of a vertical transform set and a primary vertical transform specifying flag in a case in which a size of a picture height of the transform block to be processed of chrominance is greater than a predetermined threshold value.
  • the information regarding the inverse orthogonal transform can include a transform skip flag indicating whether an inverse orthogonal transform process is to be skipped.
  • a value of the transform skip flag of chrominance can be set to a value of the transform skip flag of luminance.
  • the information regarding the inverse orthogonal transform can include a secondary transform identifier indicating which inverse secondary transform is to be applied.
  • a value of the secondary transform identifier of chrominance can be set to a value of the secondary transform identifier of luminance.
  • An image processing method of the first aspect of the present technology is an image processing method including: performing an inverse orthogonal transform of chrominance using information regarding the inverse orthogonal transform of chrominance derived on the basis of information regarding an inverse orthogonal transform of luminance.
  • An image processing apparatus of a second aspect of the present technology is an image processing apparatus including: an orthogonal transform unit configured to perform an orthogonal transform of chrominance using information regarding the orthogonal transform of chrominance derived on the basis of information regarding an orthogonal transform of luminance.
  • An image processing method of the second aspect of the present technology is an image processing method including: performing an orthogonal transform of chrominance using information regarding the orthogonal transform of chrominance derived on the basis of information regarding an orthogonal transform of luminance.
  • an inverse orthogonal transform of chrominance is performed using information regarding the inverse orthogonal transform of chrominance derived on the basis of information regarding an inverse orthogonal transform of luminance.
  • an orthogonal transform of chrominance is performed using information regarding the orthogonal transform of chrominance derived on the basis of information regarding an orthogonal transform of luminance.
  • images can be processed. Particularly, deterioration in coding efficiency can be curbed.
  • FIG. 1 is a diagram illustrating a correspondence relation between transform sets and selected orthogonal transforms.
  • FIG. 2 is a diagram illustrating a correspondence relation between types of orthogonal transform and functions to be used.
  • FIG. 3 is a diagram illustrating a correspondence relation between transform sets and prediction modes.
  • FIG. 4 is a diagram illustrating an example of syntax and semantics of a transform unit.
  • FIG. 5 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 6 is a diagram illustrating examples of parameters of chrominance derived using values of luminance.
  • FIG. 7 is a diagram illustrating an example of a derivation method of adaptive primary transform flags of chrominance.
  • FIG. 8 is a diagram illustrating shapes of a CU, a PU, and a TU.
  • FIG. 9 is a block diagram illustrating an example of the main configuration of an image decoding apparatus.
  • FIG. 10 is a block diagram illustrating an example of the main configuration of an inverse transform unit.
  • FIG. 11 is a block diagram illustrating an example of the main configuration of a chrominance adaptive primary transform information derivation unit.
  • FIG. 12 is a flowchart describing an example of the flow of an image decoding process.
  • FIG. 13 is a flowchart describing an example of the flow of a primary transform information decoding process.
  • FIG. 14 is a flowchart describing an example of the flow of an inverse transform process.
  • FIG. 15 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 16 is a flowchart describing an example of the flow of an inverse primary transform selection process.
  • FIG. 17 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 18 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 19 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 20 is a diagram illustrating an example of syntax and semantics of a transform unit.
  • FIG. 21 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 22 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information inference flag decoding process.
  • FIG. 23 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 24 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information derivation process.
  • FIG. 25 is a flowchart describing an example of the flow of a primary horizontal transform type derivation process.
  • FIG. 26 is a flowchart describing an example of the flow of a primary vertical transform type derivation process.
  • FIG. 27 is a diagram illustrating an example of a derivation method for a transform skip flag of a chrominance.
  • FIG. 28 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 29 is a flowchart describing an example of the flow of a transform skip flag derivation process.
  • FIG. 30 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 31 is a flowchart describing an example of the flow of a transform skip flag derivation process.
  • FIG. 32 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 33 is a flowchart describing an example of the flow of a transform skip flag derivation process.
  • FIG. 34 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 35 is a flowchart describing an example of the flow of a transform skip flag derivation process.
  • FIG. 36 is a diagram illustrating an example of syntax and semantics of a transform unit.
  • FIG. 37 is a diagram illustrating an example of syntax of residual_coding.
  • FIG. 38 is a flowchart describing an example of the flow of a transform skip flag derivation process.
  • FIG. 39 is a diagram illustrating an example of a secondary transform identifier derivation method of a chrominance.
  • FIG. 40 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 41 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 42 is a flowchart describing an example of the flow of a secondary transform identifier derivation process for a chrominance.
  • FIG. 43 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 44 is a flowchart describing an example of the flow of a secondary transform identifier derivation process for a chrominance.
  • FIG. 45 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 46 is a flowchart describing an example of the flow of a secondary transform identifier derivation process for a chrominance.
  • FIG. 47 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 48 is a flowchart describing an example of the flow of a secondary transform identifier derivation process for a chrominance.
  • FIG. 49 is a diagram illustrating an example of syntax of a transform unit.
  • FIG. 50 is a flowchart describing an example of the flow of a secondary transform identifier derivation process for a chrominance.
  • FIG. 51 is a block diagram illustrating an example of the main configuration of an image encoding apparatus.
  • FIG. 52 is a block diagram illustrating an example of the main configuration of a transform unit.
  • FIG. 53 is a block diagram illustrating an example of the main configuration of a chrominance adaptive primary transform information derivation unit.
  • FIG. 54 is a flowchart describing an example of the flow of an image decoding process.
  • FIG. 55 is a flowchart describing an example of the flow of a transform process.
  • FIG. 56 is a flowchart describing an example of the flow of a primary transform selection process.
  • FIG. 57 is a flowchart describing an example of the flow of a primary transform information encoding process.
  • FIG. 58 is a flowchart describing an example of the flow of a chrominance adaptive primary transform information inference flag encoding process.
  • FIG. 59 is a flowchart describing an example of the flow of a transform skip flag encoding process.
  • FIG. 60 is a flowchart describing an example of the flow of a transform skip flag encoding process.
  • FIG. 61 is a flowchart describing an example of the flow of a transform skip flag encoding process.
  • FIG. 62 is a flowchart describing an example of the flow of a transform skip flag encoding process.
  • FIG. 63 is a flowchart describing an example of the flow of a transform skip flag encoding process.
  • FIG. 64 is a flowchart describing an example of the flow of a secondary transform identifier encoding process for a chrominance.
  • FIG. 65 is a flowchart describing an example of the flow of a secondary transform identifier encoding process for a chrominance.
  • FIG. 66 is a flowchart describing an example of the flow of a secondary transform identifier encoding process for a chrominance.
  • FIG. 67 is a flowchart describing an example of the flow of a secondary transform identifier encoding process for a chrominance.
  • FIG. 68 is a flowchart describing an example of the flow of a secondary transform identifier encoding process for a chrominance.
  • FIG. 69 is a block diagram illustrating a principal configuration example of a computer.
  • FIG. 70 is a block diagram illustrating an example of a schematic configuration of a television apparatus.
  • FIG. 71 is a block diagram illustrating an example of a schematic configuration of a mobile telephone.
  • FIG. 72 is a block diagram illustrating an example of a schematic configuration of a recording/reproducing apparatus.
  • FIG. 73 is a block diagram illustrating an example of a schematic configuration of an imaging apparatus.
  • FIG. 74 is a block diagram illustrating one example of a schematic configuration of a video set.
  • FIG. 75 is a block diagram illustrating one example of a schematic configuration of a video processor.
  • FIG. 76 is a block diagram illustrating another example of a schematic configuration of a video processor.
  • FIG. 77 is a block diagram illustrating one example of a schematic configuration of a network system.
  • Non-Patent Literature 1 Joint Exploration Test Model 4 (JEM 4)
  • JEM 4 Joint Exploration Test Model 4
  • the maximum size of a coding tree unit (CTU) is expanded from 128 ⁇ 128 to 256 ⁇ 256.
  • CTU coding tree unit
  • binary trees in the horizontal/vertical directions have been introduced, and accordingly, rectangular transform blocks have also been introduced in addition to square transform blocks.
  • an adaptive primary transform (Adaptive Multiple Core Transforms (AMT)) of adaptively selecting a primary transform from a plurality of different orthogonal transforms for each primary transform in a horizontal direction PThor (which is also referred to as a primary horizontal transform) and a primary transform PTver in a vertical direction (which is also referred to as a primary vertical transform) of each transform block for luminance is disclosed.
  • AMT Adaptive Multiple Core Transforms
  • an adaptive primary transform flag apt_flag (which is also referred to as amt_flag, cu_pt_flag, or emt_flag) indicating whether or not an adaptive primary transform for luminance is to be executed in the unit of a transform block of luminance (which is also referred to as a luminance transform block) is 0 (false)
  • a discrete cosine transform (DCT)-II or a discrete sine transform (DST)-VII is (uniquely) decided using mode information as an orthogonal transform applied to a horizontal primary transform and a vertical primary transform.
  • an adaptive primary transform flag apt_flag of luminance is 1 (true) as in the table (LUT_TrSetToTrTypIdx) illustrated in FIG. 1
  • the DST-VII, the DCT-VIII, and the like illustrated in FIG. 1 indicate types of orthogonal transform, and each of the functions shown in the table of FIG. 2 is used for them.
  • identifiers corresponding to orthogonal transforms of respective types are defined. For example, if a type of orthogonal transform (Transform Type) is DCT-II, 0 is allocated as a value of the transform type identifier TrTypeIdx.
  • Selection (decision) of a transform set TransformSet varies depending on a prediction type of a coding unit (CU: which is also referred to as a unit of coding) to which a transform block to be processed belongs. For example, if the prediction type is intra prediction, selection is made on the basis of intra prediction mode (IntraPredMode) as illustrated in the table of FIG. 3 (LUT_IntraModeToTrSet). For example, selection is made to set a transform set identifier TrSetIdx for designating a transform set TrSet corresponding to a transform set (TrSetH or TrSetV) of each direction as illustrated in the following formulas (1) and (2).
  • IntraPredMode intra prediction mode
  • TrSetIdx for designating a transform set TrSet corresponding to a transform set (TrSetH or TrSetV) of each direction as illustrated in the following formulas (1) and (2).
  • TrSetH represents a transform set of a primary horizontal transform PThor (also referred to as a primary horizontal transform set)
  • TrSetV represents a transform set of a primary vertical transform PTver (also referred to as a primary vertical transform set).
  • a lookup table LUT_IntraModeToTrSet represents the correspondence table of FIG. 3 .
  • the value of InterTrSet is 3 in the case of the table of FIG. 1 .
  • TrSet H InterTrSetIdx (3)
  • TrSet V InterTrSetIdx (4)
  • orthogonal transform in the selected transform set TrSet is selected using a primary horizontal transform specifying flag pt_hor_flag.
  • which orthogonal transform in the selected transform set TrSet is to be applied to the primary vertical transform is selected using a primary vertical transform specifying flag pt_ver_flag.
  • these are derived from the definition table of transform sets (LUT_TrSetToTrTypeIdx) illustrated in FIG. 1 using the primary ⁇ horizontal, vertical ⁇ transform set TrSet ⁇ H,V ⁇ , and the primary ⁇ horizontal, vertical ⁇ transform specifying flag pt_ ⁇ hor, ver ⁇ _flag as arguments as illustrated in the following formulas (5) and (6).
  • TrTypeIdx H LUT_TrSetToTrTypeIdx[TrSet H ][pt_hor_flag] (5)
  • TrTypeIdx V LUT_TrSetToTrTypeIdx[TrSet V ][pt_ver_flag] (6)
  • the primary horizontal transform specifying flag pt_hor_flag is 0, the value “4” of the transform type identifier TrTypeIdx indicating DST-VII is set for the horizontal transform type identifier TrTypeIdxH designating the type of the orthogonal transform of the primary horizontal transform PThor as illustrated in FIG. 1 , and in a case in which the primary horizontal transform specifying flag pt_hor_flag is 1, the value “1” of the transform type identifier TrTypeIdx indicating DCT-V is set for the horizontal transform type identifier TrTypeIdxH.
  • a primary transform identifier pt_idx is derived from the primary horizontal transform specifying flag pt_hor_flag and the primary vertical transform specifying flag pt_ver_flag using the following formula (7).
  • pt_idx (pt_ver_flag ⁇ 1)+pt_hor_flag (7)
  • the upper 1 bit of the primary transform identifier pt_idx corresponds to the primary vertical transform specifying flag and the lower 1 bit corresponds to the value of the primary horizontal transform specifying flag.
  • Encoding is performed by applying arithmetic coding to a bin string of the derived primary transform identifier pt_idx and generating a bit string. Note that the upper 1 bit of the pt_idx may be set as the primary horizontal transform specifying flag and the lower 1 bit may be set as the primary vertical transform specifying flag.
  • an adaptive primary transform is not applied to transform blocks of chrominance (Cb and Cr), and DCT-II is selected at all times as a type of orthogonal transform such as the primary horizontal transform and the primary vertical transform.
  • a of FIG. 4 is an example of a syntax table of a transform unit TU, and B of FIG. 4 shows an example of semantics corresponding thereto.
  • the residual data presence flag cbf[x0][y0][compID](coded_block_flag) shown in the row denoted by reference symbol SYN11 is a flag indicating whether or not there are one or more non-zero coefficients in a transform block of a color signal designated with a color signal identifier compID as illustrated in B of FIG. 4 .
  • the adaptive primary transform enabled flag apt_enabled_flag shown in A of FIG. 4 is information regarding permission for an adaptive primary transform as illustrated in B of FIG. 4 .
  • the value of the flag is 1 (true)
  • it indicates that information regarding the adaptive primary transform is present in the encoded data.
  • the value of the flag is 0 (false)
  • it indicates that no information regarding the adaptive primary transform is present in the encoded data.
  • the adaptive primary transform flag apt_flag of the luminance transform block shown in the row denoted by reference symbol SYN12 in the syntax shown in A of FIG. 4 is a flag indicating whether or not the adaptive primary transform is to be applied to the transform block of luminance as illustrated in B of FIG. 4 .
  • the value of the flag is 1 (true)
  • the adaptive primary transform is applied
  • the value of the flag is 0 (false)
  • the application primary transform is not applied.
  • the adaptive primary transform flag apt_flag of luminance is encoded (decoded). In a case in which there is no apt_flag in encoded data, the value of apt_flag is interpreted to be 0.
  • the residual data residual_coding ( ) of the transform block of the color signal designated with the color signal identifier compID shown in the row denoted by reference symbol SYN13 is encoded (decoded) in a case in which the residual data presence flag of the corresponding color signal is 1 (true).
  • FIG. 5 illustrates an example of syntax of residual_coding.
  • maxPTSize a predetermined threshold value
  • Non-Patent Literature 1 although the adaptive primary transform can be applied only to luminance (Y), it is not applied to chrominances (Cb and Cr). Thus, there is concern of the coding efficiency of the primary transform for the chrominances deteriorating more in comparison to that for luminance.
  • an adaptive primary transform flag apt_flag and a primary transform identifier pt_idx for each transform block of a chrominance (Cb or Cr), similarly to luminance (Y).
  • a primary transform flag apt_flag and a primary transform identifier pt_idx for a chrominance is encoded, there is concern of the code amount increasing and coding efficiency deteriorating.
  • the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx should be decoded for each of the color signals (Y, Cb, and Cr), and thus there is concern of the processing amount increasing.
  • information regarding an (inverse) orthogonal transform for a chrominance is derived on the basis of information regarding an (inverse) orthogonal transform for luminance. That is, using the information regarding the (inverse) orthogonal transform for a chrominance derived on the basis of the (inverse) orthogonal transform for luminance, the (inverse) orthogonal transform for a chrominance is performed. For example, using the information regarding the (inverse) orthogonal transform for a chrominance derived on the basis of the (inverse) orthogonal transform for luminance, an (inverse) orthogonal transform unit that performs the (inverse) orthogonal transform for a chrominance is provided in an image processing apparatus.
  • an orthogonal transform and an inverse orthogonal transform are inverse processes with respect to each other, and it is assumed that data that has not been orthogonally transformed can be restored by, for example, performing an inverse orthogonal transform on the orthogonally transformed data.
  • information regarding an orthogonal transform is information to be used in the orthogonal transform
  • information regarding an inverse orthogonal transform is information to be used in the inverse orthogonal transform
  • information regarding the information can be used in the orthogonal transform there also is a case in which the information can be used in the orthogonal transform. That is, in the present specification, there is a case in which information regarding an orthogonal transform and information regarding the inverse orthogonal transform refer to the same information (the information regarding both transforms may include the same information). The same also applies to the relations between a primary transform and the inverse primary transform, and a secondary transform and the inverse secondary transform.
  • the information may include an adaptive primary transform flag apt_flag indicating whether an adaptive (inverse) primary transform to be used as an (inverse) primary transform by adaptively selecting any of a plurality of different inverse orthogonal transforms is applied to a transform block to be processed.
  • the information may also include a primary transform identifier pt_idx indicating which (inverse) primary transform is to be applied to the (inverse) primary transforms in the vertical direction and the horizontal direction.
  • a primary transform identifier of the chrominance (Cb) pt_idx[Cb] and a primary transform identifier of the chrominance (Cr) pt_idx[Cr] on the basis of a primary transform identifier of luminance pt_idx[Y]
  • a primary transform identifier of a chrominance may be common for (shared between) Cb and Cr.
  • an adaptive (inverse) primary transform is not employed for a chrominance, and an adaptive primary transform flag apt_flag and a primary transform identifier pt_idx for the chrominance are omitted at all times as indicated in the row of No. #0. Since no adaptive (inverse) primary transform is used in such cases, there is concern of coding efficiency deteriorating as described above.
  • an adaptive (inverse) primary transform may be set to be applicable to the chrominance, and further an adaptive primary transform flag apt_flag and a primary transform identifier pt_idx for the chrominance can be inferred from an adaptive primary transform flag apt_flag and a primary transform identifier pt_idx for luminance at all times as indicated in the row of No. #1.
  • a value of an adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to a value of an adaptive primary transform flag apt_flag [Y] of luminance.
  • a value of a primary transform identifier pt_idx[Cb/Cr] of a chrominance may be set to a value of a primary transform identifier pt_idx[Y] of luminance, and in a case in which the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance is false, a value of the primary transform identifier pt_idx[Cb/Cr] of the chrominance may be set to a predetermined value.
  • the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to the value of the adaptive primary transform flag apt_flag [Y] of luminance
  • the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance may be set to 0 (false) as indicated in the row of No. #2.
  • the adaptive primary transform flag of luminance can be used only in the case of the inter prediction mode in which tendencies of residual signals are similar, and thus it is possible to curb deterioration in coding efficiency.
  • the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to the value of the adaptive primary transform flag apt_flag [Y] of luminance as indicated in the row of No.
  • the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance may be set to 0 (false). In this manner, not only can the adaptive (inverse) primary transform be applied to the chrominance, but also the adaptive primary transform flag of luminance can be used only in the case of the prediction mode in which tendencies of residual signals are similar, and thus it is possible to curb deterioration in coding efficiency.
  • the adaptive (inverse) primary transform is set to be applicable to the chrominance as well and further, for example, the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy as indicated in the row of No.
  • the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to the value of the adaptive primary transform flag apt_flag [Y] of luminance, and in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy (IntraBC; also referred to as “in-screen motion compensation), the value of the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance may be set to 0 (false).
  • the adaptive primary transform flag of luminance can be used only in the case of the prediction mode in which tendencies of residual signals are similar, and thus it is possible to curb deterioration in coding efficiency.
  • the adaptive (inverse) primary transform is set to be applicable to the chrominance as well and further, for example, a chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag indicating whether the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance is inferred on the basis of the adaptive primary transform flag apt_flag [Y] of luminance is 1 (true) as indicated in the row of No.
  • the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance may be set to the value of the adaptive primary transform flag apt_flag [Y] of luminance, and in a case in which the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag is 0 (false), the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to 0 (false).
  • the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to the value of the adaptive primary transform flag apt_flag [Y] of luminance, and in a case in which the size of the short side of the transform block is smaller than the threshold value, the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be set to 0 (false).
  • the adaptive (inverse) primary transform is set to be applicable to the chrominance as well and further, for example, the size of the picture width of a transform block with the chrominance to be processed is equal to or smaller than a predetermined threshold value as indicated in the row of No. #7, the transform type of the (inverse) primary horizontal transform may be set to a predetermined transform type, and in a case in which the size of the picture width of the transform block is greater than the threshold value, the transform type of the (inverse) primary horizontal transform may be set on the basis of a horizontal transform set and a primary horizontal transform specifying flag.
  • the transform type of the (inverse) primary vertical transform may be set to a predetermined transform type, and in a case in which the size of the picture height of the transform block is greater than the threshold value, the transform type of the (inverse) primary vertical transform may be set on the basis of a vertical transform set and a primary vertical transform specifying flag.
  • each of the cases described above may be combined with row of No. #7, like the case in the row of No. #8. In this manner, effects obtained in each of the cases can be exhibited.
  • each of the above-described cases can also be combined with another case that is not described above. For example, in a case in which the prediction type of a coding block to be processed is intra prediction in the case of the row of No. #2, the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance may be transmitted (signaled).
  • the adaptive primary transform identifier pt_idx[Cb/Cr] of a chrominance is further transmitted (signaled). That is, encoding and decoding of the adaptive primary transform identifier pt_idx[Cb/Cr] of a chrominance are performed in this case.
  • Shapes of a coding unit (CU), a prediction unit (PU), and a TU which are partial regions (processing units) of a moving image in a bit stream (encoded data) will be described here.
  • a tree structure in a quad tree (Quad-Tree) shape or a binary tree (Binary-Tree) shape in a horizontal direction or a vertical direction is consequently formed.
  • a size of a largest coding unit is 128 ⁇ 128, there is a possibility of a size of a CU (a size in a horizontal direction w ⁇ a size in a vertical direction h) being a size of a square such as 128 ⁇ 128, 64 ⁇ 64, 32 ⁇ 32, 16 ⁇ 16, 8 ⁇ 8, or 4 ⁇ 4 as well as a size of a rectangle such as 128 ⁇ 64, 128 ⁇ 32, 128 ⁇ 16, 128 ⁇ 8, 128 ⁇ 4, 64 ⁇ 128, 32 ⁇ 128, 16 ⁇ 128, 8 ⁇ 128, 4 ⁇ 128, 64 ⁇ 32, 64 ⁇ 16, 64 ⁇ 8, 64 ⁇ 4, 32 ⁇ 64, 16 ⁇ 64, 8 ⁇ 64, 4 ⁇ 64, 32 ⁇ 16, 32 ⁇ 8, 32 ⁇ 4, 16 ⁇ 32, 8 ⁇ 32, 4 ⁇ 32, 16 ⁇ 8, 16 ⁇ 4, 8 ⁇ 16, 4 ⁇ 16, 8 ⁇ 4, or 4 ⁇ 8 as
  • a TU includes a transform block of luminance (Y) and a transform block of a chrominance (Cb/Cr).
  • Y luminance
  • Cb/Cr transform block of a chrominance
  • a color format 4:2:0 (e.g., YUV420)
  • the proportion of a picture size of a chrominance to a picture size of luminance with respect to a picture height and a picture width is 1 ⁇ 2.
  • the size of a transform block of luminance is 8 ⁇ 4
  • the size of a corresponding transform block of a chrominance is 4 ⁇ 2.
  • the proportion of a picture size of a chrominance to a picture size of luminance is 1 ⁇ 2 with respect to a picture height and 1 with respect to a picture width.
  • the size of a transform block of luminance is 8 ⁇ 4
  • the size of a transform block for a corresponding chrominance is 8 ⁇ 2.
  • the proportion of the picture size of a chrominance to the picture size of luminance is 1 with respect to the picture height and 1 with respect to the picture width.
  • the size of a transform block of luminance is 8 ⁇ 4
  • the size of a transform block of a corresponding chrominance is 8 ⁇ 4.
  • luminance (Y) and chrominance (Cb/Cr) may be encoded as different CUs. Since the luminance and chrominance can have different CU division structures in this case, an effect of improving coding efficiency of I-slices is exhibited.
  • description will be provided below on the assumption that information of luminance and chrominance is included in the same CU for the sake of convenience, the invention is not limited thereto.
  • FIG. 9 is a block diagram illustrating an example of a configuration of an image decoding apparatus which is an aspect of an image processing apparatus to which the present technology has been applied.
  • the image decoding apparatus 100 illustrated in FIG. 9 is an apparatus that decodes encoded data obtained by encoding a predictive residual of an image and a predictive image as in the AVC or HEVC.
  • the image decoding apparatus 100 implements, for example, a technology proposed by HEVC or the Joint Video Exploration Team (VET).
  • the image decoding apparatus 100 has a decoding unit 111 , an inverse quantization unit 112 , an inverse transform unit 113 , an arithmetic operation unit 114 , a frame memory 115 , and a prediction unit 116 .
  • the prediction unit 116 has an intra prediction unit and an inter prediction unit which are not illustrated.
  • the image decoding apparatus 100 is an apparatus for generating a moving image #2 by decoding encoded data #1 (a bit stream).
  • the decoding unit 111 receives an input of the encoded data #1 and performs variable length decoding on a syntax value of each of syntax elements from a bit string of the encoded data #1 in accordance with a decision of a syntax table. Furthermore, each syntax element includes information such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, and residual information Rinfo.
  • Header information Hinfo such as VPS/SPS/PPS/slice header SH includes information that defines image sizes (a picture width PicWidth and a picture height PicHeight), bit depths (luminance bitDepthY and chrominance bitDepthC), a maximum value of a CU size MaxCUSize/a minimum value thereof.
  • MinCUSize a maximum depth of quadtree split (also referred to as quad-tree split) MaxQTDepth/a minimum depth thereof.
  • MinQTDepth a maximum depth of binary tree split (binary-tree split) MaxBTDepth/a minimum depth thereof.
  • MinBTDepth a maximum value of a transform skip block MaxTSSize (also referred to as a maximum transform skip block size), an on/off flag of each encoding tool (also referred to as an enabled flag), and the like.
  • an on/off flag of an encoding tool included in the header information Hinfo for example, an on/off flag for transform and quantization processes which will be introduced below.
  • the on/off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax for the encoding tool is present in encoded data.
  • a value of the on/off flag is 1 (true)
  • it indicates that the encoding tool is available in a case in which a value of the on/off flag is 0 (false)
  • An adaptive primary transform enabled flag apt_enabled_flag (also referred to as adaptive_primary_transform_enabled_flag, adaptive_pt_enabled_flag, of amt_enabled_flag) is a flag indicating whether an encoding tool that can select an adaptive primary transform (also referred to as an adaptive primary transform) as one of a transform process and an inverse process thereto is available.
  • a secondary transform enabled flag st_enabled_flag is a flag indicating whether an encoding tool that performs a secondary transform/an inverse secondary transform as one of a transform process and an inverse process thereto is available or not.
  • a transform quantization bypass enabled flag transquant_bypass_enabled_flag is a flag indicating whether an encoding tool that skips a transform and a quantization/an inverse quantization and an inverse transform as one of a transform/quantization and inverse processes thereto is available or not.
  • a transform skip flag enabled flag ts_enabled_flag is a flag indicating whether or not two-dimensional transform skip or one-dimensional transform skip is available as one of a transform process and an inverse process thereto.
  • the two-dimensional transform skip is an encoding tool that skips an orthogonal transform and an inverse process (inverse orthogonal transform) thereto including a primary transform and a secondary transform.
  • the one-dimensional transform skip is an encoding tool that skips a primary transform in a horizontal direction or a vertical direction and an inverse transform (inverse primary transform) corresponding thereto among primary transforms, and a secondary transform and an inverse secondary transform thereto.
  • IPinfo e.g., prev_intra_luma_pred_flag, mpm_i
  • the transform information Tinfo may include syntax of, for example, a picture width size TBWidth and a picture height size TBHeight of a transform block to be processed, a transform quantization bypass flag transquant_bypass_flag indicating whether or not (inverse) transform and (inverse) quantization processes are to be skipped, a transform skip flag ts_flag indicating whether or not two-dimensional transform skip is to be applied, an adaptive primary transform flag apt_flag indicating whether an adaptive primary transform is to be applied to a target transform block, a primary transform identifier pt_idx indicating which (inverse) primary transform is to be applied between(inverse) primary transforms in the vertical direction and the horizontal direction, a secondary transform identifier st_idx (also referred to as dnsst_idx, nsst_idx, or rot_idx) indicating which(inverse) secondary transform is to be applied, a scan identifier scanIdx, a quantization bypass
  • the residual information Rinfo may include, for example, a residual data presence/absence flag (cbf (coded_block_flag)), a last non-zero coefficient X coordinate (last_sig_coeff_x_pos), a last non-zero coefficient Y coordinate (last_sig_coeffy_pos), a sub-block non-zero coefficient presence/absence flag (coded_sub_block_flag), a non-zero coefficient presence/absence flag (sig_coeff_flag), a flag indicating whether the level of a non-zero coefficient is greater than 1 (gr1_flag) (also referred to as a GR1 flag), a flag indicating whether the level of a non-zero coefficient is greater than 2 flag (gr2_flag) (also be referred to as a GR2 flag), a sign indicating the positivity/negativity of a non-zero coefficient (sign_flag) (also referred to as a sign symbol), the remaining level of
  • the decoding unit 111 derives a quantized transform coefficient level level of each coefficient position in each transform block with reference to residual information Rinfo.
  • the decoding unit 111 supplies prediction mode information Pinfo, quantized transform coefficient level level, and transform information Tinfo obtained from decoding to each block.
  • the decoding unit 111 supplies the prediction mode information Pinfo to the inverse transform unit 113 and the prediction unit 116 , the quantized transform coefficient level level to the inverse quantization unit 112 , and the transform information Tinfo to the inverse transform unit 113 and the inverse quantization unit 112 .
  • the inverse quantization unit 112 receives an input of the transform information Tinfo and the quantized transform coefficient level level, performs scaling (inverse quantization) on the value of the quantized transform coefficient level level on the basis of the transform information Tinfo, and outputs the transform coefficient that has undergone inverse quantization Coeff_IQ to the inverse transform unit 113 .
  • the inverse transform unit 113 receives an input of the transform coefficient Coeff_IQ, the transform information Tinfo, and the prediction mode information Pinfo, applies an inverse transform on the transform coefficient Coeff_IQ on the basis of the transform information Tinfo, the prediction mode information Pinfo, and the like, derives a predictive residual D′, and outputs the predictive residual to the arithmetic operation unit 114 .
  • the inverse transform is the inverse process of the transform process performed on the encoding side or the like.
  • an inverse orthogonal transform such as an inverse process to an orthogonal transform performed on an encoding side or the like is included in the inverse transform.
  • a primary transform or a secondary transform is performed as an orthogonal transform on the encoding side
  • an inverse primary transform which is an inverse process to the primary transform an inverse secondary transform which is an inverse process to the secondary transform, or the like is included in the inverse transform. Details of the inverse transform unit 113 will be described below.
  • the arithmetic operation unit 114 receives an input of the predictive residual D′ and a predictive image P supplied from the prediction unit 116 , adds the predictive residual D′ to the predictive image P (predictive signal) corresponding to the predictive residual D′ as illustrated in the following formula (8), derives a local decoded image Rec, and supplies the local decoded image to the frame memory 115 or the outside of the image decoding apparatus 100 .
  • the frame memory 115 receives an input of the local decoded image Rec supplied from the arithmetic operation unit 114 , reconstructs a decoded image in each picture unit, and then stores the decoded image in the frame memory 115 .
  • the frame memory 115 reads the decoded image specified by the prediction mode information Pinfo of the prediction unit 116 as a reference image from the buffer, and supplies the decoded image to the prediction unit 116 .
  • the frame memory 115 may store the header information Hinfo, the prediction mode information Pinfo, the transform information Tinfo, and the like relating to the generation of the decoded image in the buffer inside the frame memory.
  • the prediction unit 116 receives an input of the prediction mode information Pinfo, generates the predictive image P using, as a reference image, the decoded image stored in the frame memory 115 specified by the prediction mode information PInfo using a prediction method specified by the prediction mode information Pinfo, and outputs the predictive image to the arithmetic operation unit 114 .
  • FIG. 10 is a block diagram illustrating an example of the main configuration of the inverse transform unit 113 of the image decoding apparatus 100 of FIG. 9 .
  • the inverse transform unit 113 includes a switch 121 , an inverse secondary transform unit 122 , and an inverse primary transform unit 123 as illustrated in FIG. 9 .
  • the switch 121 skips the inverse secondary transform unit 122 and the inverse primary transform unit 123 , and outputs the transform coefficient Coeff_IQ as predictive residual D′.
  • the inverse secondary transform unit 122 performs a process relating to an inverse secondary transform that is the inverse process to a secondary transform which is a predetermined transform process, for example, an orthogonal transform, or the like.
  • the inverse secondary transform unit 122 receives an input of a secondary transform identifier st_idx, a scan identifier scanIdx indicating a scanning method of a transform coefficient and the transform coefficient Coeff_IQ, derives a transform coefficient Coeff_IS (also referred to as a primary transform coefficient Coeff_P) that has undergone an inverse secondary transform, and supplies the transform coefficient to the inverse primary transform unit 123 .
  • the inverse secondary transform unit 122 executes the process of the inverse secondary transform corresponding to the secondary transform identifier st_idx on the transform coefficient Coeff_IQ, and outputs the transform coefficient Coeff_IS that has undergone the inverse secondary transform.
  • the inverse secondary transform unit 122 skips the inverse secondary transform and outputs the transform coefficient Coeff_IQ as the transform coefficient Coeff_IS that has undergone the inverse secondary transform.
  • the inverse primary transform unit 123 performs a process relating to an inverse primary transform that is the inverse process to a primary transform that is a predetermined transform process, for example, an orthogonal transform.
  • the inverse primary transform unit 123 receives inputs of a color signal identifier compID, adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance, a primary transform identifier of luminance pt_idx[COMPONENT_Y], prediction mode information Pinfo, sizes of a transform block (a logarithmic value of the picture width log 2TBWSize and a logarithmic value of the picture height log 2TBHSize), and the transform coefficient Coeff_IS that has undergone the inverse secondary transform.
  • a color signal identifier compID adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance
  • a primary transform identifier of luminance pt_idx[COMPONENT_Y] prediction mode information Pinfo
  • the inverse primary transform unit 123 selects the type of the inverse primary horizontal transform TrTypeIdxH and the type of the inverse primary vertical transform TrTypeIdxV of a color signal specified by the color signal identifier compID with reference to the prediction mode information PInfo, the color signal identifier compID, the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance, the primary transform identifier of luminance pt_idx[COMPONENT_Y], and primary transform identifier of luminance pt_idx[COMPONENT_Y], performs an inverse primary vertical transform defined by the inverse primary vertical transform type TrTypeIdxV and the picture height log 2TBHSize of the transform block and the inverse primary horizontal transform defined by the inverse primary horizontal transform type TrTypeIdxH and the picture width log 2TBWSize of the transform block on the transform coefficient Coeff_IS that has undergone the inverse secondary, and then derives and outputs the predictive residual D′ that has undergone the inverse
  • the inverse primary transform unit 123 has a chrominance adaptive primary transform information derivation unit 131 , an inverse primary transform selection unit 132 , an inverse primary vertical transform unit 133 , and an inverse primary horizontal transform unit 134 as illustrated in FIG. 11 .
  • FIG. 11 is a function block diagram illustrating an example of the main configuration of functions of the chrominance adaptive primary transform information derivation unit 131 .
  • the chrominance adaptive primary transform information derivation unit 131 has an apt_flag derivation unit 151 and a pt_idx derivation unit 152 as illustrated in FIG. 11 .
  • the apt_flag derivation unit 151 performs a process relating to derivation of the adaptive primary transform flag apt_flag of a chrominance.
  • the apt_flag derivation unit 151 may derive the adaptive primary transform flag apt_flag[compID] of the chrominance as illustrated in the following formula (10) with reference to a residual data presence flag cbf_luma of luminance.
  • apt_flag[compID] cbf_luma ?apt_flag[COMPONENT_ Y ]: 0 (10)
  • the pt_idx derivation unit 152 performs a process relating to the derivation of a primary transform identifier pt_idx of a chrominance.
  • the pt_idx derivation unit 152 further sets the primary transform identifier pt_idx[compID] of the chrominance transform block to the value of the primary transform identifier for the luminance transform block pt_idx[COMPONENT_Y] corresponding to the chrominance transform block. In other cases, a predetermined value is set.
  • the chrominance adaptive primary transform information derivation unit 131 can derive the adaptive primary transform information of the chrominance transform block on the basis of the adaptive primary transform information of luminance corresponding to the chrominance transform block. Therefore, it is possible to reduce the decoding processing amount without decoding the adaptive primary transform information of the chrominance transform block from the encoded data.
  • the inverse primary transform selection unit 132 receives an input of the prediction mode information PInfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • the inverse primary transform selection unit 132 derives the transform type TrTypeIdxH of the inverse primary horizontal transform and the transform type TrTypeIdxV of the inverse primary vertical transform of the color signal specified by the color signal identifier compID with reference to the prediction mode information PInfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID, and supplies the results to the inverse primary horizontal transform unit 134 and the inverse primary vertical transform unit 133 .
  • the inverse primary vertical transform unit 133 receives an input of the transform coefficient Coeff_IS that has undergone the inverse secondary transform, the transform type TrTypeIdxV of the inverse primary vertical transform, and the log 2TBHSize (the logarithmic value of the picture height) indicating the picture height of the transform block for each transform block of each color signal.
  • the inverse primary vertical transform unit 133 executes an inverse primary vertical transform IPver defined by the transform type TrTypeIdxV and the picture height of the transform block on the transform coefficient Coeff_IS that has undergone the inverse secondary transform and supplies the result to the inverse primary horizontal transform unit 134 as a transform coefficient Coeff_IPver that has undergone the inverse primary vertical transform.
  • the inverse primary vertical transform IPver is the inverse transform to a primary vertical transform Pver.
  • the inverse primary horizontal transform unit 134 receives inputs of the transform coefficient Coeff_IPver that has undergone the inverse primary vertical transform, the transform type TrTypeIdxH of the inverse primary horizontal transform, and log 2TBWSize (the logarithmic value of the picture width) indicating the picture height of the transform block for each transform block of each color signal.
  • the inverse primary horizontal transform unit 134 executes an inverse primary horizontal transform IPhor defined by the transform type TrTypeIdxH and the picture width of the transform block on the transform coefficient Coeff_IPver that has undergone the inverse primary vertical transform, and outputs the result to the outside of the inverse transform unit 113 as the predictive residual D′ (supplies the result to the arithmetic operation unit 114 ).
  • the inverse primary horizontal transform IPhor is the inverse transform of a primary horizontal transform Phor.
  • the adaptive primary transform flag apt_flag[compID] is 0 (false)
  • a predetermined e.g., DCT-II
  • CuPredMode is intra prediction
  • setting is made on the basis of an intra prediction mode (IntraPredMode) as in the table (LUT_IntraModeToTrSet) shown in FIG. 3 .
  • IntraPredMode intra prediction mode
  • TrSetIdx specifying a corresponding transform set TrSet
  • TrSetV transform set in each of the directions
  • TrSetH represents a transform set of a primary horizontal transform PThor (referred to as a primary horizontal transform set)
  • TrSetV represents a transform set of a primary vertical transform PTver (also referred to as a primary vertical transform set).
  • a lookup table LUT_IntraModeToTrSet is the correspondence table of FIG. 3 .
  • a transform set identifier dedicated to inter prediction may be allocated. Since inter prediction and intra block copy tend to have similar residual in general, it is reasonable to allocate the same transform set.
  • a transform set identifier dedicated to intra block copy may be allocated.
  • the primary transform identifier pt_idx[compID] has a 2-bit value, the upper 1 bit thereof corresponds to the primary vertical transform specifying flag pt_ver_flag, and the lower 1 bit thereof corresponds to the primary horizontal transform specifying flag pt_hor_flag.
  • the inverse primary transform selection unit 132 selects the transform type of the orthogonal transform to be used in the inverse primary transform from the transform sets TrSetH and TrSetV selected in each of horizontal/vertical direction using each of the primary horizontal transform specifying flag pt_hor_flag and the primary vertical transform specifying flag pt_ver_flag.
  • the inverse primary transform selection unit 132 determines the transform type TrTypeIdxH of the orthogonal transform to be applied to the inverse primary horizontal transform on the basis of the correspondence table for the transform sets and the transform types (LUT_TrSetToTrTypeIdx) illustrated in FIG. 1 , the primary horizontal transform set TrSetH, and the primary horizontal transform specifying flag pt_hor_flag as illustrated in the following formula (16).
  • TrTypeIdx H LUT_TrSetToTrTypeIdx[TrSet H ][pt_hor_flag] (16)
  • the inverse primary transform selection unit 132 determines the transform type TrTypeIdxV of the orthogonal transform to be applied to the inverse primary vertical transform on the basis of the correspondence table for the transform sets and the transform types (LUT_TrSetToTrTypeIdx) illustrated in FIG. 1 , the primary vertical transform set TrSetVx, and the primary vertical transform specifying flag pt_ver_flag as illustrated in the following formula (17).
  • TrTypeIdx V LUT_TrSetToTrTypeIdx[TrSet V ][pt_ver_flag] (17)
  • a transform type determined with the values of the transform set identifier TrSetIdx of the table shown in FIG. 1 , the primary horizontal transform specifying flag, and the primary vertical transform specifying flag may be set to be changeable within an executable range.
  • the inverse primary transform selection unit 132 supplies the transform type TrTypeIdxH of the inverse primary horizontal transform IPThor and the transform type TrTypeIdxV of the inverse primary vertical transform IPTver of the color signal specified by the color signal identifier compID to each of the inverse primary vertical transform unit 133 and the inverse primary horizontal transform unit 134 .
  • the inverse primary transform unit 123 can apply an inverse adaptive primary transform selected for luminance to a transform block of the chrominance in a case in which a residual signal of the color signal shows a similar tendency to a residual signal of luminance.
  • an inverse primary transform process on the residual signal of the chrominance with higher coding efficiency than in the related art.
  • the decoding unit 111 decodes a bit stream (encoded data) supplied to the image decoding apparatus 100 and thereby obtains information such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and a quantized transform coefficient level level in Step S 101 .
  • Step S 102 the inverse quantization unit 112 performs inverse quantization on the quantized transform coefficient level level obtained from the process of Step S 101 and thus derives a transform coefficient Coeff_IQ.
  • the inverse quantization is the inverse process to quantization performed in an image encoding process, which will be described below, and is a similar process to inverse quantization performed in the image encoding process.
  • Step S 103 the inverse transform unit 113 performs an inverse transform on the transform coefficient Coeff_IQ obtained from the process of Step S 102 and thus derives predictive residual D′.
  • the inverse transform is the inverse process to a transform process performed in the image encoding process, which will be described below, and is a similar process to an inverse transform performed in the image encoding process.
  • Step S 104 the prediction unit 116 performs prediction in the same prediction mode as the prediction performed at the time of encoding on the basis of the prediction mode information Pinfo and generates a predictive image.
  • Step S 105 the arithmetic operation unit 114 adds the predictive image obtained from the process of Step S 104 to the predictive residual D′ obtained from the process of Step S 103 and thus obtains a decoded image.
  • Step S 106 the arithmetic operation unit 114 outputs the decoded image obtained from the process of Step S 105 to the outside of the image decoding apparatus 100 .
  • Step S 107 the frame memory 115 stores the decoded image obtained from the process of Step S 105 .
  • Step S 107 When the process of Step S 107 ends, the image decoding process ends.
  • the decoding unit 111 also appropriately decodes information such as the adaptive primary transform flag apt_flag, the primary transform identifier pt_idx, and the like.
  • the decoding unit 111 decodes, for example, the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance and the primary transform identifier pt_idx[COMPONENT_Y] of luminance.
  • the decoding unit 111 omits decoding of the information of the elements.
  • the decoding unit 111 executes a primary transform information decoding process in Step S 101 .
  • a primary transform information decoding process An example of the flow of the primary transform information decoding process will be described with reference to the flowchart of FIG. 13 .
  • Step S 112 the decoding unit 111 can decode the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance included in the encoded data #1.
  • Step S 113 the decoding unit 111 can decode the primary transform identifier pt_idx[COMPONENT_Y] of luminance included in the encoded data #1.
  • the switch 121 determines whether the transform skip flag ts_flag is 2D_TS (a mode of two-dimensional transform skip) or the transform quantization bypass flag transquant_bypass_flag is 1 (true) in Step S 121 .
  • the switch 121 In a case in which it is determined that the transform skip identifier ts_idx is 2D_TS or the transform quantization bypass flag is 1 (true), the switch 121 outputs the transform coefficient Coeff_IQ to the outside as predictive residual D′ (supplied to the arithmetic operation unit 114 ), the inverse transform process ends and the process returns to FIG. 12 .
  • Step S 121 supplies the transform coefficient Coeff_IQ to the inverse secondary transform unit 122 , and the process proceeds to Step S 122 .
  • Step S 122 the inverse secondary transform unit 122 performs the inverse secondary transform on the input transform coefficient Coeff_IQ on the basis of the secondary transform identifier st_idx, and derives and outputs the transform coefficient Coeff_IS that has undergone the inverse secondary transform.
  • Step S 124 ends, the process proceeds to Step S 125 .
  • Step S 125 the inverse primary transform selection unit 132 derives the transform type TrTypeIdxH of the inverse primary horizontal transform and the transform type TrTypeIdxV of the inverse primary vertical transform of the color signal specified by the color signal identifier compID with reference to the prediction mode information Pinfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • Step S 126 the inverse primary vertical transform unit 133 executes the inverse primary vertical transform IPver defined by the transform type TrTypeIdxV of the inverse primary vertical transform and the picture height of the transform block on the transform coefficient Coeff_IS that has undergone the inverse secondary transform of the transform block for each transform block specified by the color signal identifier compID and outputs the result as a transform coefficient Coeff_IPver that has undergone the inverse primary vertical transform.
  • Step S 127 the inverse primary horizontal transform unit 134 executes the inverse primary horizontal transform IPhor defined by the transform type TrTypeIdxH of the inverse primary horizontal transform and the picture height of the transform block on the transform coefficient Coeff_IPver that has undergone the inverse primary vertical transform of the transform block for each transform block specified by the color signal identifier compID and outputs the result as predictive residual D′.
  • the inverse transform process ends, and the process returns to FIG. 12 .
  • the apt_flag derivation unit 151 of the chrominance adaptive primary transform information derivation unit 131 sets a value of the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance for a value of the adaptive primary transform flag apt_flag[compID] of the chrominance in Step S 131 .
  • Step S 132 the pt_idx derivation unit 152 of the chrominance adaptive primary transform information derivation unit 131 determines whether or not the adaptive primary transform flag apt_flag[compID] of the chrominance is true. When it is determined to be true, the process proceeds to Step S 133 .
  • Step S 133 the pt_idx derivation unit 152 sets a value of the primary transform identifier pt_idx[COMPONENT_Y] of luminance for the primary transform identifier pt_idx[compID] of the chrominance.
  • the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • Step S 134 the pt_idx derivation unit 152 sets a predetermined value for the primary transform identifier pt_idx[compID] of the chrominance.
  • Step S 125 of FIG. 14 Next, an example of the flow of the inverse primary transform selection process executed in Step S 125 of FIG. 14 will be described with reference to the flowchart of FIG. 16 .
  • the inverse primary transform selection unit 132 determines whether or not the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID is 1 (true) in Step S 141 . In a case in which the adaptive primary transform flag apt_flag[compID] is determined to be 1 (true), the process proceeds to Step S 142 .
  • Step S 142 the inverse primary transform selection unit 132 selects a transform set TrSetV (primary horizontal transform set) of the inverse primary vertical transform and a transform set TrSetH (primary vertical transform set) of the inverse primary horizontal transform on the basis of the prediction mode information PInfo.
  • TrSetV primary horizontal transform set
  • TrSetH primary vertical transform set
  • Step S 143 the inverse primary transform selection unit 132 derives a primary horizontal transform specifying flag pt_hor_flag and a primary vertical transform specifying flag pt_ver_flag from the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • Step S 144 the inverse primary transform selection unit 132 selects the transform type TrTypeIdxH of an orthogonal transform applied as the inverse primary horizontal transform IPThor with reference to the primary horizontal transform set TrSetH and the primary horizontal transform specifying flag pt_hor_flag.
  • Step S 145 the inverse primary transform selection unit 132 selects the transform type TrTypeIdxV of an orthogonal transform applied as the inverse primary vertical transform IPTver with reference to the primary vertical transform set TrSetV and the primary vertical transform specifying flag pt_ver_flag.
  • the inverse primary transform selection process ends, and the process returns to FIG. 14 .
  • Step S 141 in a case in which it is determined that the adaptive primary transform flag apt_flag[compID] is 0 (false), the process proceeds to Step S 146 .
  • the inverse primary transform selection unit 132 derives the transform type TrTypeIdxH of the inverse primary horizontal transform IPThor and the transform type TrTypeIdxV of the inverse primary vertical transform IPTver using the method in accordance with the value of the adaptive primary transform flag apt_flag[compID] of the chrominance.
  • the inverse transform unit 113 included in the image decoding apparatus 100 can apply the inverse adaptive primary transform selected for luminance to the transform block of the chrominance in a case in which a residual signal of a chrominance shows a similar tendency to a residual signal of luminance. Therefore, the inverse primary transform process can be performed on the residual signal of the chrominance with further improved coding efficiency than in the related art. In addition, the processing amounts of encoding and decoding can be reduced while mitigating deterioration in coding efficiency for the chrominance in comparison to the case in which the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx are explicitly decoded for each of luminance and the chrominance.
  • the derivation of the adaptive primary transform flag of the chrominance may be limited to a case in which the prediction type is an inter CU.
  • the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set on the basis of the adaptive primary transform flag apt_flag[COMPONENT_Y] of a transform block of luminance corresponding to the transform block of the chrominance.
  • the value of the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set to 0 (false).
  • the flowchart corresponds to the flowchart of FIG. 15 .
  • the apt_flag derivation unit 151 determines whether or not the prediction type CUPredMode of a CU to which a transform block to be processed belongs is inter prediction (MODE_INTER) in Step S 151 In a case in which the prediction type is determined to be inter prediction, the process proceeds to Step S 152 .
  • Step S 152 the apt_flag derivation unit 151 sets a value of the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance for the value of the adaptive primary transform flag apt_flag[compID] of the chrominance as in the case of Step S 131 of FIG. 15 .
  • Step S 152 ends, the process proceeds to Step S 154 .
  • Step S 153 the apt_flag derivation unit 151 sets the adaptive primary transform flag apt_flag[compID] of the chrominance to 0 (false).
  • Step S 154 the process proceeds to Step S 154 .
  • Step S 154 to Step S 156 is executed similarly to each of the processes of Step S 132 to Step S 134 of FIG. 15 .
  • the process of Step S 155 or Step S 156 ends, the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • the prediction type is intra prediction
  • intra prediction modes are different between luminance and the chrominance.
  • the residual signals of the luminance and the chrominance show different tendencies
  • the adaptive primary transform information of the luminance is applied to the chrominance, there is a possibility of coding efficiency deteriorating. Therefore, by applying the adaptive primary transform information of the luminance to the chrominance only in the case in which the chrominance adaptive primary transform information derivation process is executed as illustrated in FIG. 17 and the prediction type is inter prediction, a decrease in coding efficiency can be further curbed than in the case of FIG. 15 .
  • the derivation of the adaptive primary transform flag of the chrominance may be limited to an inter CU or an intra CU and when a luminance intra prediction mode is intra block copy (in-screen motion compensation).
  • the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set on the basis of the adaptive primary transform flag apt_flag[COMPONENT_Y] with respect to the transform block of luminance corresponding to the transform block of the chrominance.
  • the value of the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set to 0 (false).
  • the conditional determination may be performed using the intra prediction mode IntraPredModeY of luminance in the above-described formula (19).
  • the flowchart corresponds to the flowchart of FIG. 15 .
  • the apt_flag derivation unit 151 determines whether or not the prediction type CUPredMode of a CU to which a transform block to be processed belongs is inter prediction (MODE_INTER) in Step S 171 . In a case in which the prediction type is determined to be intra prediction, the process proceeds to Step S 172 .
  • Step S 172 the apt_flag derivation unit 151 determines whether or not the intra prediction mode IntraPredModeC of the chrominance is intra block copy (IntraBC). In a case in which the mode is determined to be intra block copy (IntraBC), the process proceeds to Step S 173 . In addition, the prediction type CUPredMode of the CU to which the transform block to be processed belongs is determined to be inter prediction (MODE_INTER) in Step S 171 , the process proceeds to Step S 173 .
  • the prediction type CUPredMode of the CU to which the transform block to be processed belongs is determined to be inter prediction (MODE_INTER) in Step S 171 , the process proceeds to Step S 173 .
  • Step S 173 the apt_flag derivation unit 151 sets a value of the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance for the value of the adaptive primary transform flag apt_flag[compID] of the chrominance as in the case of Step S 131 of FIG. 15 .
  • Step S 173 ends, the process proceeds to Step S 175 .
  • Step S 174 the apt_flag derivation unit 151 sets the value of the adaptive primary transform flag apt_flag[compID] of the chrominance to 0 (false).
  • Step S 175 the process proceeds to Step S 175 .
  • Step S 175 to Step S 177 is executed similarly to each of the processes of Step S 132 to Step S 134 of FIG. 15 .
  • the process of Step S 176 or Step S 177 ends, the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • the derivation of the adaptive primary transform flag of the chrominance may be limited to an inter CU or an intra CU and when a luminance intra prediction mode are equivalent to a chrominance intra prediction mode.
  • the adaptive primary transform flag apt_flag[compID] of the transform block of the chrominance is set on the basis of the adaptive primary transform flag apt_flag[COMPONENT_Y] with respect to the transform block of luminance corresponding to the transform block of the chrominance.
  • the value of the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set to 0 (false).
  • the flowchart corresponds to the flowchart of FIG. 15 .
  • the apt_flag derivation unit 151 determines whether or not the prediction type CUPredMode of a CU to which a transform block to be processed belongs is inter prediction (MODE_INTER) in Step S 181 . In a case in which the prediction type is determined to be intra prediction, the process proceeds to Step S 182 .
  • Step S 182 the apt_flag derivation unit 151 determines whether or not the intra prediction mode IntraPredModeC of the chrominance is equal to the intra prediction mode IntraPredModeY of luminance. In a case in which the modes are equal, the process proceeds to Step S 183 . In addition, the intra prediction mode IntraPredModeC of the chrominance is determined not to be equal to the intra prediction mode IntraPredModeY of luminance in Step S 182 , the process proceeds to Step S 184 .
  • Step S 183 to Step S 187 is executed similarly to each of the processes of Step S 173 to Step S 177 of FIG. 18 .
  • the process of Step S 186 or Step S 187 ends, the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • the prediction type of the CU is intra prediction, that is, luminance and the chrominance have the equal intra prediction mode
  • residual signals of luminance and the chrominance have similar tendencies.
  • the adaptive primary transform information of luminance can be applied to the chrominance in the case in which the prediction type is intra prediction and the intra prediction mode of luminance has the same value as the intra prediction mode of the chrominance, and thus coding efficiency can be further improved than in the case of ⁇ Modified example #1>.
  • a flag indicating whether or not information regarding an adaptive primary transform of a chrominance is set on the basis of an adaptive primary transform of luminance may be added and the setting of the information regarding the adaptive primary transform of the chrominance may be controlled using the flag.
  • the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set on the basis of the adaptive primary transform flag apt_flag[COMPONENT_Y] with respect to the transform block of luminance of the transform block of the chrominance.
  • the value of the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set to 0 (false).
  • FIG. 20 An example of syntax of a TU in that case is illustrated in A of FIG. 20 . Semantics of each parameter is as illustrated in B of FIG. 20 . In addition, an example of syntax of residual coding included in the syntax of A of FIG. 20 is illustrated in FIG. 21 .
  • the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag is decoded by the decoding unit 111 in accordance with the syntax table shown in FIG. 21 .
  • the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance is 1 (true)
  • the transform skip flag ts_flag[compID] of the color signal corresponding to the color signal identifier compID is 0 (false)
  • the transform quantization bypass flag flag is 0 (false)
  • the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag is decoded.
  • the value of the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag is inferred to be 0.
  • the decoding unit 111 performs such decoding in Step S 101 of FIG. 12 by executing the chrominance adaptive primary transform information inference flag decoding process.
  • An example of the flow of the chrominance adaptive primary transform information inference flag decoding process will be described with reference to the flowchart of FIG. 22 .
  • Step S 193 the decoding unit 111 determines whether or not the transform quantization bypass flag transquant_bypass_flag is 1 (true). In a case in which the transform quantization bypass flag is determined to be 0 (false) (!transquant_bypass_flag), the process proceeds to Step S 194 .
  • Step S 194 the decoding unit 111 determines whether or not the transform skip flag ts_flag[compID] of the color signal corresponding to the color signal identifier compID is 1 (true). In a case in which the transform skip flag ts_flag[compID] is determined to be 0 (false) (! ts_flag[compID]), the process proceeds to Step S 195 .
  • Step S 195 the decoding unit 111 decodes the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag[compID] corresponding to the color signal identifier compID from a bit string of the encoded data #1 and outputs the result as a part of transform information Tinfo.
  • the chrominance adaptive primary transform information inference flag decoding process ends, and the process returns to FIG. 12 .
  • Step S 191 the process proceeds to Step S 196 .
  • the adaptive primary transform flag apt_flag [COMPONENT_Y] of luminance is determined to be 0 (false) in Step S 192 (!apt_flag[COMPONENT_Y])
  • the process proceeds to Step S 196 .
  • the transform quantization bypass flag is determined to be 1 (true) (transquant_bypass_flag) in Step S 193
  • the process proceeds to Step S 196 .
  • the transform skip flag ts_flag[compID] is 1 (true) (ts_flag[compID]) in Step S 194
  • the process proceeds to Step S 196 .
  • the chrominance adaptive primary transform information inference flag decoding process ends, and the process returns to FIG. 12 .
  • the chrominance adaptive primary transform information derivation unit 131 performs the chrominance adaptive primary transform information derivation process using the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag[compID] decoded or set as described above.
  • An example of the flow of the chrominance adaptive primary transform information derivation process of the case will be described with reference to the flowchart of FIG. 23 . This flowchart corresponds to the flowchart of FIG. 15 .
  • the apt_flag derivation unit 151 determines whether or not the value of the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag[compID] is true in Step S 201 . In a case in which it is determined to be true, the process proceeds to Step S 202 . In addition, in a case in which the value of the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag[compID] is determined to be false in Step S 201 , the process proceeds to Step S 203 .
  • Step S 202 to Step S 206 is executed similarly to each of the processes of Step S 183 to Step S 187 of FIG. 19 .
  • the process of Step S 205 or Step S 206 ends, the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • adaptive primary transform information of luminance is to be applied to a chrominance can be explicitly controlled on the basis of the chrominance adaptive primary transform information inference flag with respect to signals showing difference tendencies in residual of luminance and residual of the chrominance. Therefore, the adaptive primary transform can be applied to the chrominance with higher efficiency than in the case of FIG. 15 , and thus coding efficiency can be improved.
  • the derivation of the adaptive primary transform flag of a chrominance may be limited to a case in which the size of a short side of a transform block of the chrominance is greater than or equal to a predetermined threshold value.
  • the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set on the basis of the adaptive primary transform flag apt_flag[COMPONENT_Y] with respect to the transform block of luminance corresponding to the transform block of the chrominance.
  • the value of the adaptive primary transform flag apt_flag[compID] with respect to the transform block of the chrominance is set to 0 (false).
  • the apt_flag derivation unit 151 determines whether or not the size of a short side of a transform block to be processed (min (log 2TBWSize, log 2TBHSize)) is greater than or equal to a predetermined threshold value TH in Step S 211 .
  • Step S 212 the process proceeds to Step S 212 .
  • the process proceeds to Step S 213 .
  • Step S 212 to Step S 216 is executed similarly to each of the processes of Step S 202 to Step S 206 of FIG. 23 .
  • the process of Step S 215 or Step S 216 ends, the chrominance adaptive primary transform information derivation process ends, and the process returns to FIG. 14 .
  • the size of the short side of the transform block of the chrominance is smaller than the predetermined value (e.g., 4)
  • the predetermined value e.g. 4
  • the improvement range of coding efficiency is small. For this reason, by not applying the adaptive primary transform in a case in which the size of the short side of the transform block of the chrominance is smaller than the predetermined value, a circuit scale with respect to the transform used in the adaptive primary transform for a size smaller than the predetermined value can be reduced while mitigating a decrease in coding efficiency.
  • the inverse primary transform selection unit 132 determines the type of an orthogonal transform to be applied to an inverse transform of each direction on the basis of the correspondence table of transform sets and transform types (LUT_TrSetToTrTypeIdx) shown in FIG. 1 , a primary ⁇ horizontal, vertical ⁇ transform set TrSet ⁇ H,V ⁇ , and a primary ⁇ horizontal, vertical ⁇ transform specifying flag pt_ ⁇ hor,ver ⁇ _flag, the technology is not limited thereto.
  • the transform type of each direction may be determined further with reference to the size of the transform block of the chrominance.
  • the transform type TrTypeIdxH of the (inverse) primary horizontal transform may be set to a predetermined transform type (DCT-2), and in other cases, the transform type TrTypeIdxH of the (inverse) primary horizontal transform may be set on the basis of the horizontal transform set TrSetH, and the primary horizontal transform specifying flag pt_hor_flag.
  • a specific example of the threshold value TH may be set to, for example, 1 or 2.
  • the logarithmic value log 2TBWSize of the picture width of the transform block may be replaced with a picture width TBW. In this case, a threshold value TH′ is set to 1 ⁇ TH.
  • Step S 144 of FIG. 16 An example of the flow of the primary horizontal transform type derivation process executed in Step S 144 of FIG. 16 as described above will be described with reference to the flowchart of FIG. 25 .
  • the inverse primary transform selection unit 132 determines whether or not the processing object is a transform block of luminance in Step S 221 . Ina case in which the processing object is determined to be a transform block of a chrominance, the process proceeds to Step S 222 .
  • Step S 222 the inverse primary transform selection unit 132 determines whether or not the picture width of the transform block of the chrominance (the logarithmic value log 2TBWSize of the picture width) is equal to or smaller than the threshold value TH.
  • Step S 223 the process proceeds to Step S 223 .
  • the processing object is determined to be a transform block of luminance in Step S 221 .
  • Step S 223 the inverse primary transform selection unit 132 sets the transform type TrTypeIdxH of the (inverse) primary horizontal transform on the basis of the correspondence table of transform sets and transform types (LUT_TrSetToTrTypeIdx) shown in FIG. 1 , the primary horizontal transform set TrSetH, and the primary horizontal transform specifying flag pt_hor_flag.
  • the primary horizontal transform type derivation process ends, and the process returns to FIG. 16 .
  • Step S 224 the inverse primary transform selection unit 132 sets the transform type TrTypeIdxH of the (inverse) primary horizontal transform to a predetermined value.
  • the predetermined transform type (DCT-2) may be set as the transform type TrTypeIdxV of the (inverse) primary vertical transform, and in other cases, the transform type TrTypeIdxV of the (inverse) primary vertical transform may be set on the basis of the vertical transform set identifier TrSetV and the primary vertical transform specifying flag pt_ver_flag.
  • a specific example of the threshold value TH may be set to, for example, 1 or 2.
  • the logarithmic value log 2TBHSize of the picture height of the transform block may be replaced with a picture height TBH.
  • a threshold value TH′ is set to 1 ⁇ TH.
  • Step S 145 of FIG. 16 An example of the flow of the primary vertical transform type derivation process executed in Step S 145 of FIG. 16 as described above will be described with reference to the flowchart of FIG. 26 .
  • the inverse primary transform selection unit 132 determines whether or not the processing object is a transform block of luminance in Step S 231 . Ina case in which the processing object is determined to be a transform block of a chrominance, the process proceeds to Step S 232 .
  • Step S 232 the inverse primary transform selection unit 132 determines whether or not the picture height of the transform block of the chrominance (the logarithmic value log 2TBHSize of the picture height) is equal to or smaller than the threshold value TH.
  • Step S 233 the process proceeds to Step S 233 .
  • the processing object is determined to be a transform block of luminance in Step S 231 .
  • Step S 233 the inverse primary transform selection unit 132 sets the transform type TrTypeIdxV of the (inverse) primary vertical transform on the basis of the correspondence table of transform sets and transform types (LUT_TrSetToTrTypeIdx) shown in FIG. 1 , the primary vertical transform set TrSetV, and the primary vertical transform specifying flag pt_ver_flag.
  • the primary vertical transform type derivation process ends, and the process returns to FIG. 16 .
  • Step S 234 the inverse primary transform selection unit 132 sets the transform type TrTypeIdxV of the (inverse) primary vertical transform to a predetermined value.
  • the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx have been described as information regarding the (inverse) orthogonal transform in the first embodiment, details of the information regarding the (inverse) orthogonal transform are arbitrary and are not limited to the parameters.
  • a transform skip flag ts_flag indicating whether the (inverse) orthogonal transform process is to be skipped may be included as illustrated in the table of FIG. 6 .
  • a transform skip flag ts_flag of a chrominance is transmitted (signaled) from an encoding side to a decoding side as indicated in the row of No. #0. If transform skip flags ts_flag not only of luminance but also a chrominance are explicated encoded as described above, there is concern of the code amount increasing and coding efficiency deteriorating. In this case, the encoding side also should perform the settings and encoding of the transform skip flag ts_flag for each of signals (Y, Cb, and Cr) and thus there is also concern of the processing amount increasing. In addition, the decoding side should decode the transform skip flag ts_flag for each of the signals (Y, Cb, and Cr) and thus there is concern of the processing amount increasing.
  • the transform skip flag ts_flag of a chrominance is inferred from the transform skip flag ts_flag of luminance at all times, for example, as indicated in the row of No. #1.
  • a value of the transform skip flag ts_flag[compID] of a chrominance may be set to a value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in a case in which the prediction type of the coding block is intra prediction, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from encoded data).
  • the value of the transform skip flag ts_flag of luminance can be used only in a case of the inter prediction mode in which residual signals thereof show similar tendencies, and thus deterioration in coding efficiency can be further curbed.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data). In this manner, the value of the transform skip flag ts_flag of luminance can be used only in the case of the prediction mode in which the residual signals thereof show similar tendencies
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data). In this manner, the value of the transform skip flag ts_flag of luminance can be used only in the case of the prediction mode in which the residual signals thereof show similar tendencies, and thus deterioration in coding efficiency can be further curbed
  • a chrominance transform skip information inference flag chroma_ts_info_infer_flag indicating whether the transform skip flag ts_flag[compID] of the chrominance is to be inferred on the basis of the transform skip flag ts_flag[COMPONENT_Y] of luminance is 1 (true), for example, as indicated in the row of No.
  • the value of the transform skip flag ts_flag[copmID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in a case in which the chrominance transform skip information inference flag chroma_ts_info_infer_flag is 0 (false), the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • each of the above-described cases can be arbitrarily combined similarly to the case of FIG. 7 . In this manner, the effects obtained in each of the cases can be exhibited. In addition, each of the above-described cases can also be combined with other cases that are not described above.
  • the transform skip flag ts_flag of the chrominance may be inferred from the transform skip flag ts_flag of luminance at all times as described with reference to the row of No. #1 of FIG. 27 .
  • An example of syntax of residual_coding of that case is illustrated in FIG. 28 .
  • the setting is performed using the transform skip flag of luminance without signaling the transform skip flag ts_flag.
  • the decoding unit 111 sets the transform skip flag ts_flag[compID] of the chrominance by executing a transform skip flag derivation process.
  • An example of the flow of the transform skip flag derivation process will be described with reference to the flowchart of FIG. 29 .
  • the decoding unit 111 determines whether or not the transform skip flag enabled flag ts_enabled_flag is true in Step S 241 . In a case in which the transform skip flag enabled flag ts_enabled_flag is determined to be true, the process proceeds to Step S 242 . In Step S 242 , the decoding unit 111 determines whether or not the transform quantization bypass enabled flag transquant_bypass_enabled_flag is false. In a case in which the transform quantization bypass enabled flag transquant_bypass_enabled_flag is false, the process proceeds to Step S 243 .
  • Step S 243 the decoding unit 111 determines whether or not the size of the transform block to be processed is equal to or smaller than a maximum transform skip block size. In a case in which the size is determined to be equal to or smaller than the maximum transform skip block size, the process proceeds to Step S 244 .
  • Step S 245 the decoding unit 111 sets a value of the transform skip flag ts_flag[COMPONENT_Y] of luminance for a value of the transform skip flag ts_flag[compID] of the chrominance.
  • the transform skip flag derivation process ends.
  • Step S 244 the process proceeds to Step S 246 . Since the transform skip flag ts_flag[compID] of luminance is signaled, the decoding unit 111 decodes the encoded data and thereby obtains the transform skip flag ts_flag[compID] of luminance in Step S 246 . When the process of Step S 246 ends, the transform skip flag derivation process ends.
  • Step S 241 a case in which the transform skip flag enabled flag ts_enabled_flag is determined to be false in Step S 241 , a case in which the transform quantization bypass enabled flag transquant_bypass_enabled_flag is determined to be true in Step S 242 , or a case in which the size of the transform block of the processing object is determined to be greater than the maximum transform skip block size in Step S 243 , the transform skip flag derivation process ends.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in the case in which the prediction type of the coding block is intra prediction, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • FIG. 30 An example of syntax of residual_coding of that case is illustrated in FIG. 30 .
  • Step S 256 is executed similarly to the process of Step S 245 of FIG. 29 . That is, the decoding unit 111 sets the value of the transform skip flag ts_flag[compID] of the chrominance to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • the transform skip flag derivation process ends.
  • Step S 257 is executed similarly to the process of Step S 246 of FIG. 29 . That is, the decoding unit 111 decodes the encoded data and thereby obtains the signaled transform skip flag ts_flag[compID]. When the process of Step S 257 ends, the transform skip flag derivation process ends.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted using the value of the transform skip flag ts_flag of luminance only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, in the case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • FIG. 32 An example of syntax of residual_coding in that case is illustrated in FIG. 32 .
  • the transform skip flag ts_flag is signaled in the case in which the prediction type is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • the transform skip flag ts_flag is not signaled. That is, the transform skip flag ts_flag is set using the transform skip flag ts_flag of luminance in that case.
  • Step S 265 the process proceeds to Step S 266 .
  • Step S 267 is executed similarly to the process of Step S 256 of FIG. 31 . That is, the decoding unit 111 sets the value of the transform skip flag ts_flag[compID] of the chrominance to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • the transform skip flag derivation process ends.
  • Step S 268 is executed similarly to the process of Step S 257 of FIG. 31 . That is, the decoding unit 111 decodes the encoded data and thereby obtains the signaled transform skip flag ts_flag[compID]. When the process of Step S 268 ends, the transform skip flag derivation process ends.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted using the value of the transform skip flag ts_flag of luminance only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, in the case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • FIG. 34 An example of syntax of residual_coding in that case is illustrated in FIG. 34 .
  • the transform skip flag ts_flag is signaled in the case in which the prediction type is intra prediction of which the prediction mode is not intra block copy.
  • the transform skip flag ts_flag is not signaled. That is, the transform skip flag ts_flag is set using the transform skip flag ts_flag of luminance in that case.
  • Step S 277 is executed similarly to the process of Step S 267 of FIG. 33 . That is, the decoding unit 111 sets the value of the transform skip flag ts_flag[compID] of the chrominance to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • the transform skip flag derivation process ends.
  • Step S 278 is executed similarly to the process of Step S 268 of FIG. 33 . That is, the decoding unit 111 decodes the encoded data and thereby obtains the signaled transform skip flag ts_flag[compID]. When the process of Step S 278 ends, the transform skip flag derivation process ends.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted using the value of the transform skip flag ts_flag of luminance only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[copmID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in the case in which the chrominance transform skip information inference flag chroma_ts_info_infer_flag is 0 (false), the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • FIG. 36 An example of syntax of a transform unit in that case is illustrated in FIG. 36 .
  • a chrominance transform skip information inference flag chroma_ts_info_infer_flag is set.
  • FIG. 37 An example of syntax of residual_coding is illustrated in FIG. 37 .
  • the transform skip flag ts_flag is signaled in a case in which the chrominance transform skip information inference flag chroma_ts_info_infer_flag is false.
  • the transform skip flag ts_flag is not signaled. That is, the transform skip flag ts_flag is set using the transform skip flag ts_flag of luminance in that case.
  • Step S 284 the decoding unit 111 determines whether the processing object is a chrominance and the chrominance transform skip information inference flag chroma_ts_info_infer_flag is true. In a case in which the flag is determined to be of a chrominance and true, the process proceeds to Step S 285 . In addition, in a case in which the processing object is determined to be luminance or the chrominance transform skip information inference flag chroma_ts_info_infer_flag is determined to be false in Step S 284 , the process proceeds to Step S 286 .
  • Step S 285 is executed similarly to the process of Step S 245 of FIG. 29 . That is, the decoding unit 111 sets the value of the transform skip flag ts_flag[compID] of the chrominance to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • the transform skip flag derivation process ends.
  • Step S 286 is executed similarly to the process of Step S 246 of FIG. 29 . That is, the decoding unit 111 decodes the encoded data and thereby obtains the signaled transform skip flag ts_flag[compID]. When the process of Step S 286 ends, the transform skip flag derivation process ends.
  • a secondary transform identifier st_idx indicating which (inverse) secondary transform is to be applied may be included as shown in the table of FIG. 6 .
  • a secondary transform identifier chroma_st_idx of a chrominance common for Cb and Cr
  • a secondary transform identifier st_idx[Y] of luminance for example, an increase in overhead of the code amount of the syntax by which the chrominance (Cb or Cr) is signaled can be curbed as well.
  • the secondary transform identifier st_idx of a chrominance is also transmitted (signaled) to the decoding side from the encoding side as indicated in the row of No. #0.
  • FIG. 40 is an example of syntax of a TU in the related art. As is indicated in the syntax of FIG. 40 , the secondary transform identifier chroma_st_idx of the chrominance is signaled as well. If the secondary transform identifier st_idx is explicitly encoded not only for luminance but also for the chrominance, there is concern of the code amount increasing the coding efficiency deteriorating.
  • the encoding side should perform the settings and encoding of the secondary transform identifier st_idx for each of signals (Y, Cb, and Cr), and thus there is concern of the processing amount increasing.
  • the decoding side should decode the secondary transform identifier st_idx for each of the signals (Y, Cb, and Cr), and thus there is concern of the processing amount increasing.
  • the secondary transform identifier chroma_st_idx of the chrominance is inferred from the secondary transform identifier st_idx of luminance at all times, for example, as indicated in the row of No. #1.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance.
  • the transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted, and thus overhead of the code amount can be reduced and deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in a case in which the prediction type of the coding block is intra prediction, the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • the value of the secondary transform identifier st_idx of luminance can be used only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be further curbed.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other, the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data). In this manner, the value of the secondary transform identifier st_idx of luminance can be used only in the case of the prediction mode
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy, the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data). In this manner, the value of the secondary transform identifier st_idx of luminance can be used only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deteriorat
  • the value of the secondary transform identifier st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in a case in which the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is 0 (false), the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from
  • each of the above-described cases can be arbitrarily combined similarly to the case of FIG. 7 . In this manner, the effects obtained in each of the cases can be exhibited. In addition, each of the above-described cases can also be combined with other cases that are not described above.
  • the secondary transform identifier chroma_st_idx of the chrominance may be inferred from the secondary transform identifier st_idx of luminance at all times as described with reference to the row of No. #1 of FIG. 39 .
  • An example of syntax of a TU in that case is illustrated in FIG. 41 .
  • the setting is performed using the secondary transform identifier st_idx of luminance without signaling the secondary transform identifier chroma_st_idx of the chrominance as illustrated in FIG. 41 .
  • the decoding unit 111 executes a secondary transform identifier derivation process for the chrominance and thereby sets the secondary transform identifier chroma_st_idx of the chrominance.
  • a secondary transform identifier derivation process for the chrominance executes a secondary transform identifier derivation process for the chrominance and thereby sets the secondary transform identifier chroma_st_idx of the chrominance.
  • the decoding unit 111 determines whether or not the secondary transform enabled flag st_enabled_flag is true in Step S 291 . In a case in which the secondary transform enabled flag st_enabled_flag is determined to be true, the process proceeds to Step S 292 . In Step S 292 , the decoding unit 111 determines whether or not a transform quantization bypass enabled flag transquant_bypass_enabled_flag is false. In a case in which the transform quantization bypass enabled flag transquant_bypass_enabled_flag is determined to be false, the process proceeds to Step S 293 .
  • Step S 293 the decoding unit 111 determines whether or not the number of non-zero coefficients (numNonZeroCoefTH) of the chrominance is greater than or equal to a predetermined threshold value (stNumZeroCoefTH). In a case in which the number of non-zero coefficients of the chrominance is determined to be greater than or equal to the predetermined threshold value, the process proceeds to Step S 294 .
  • Step S 294 the decoding unit 111 sets the value of the secondary transform identifier chroma_st_idx of the chrominance to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier derivation process for the chrominance ends.
  • the secondary transform identifier derivation process for the chrominance ends.
  • the secondary transform identifier derivation process for the chrominance ends.
  • the secondary transform identifier derivation process for the chrominance ends.
  • the secondary transform identifier derivation process for the chrominance ends.
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance as described with reference to the row of No. #2 of FIG. 39 , and in the case in which the prediction type of the coding block is intra prediction, the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • FIG. 43 An example of syntax of a TU in that case is illustrated in FIG. 43 .
  • the secondary transform identifier chroma_st_idx of the chrominance is signaled.
  • the secondary transform identifier chroma_st_idx of the chrominance is set using the secondary transform identifier st_idx of luminance.
  • Step S 305 is executed similarly to the process of Step S 294 of FIG. 42 . That is, the decoding unit 111 sets the value of the secondary transform identifier chroma_st_idx of the chrominance to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier derivation process for the chrominance ends.
  • Step S 306 the decoding unit 111 decodes the encoded data and thereby obtains the signaled secondary transform identifier st_idx.
  • the secondary transform identifier derivation process for the chrominance ends.
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted using the value of the secondary transform identifier st_idx of luminance only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the decoding unit 111 sets the value of the secondary transform identifier chroma_st_idx of the chrominance to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • FIG. 45 An example of syntax of a TU in that case is illustrated in FIG. 45 .
  • the secondary transform identifier chroma_st_idx is signaled.
  • the secondary transform identifier chroma_st_idx of the chrominance is set to the secondary transform identifier st_idx of luminance.
  • Step S 316 is executed similarly to the process of Step S 305 of FIG. 44 .
  • the process of Step S 317 is executed similarly to the process of Step S 306 of FIG. 44 .
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted using the value of the secondary transform identifier st_idx of luminance only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance is set to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier chroma st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • FIG. 47 An example of syntax of a TU in that case is illustrated in FIG. 47 .
  • the secondary transform identifier st_idx is not signaled only in a case in which the identifier is of the chrominance and the prediction type is inter prediction, or a case in which the identifier is of the chrominance and prediction type is intra prediction of which the prediction mode is intra block copy. That is, in this case, the secondary transform identifier chroma_st_idx is set using the secondary transform identifier st_idx of luminance.
  • Step S 324 the process proceeds to Step S 325 .
  • the process proceeds to Step S 326 .
  • Step S 326 is executed similarly to the process of Step S 316 of FIG. 46 . That is, the decoding unit 111 sets the value of the secondary transform identifier chroma_st_idx of the chrominance to the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier derivation process for the chrominance ends.
  • Step S 327 is executed similarly to the process of Step S 317 of FIG. 46 . That is, the decoding unit 111 can decode the encoded data and thereby obtains the signaled secondary transform identifier st_idx.
  • the secondary transform identifier derivation process for the chrominance ends.
  • transmission (encoding and decoding) of the secondary transform identifier st_Idx of the chrominance can be omitted using the value of the secondary transform identifier st_idx of luminance only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in the case in which the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is 0 (false), the secondary transform identifier st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • FIG. 49 An example of syntax of a TU in that case is illustrated in FIG. 49 .
  • Step S 334 the decoding unit 111 decodes and acquires the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag.
  • Step S 335 the decoding unit 111 determines whether or not the acquired chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is true. In a case in which the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is determined to be true, the process proceeds to Step S 336 . In addition, chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is determined to be false, the process proceeds to Step S 337 .
  • Step S 336 is executed similarly to the process of Step S 326 of FIG. 48 . That is, the decoding unit 111 sets the value of the secondary transform identifier chroma_st_idx of the chrominance to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier derivation process for the chrominance ends.
  • Step S 337 is executed similarly to the process of Step S 327 of FIG. 48 . That is, the decoding unit 111 can decode the encoded data and thereby obtains the signaled secondary transform identifier chroma_st_idx of the chrominance.
  • the secondary transform identifier derivation process for the chrominance ends.
  • FIG. 51 is a block diagram illustrating an example of the main configuration of an image encoding apparatus which is an aspect of the image processing apparatus to which the present technology is applied.
  • the image encoding apparatus 400 illustrated in FIG. 51 is an image encoding apparatus corresponding to the image decoding apparatus 100 of FIG. 9 and generates encoded data (bit stream) which has been decoded by the image decoding apparatus 100 by encoding an imaging using an encoding method corresponding to a decoding method used by the image decoding apparatus 100 .
  • the image encoding apparatus 400 implements, for example, a technology proposed by HEVC or JVET.
  • FIG. 51 illustrates main configurations such as flows of processing units and data, and the like, and FIG. 51 is not illustrating entire configurations. That is, there may be processing units in the image encoding apparatus 400 that are not illustrated as blocks in FIG. 51 or flows of processes and data that are not indicated by arrows and the like in FIG. 51 .
  • the image encoding apparatus 400 has a control unit 411 , an arithmetic operation unit 412 , a transform unit 413 , a quantization unit 414 , an encoding unit 415 , an inverse quantization unit 416 , an inverse transform unit 417 , an arithmetic operation unit 418 , a frame memory 419 , and a prediction unit 420 as illustrated in FIG. 51 .
  • the control unit 411 divides a moving image #2 into blocks which are units of processing (CUs, PUs, transform blocks, etc.) on the basis of the block size of the units of processing specified from the outside or in advance, and inputs an image I corresponding to the divided blocks to the arithmetic operation unit 412 .
  • the control unit 411 determines encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, etc.) to be supplied to each of the blocks, for example, on the basis of Rate-Distortion Optimization (RDO).
  • RDO Rate-Distortion Optimization
  • the header information Hinfo is supplied to each of the blocks.
  • the prediction mode information Pinfo is supplied to the transform unit 413 , the encoding unit 415 , and the prediction unit 420 .
  • the transform information Tinfo is supplied to the encoding unit 415 , the transform unit 413 , the quantization unit 414 , the inverse quantization unit 416 , and the inverse transform unit 417 .
  • the arithmetic operation unit 412 receives the input of the image I corresponding to the blocks which are the units of processing and a predictive image P supplied from the prediction unit 420 , subtracts the predictive image P from the image I as indicated in the formula (25), thereby derives predictive residual D, and supplies the predictive residual to the transform unit 413 .
  • the transform unit 413 is a processing unit that performs an inverse transform which is an inverse process to a transform process performed by the inverse transform unit 417 , receives the input of the predictive residual D, the prediction mode information Pinfo, and the transform information Tinfo, applies a transform to the predictive residual D on the basis of the prediction mode information Pinfo and the transform information Tinfo, thereby derives a transform coefficient Coeff, and supplies the transform coefficient to the quantization unit 414 .
  • the quantization unit 414 is an inverse process to the inverse quantization unit 416 , receives the input of the transform information Tinfo and the transform coefficient Coeff, scales (quantizes) the transform coefficient Coeff on the basis of the transform information Tinfo, and supplies the quantized transform coefficient, that is, a quantized transform coefficient level level to the encoding unit 415 .
  • the encoding unit 415 is an inverse process to the decoding unit 111 ( FIG. 9 ), transforms the encoding parameters (the header information, the prediction mode information Pinfo, and the transform information Tinfo) supplied from the control unit 411 , and the quantized transform coefficient level level supplied from the quantization unit 414 into syntax values of each of syntax elements in accordance with the definitions of a syntax table, makes each of the syntax values into variable length codes (e.g., arithmetic code), and thereby generates bit strings.
  • variable length codes e.g., arithmetic code
  • the encoding unit 415 derives residual information Rinfo from the quantized transform coefficient level level, makes the residual information Rinfo into variable length codes, and generates bit strings. In addition, the encoding unit 415 multiplexes the bit strings of each of the syntax elements that have been made into variable length codes, and then generates and outputs encoded data #1.
  • the inverse quantization unit 416 is a processing unit similar to the inverse quantization unit 112 ( FIG. 9 ) and performs a process similar to that of the inverse quantization unit 112 in the image encoding apparatus 400 .
  • the inverse transform unit 417 is a processing unit similar to the inverse transform unit 113 ( FIG. 9 ) and performs a process similar to that of the inverse transform unit 113 d in the image encoding apparatus 400 .
  • the frame memory 419 is a processing unit similar to the frame memory 115 ( FIG. 9 ) and performs a process similar to that of the frame memory 115 in the image encoding apparatus 400 .
  • the prediction unit 420 is a processing unit similar to the prediction unit 116 ( FIG. 9 ) and performs a process similar to that of the prediction unit 116 in the image encoding apparatus 400 .
  • FIG. 52 is a block diagram illustrating an example of the main configuration of the transform unit 413 .
  • the transform unit 413 has a switch 431 , a primary transform unit 432 , and a secondary transform unit 433 .
  • the primary transform unit 432 and the secondary transform unit 433 are skipped, and the predictive residual D is output as a transform coefficient Coeff.
  • the primary transform unit 432 performs a process relating to a primary transform that is a predetermined transform process, for example, an orthogonal transform.
  • the primary transform is an inverse process of the inverse primary transform executed by the inverse primary transform unit 123 or the like of the image decoding apparatus 100 .
  • the primary transform unit 432 receives inputs of, for example, the color signal identifier compID, the adaptive primary transform flag apt_flag[COMPONENT_Y] of luminance, the primary transform identifier pt_idx[COMPONENT_Y] of luminance, the prediction mode information PInfo, sizes of the transform block (a logarithmic value log 2TBWSize of the picture width and a logarithmic value log 2TBHSize of a picture height), and predictive residual D.
  • the color signal identifier compID receives inputs of, for example, the color signal identifier compID, the adaptive primary transform flag apt_flag[COMPONENT_Y] of luminance, the primary transform identifier pt_idx[COMPONENT_Y] of luminance, the prediction mode information PInfo, sizes of the transform block (a logarithmic value log 2TBWSize of the picture width and a logarithmic value log 2TBHSize of a picture height), and predictive residual D.
  • the primary transform unit 432 selects a transform type TrTypeIdxH of a primary horizontal transform and a transform type TrTypeIdxV of a primary vertical transform of the color signal specified by the color signal identifier compID with reference to the prediction mode information PInfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[COMPONENT_Y] of luminance, the primary transform identifier pt_idx[COMPONENT_Y] of luminance, and the primary transform identifier pt_idx[COMPONENT_Y] of luminance, performs the primary horizontal transform defined by the primary horizontal transform type TrTypeIdxH and the picture width log 2TBWSize of the transform block and the primary vertical transform defined by the primary vertical transform type TrTypeIdxV and the picture height log 2TBHSize of the transform block on the predictive residual D. and thereby derives and outputs a transform coefficient Coeff_P that has undergone the primary transform (also referred to as a primary transform coefficient)
  • the primary transform unit 432 will be described below in more detail. As illustrated in FIG. 52 , the primary transform unit 432 has a chrominance adaptive primary transform information derivation unit 441 , a primary transform selection unit 442 , the primary horizontal transform unit 443 , and the primary vertical transform unit 444 .
  • FIG. 53 is a functional block diagram illustrating an example of the main configuration of the chrominance adaptive primary transform information derivation unit 441 .
  • the chrominance adaptive primary transform information derivation unit 441 has an apt_flag derivation unit 451 and a pt_idx derivation unit 452 .
  • the apt_flag derivation unit 451 performs a process relating to the derivation of the adaptive primary transform flag apt_flag of the chrominance.
  • the pt_idx derivation unit 452 performs a process relating to the derivation of the primary transform identifier pt_idx of the chrominance.
  • each of the apt_flag derivation unit 451 and the pt_idx derivation unit 452 has a similar configuration and performs similar processes to those of the apt_flag derivation unit 151 and the pt_idx derivation unit 152 ( FIG. 11 ).
  • the chrominance adaptive primary transform information derivation unit 441 has a similar configuration and performs similar processes to those of the chrominance adaptive primary transform information derivation unit 131 .
  • the chrominance adaptive primary transform information derivation unit 441 can derive the adaptive primary transform information of the chrominance transform block on the basis of the adaptive primary transform information of luminance corresponding to the chrominance transform block. Therefore, it is possible to reduce the encoding processing amount without encoding the adaptive primary transform information of the chrominance transform block.
  • the primary transform selection unit 442 receives an input of the prediction mode information PInfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • the primary transform selection unit 442 derives the transform type TrTypeIdxH of the primary horizontal transform and the transform type TrTypeIdxV of the primary vertical transform of the color signal specified by the color signal identifier compID with reference to the prediction mode information Pinfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • the primary transform selection unit 442 supplies the derived transform type TrTypeIdxH of the primary horizontal transform to the primary horizontal transform unit 443 .
  • the primary transform selection unit 442 supplies the derive transform type TrTypeIdxV of the primary vertical transform to the primary vertical transform unit 444 .
  • operations of the primary transform selection unit 442 are basically similar to those of the inverse primary transform selection unit 132 of the image decoding apparatus 100 . That is, the description with respect to the inverse primary transform selection unit 132 in the first embodiment can be description with respect to the primary transform selection unit 442 by replacing the inverse primary horizontal transform IPThor with a primary horizontal transform PThor, replacing the inverse primary vertical transform IPTver with the primary vertical transform PTver, and replacing the inverse primary transform with the primary transform.
  • the primary horizontal transform unit 443 receives an input of the predictive residual D, the transform type TrTypeIdxH of the primary horizontal transform, and log 2TBWSize indicating the picture width of the transform block (the logarithmic value of the picture width) for each transform block of each color signal.
  • the primary horizontal transform unit 443 executes the primary horizontal transform Phor defined by the transform type TrTypeIdxH and the picture width of the transform block on the predictive residual D and outputs the result as a transform coefficient Coeff_Phor that has undergone the primary horizontal transform.
  • the primary vertical transform unit 444 receives an input of the transform coefficient Coeff_Phor that has undergone the primary horizontal transform, the transform type TrTypeIdxV of the primary vertical transform and log 2TBHSize indicating the picture height of the transform block (the logarithmic value of the picture height) for each transform block of each color signal.
  • the primary vertical transform Pver defined by the transform type TrTypeIdxV and the picture height of the transform block is executed on the transform coefficient Coeff_Phor that has undergone the primary horizontal transform and the result is output as a transform coefficient Coeff_P that has undergone the primary transform.
  • the primary transform unit 432 can apply an adaptive primary transform selected for luminance to a transform block of the chrominance in a case in which a residual signal of the color signal shows a similar tendency to a residual signal of luminance.
  • a primary transform process on the residual signal of the chrominance with higher coding efficiency than in the related art.
  • the secondary transform unit 433 performs a process relating to a secondary transform which is a predetermined transform process, for example, an orthogonal transform.
  • the secondary transform is an inverse process to the inverse secondary transform executed by the inverse secondary transform unit 122 , or the like of the image decoding apparatus 100 .
  • the secondary transform unit 433 receives an input of, for example, a secondary transform identifier st_idx, a scan identifier scanIdx indicating a scanning method of the transform coefficient, and a primary transform coefficient Coeff_P, and derives and outputs a transform coefficient Coeff that has undergone the secondary transform (also referred to as a secondary transform coefficient).
  • the secondary transform unit 433 executes the process of the secondary transform corresponding to the secondary transform identifier st_idx on the primary transform coefficient Coeff_P. and outputs a transform coefficient Coeff_S that has undergone the secondary transform.
  • the secondary transform unit 433 skips the secondary transform, and outputs the transform coefficient Coeff_P that has undergone the primary transform as the transform coefficient Coeff_S that has undergone the secondary transform.
  • control unit 411 When the image encoding process is started, the control unit 411 performs an encoding control process and performs block division, a setting of encoding parameters, and the like in Step S 401 .
  • the prediction unit 420 performs a prediction process and generates a predictive image and the like in an optimum prediction mode. For example, in the prediction process, the prediction unit 420 generates performs intra prediction and generates a predictive image and the like in an optimum intra prediction mode, and performs inter prediction and generates a predictive image and the like in an optimum inter prediction mode, and selects an optimum prediction mode among the modes on the basis of a cost function value or the like.
  • Step S 403 the arithmetic operation unit 412 calculates the difference between the input image and the predictive image in the optimum mode selected in the prediction process of Step S 402 . That is, the arithmetic operation unit 412 generates predictive residual D between the input image and the predictive image.
  • the data amount of the predictive residual D obtained as described above is reduced more than that of the original image data. Therefore, the data amount can be more compressed than in a case in which the image is encoded without change.
  • Step S 404 the transform unit 413 performs a transform process on the predictive residual D generated in the process of Step S 403 and derives a transform coefficient Coeff.
  • the transform process is an inverse process to the inverse transform process of Step S 407 and an inverse process of the inverse transform process executed in the above-described image decoding process. Details of the process of Step S 404 will be described below.
  • Step S 405 the quantization unit 414 quantizes the transform coefficient Coeff obtained from the process of Step S 404 by using a quantization parameter calculated by the control unit 411 or the like and derives a quantized transform coefficient level level.
  • Step S 406 the inverse quantization unit 416 performs inverse quantization on the quantized transform coefficient level level generated from the process of Step S 405 using characteristics corresponding to those of the quantization of Step S 405 and derives a transform coefficient Coeff_IQ.
  • Step S 407 the inverse transform unit 417 performs an inverse transform on the transform coefficient Coeff_IQ obtained from the process of Step S 406 using a method corresponding to the transform process of Step S 404 and derives predictive residual D′. Note that the inverse transform process is executed similarly to the inverse transform process executed in the above-described image decoding process.
  • Step S 408 the arithmetic operation unit 418 generates a decoded image that has been locally decoded by adding the predictive image obtained from the prediction process of Step S 402 to the predictive residual D′ derived from the process of Step S 407 .
  • Step S 409 the frame memory 419 stores the decoded image that has been locally decoded obtained from the process of Step S 408 .
  • Step S 410 the encoding unit 415 encodes the quantized transform coefficient level level obtained from the process of Step S 405 .
  • the encoding unit 415 encodes the quantized transform coefficient level level which is information regarding the image using arithmetic encoding or the like and generates encoded data.
  • the encoding unit 415 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo).
  • the encoding unit 415 derives residual information RInfo from the quantized transform coefficient level level and encodes the residual information RInfo.
  • Step S 411 the encoding unit 415 puts the encoded data of the various kinds of information generated as described above and outputs the data as a bit stream to the outside of the image encoding apparatus 400 .
  • the bit stream is transmitted to the decoding side via, for example, a transmission path or a recording medium.
  • the switch 431 determines whether or not the transform skip flag ts_flag is 2D_TS (a case in which the flag indicates two-dimensional transform skip) in Step S 421 or the transform quantization bypass flag transquant_bypass_flag is (true). In a case in which the transform skip flag ts_flag is determined to be 2D_TS or the transform quantization bypass flag is determined to be 1 (true), the transform process ends, and the process returns to FIG. 54 .
  • the switch 431 omits the orthogonal transform process (the primary transform or the secondary transform) and outputs the input predictive residual D to the outside of the transform unit 413 (supplies it to the quantization unit 414 ) as a transform coefficient Coeff.
  • Step S 421 of FIG. 55 the process proceeds to Step S 422 .
  • the switch 431 supplies the input predictive residual D to the primary transform unit 432 .
  • Step S 423 When the process of Step S 423 ends, the process proceeds to Step S 424 .
  • the process proceeds to Step S 424 .
  • Step S 424 the primary transform selection unit 442 executes a primary transform selection process, derives a transform type TrTypeIdxH of a primary horizontal transform and a transform type TrTypeIdxV of a primary vertical transform of the color signal specified by the color signal identifier compID with reference to the prediction mode information Pinfo, the color signal identifier compID, the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID, and the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID, and outputs each of the transform types to the primary horizontal transform unit 443 and the primary vertical transform unit 444 .
  • Step S 425 the primary horizontal transform unit 443 executes the primary horizontal transform Phor defined by the transform type TrTypeIdxH of the primary horizontal transform and the picture height of the transform block on the predictive residual D of the transform block for each transform block specified by the color signal identifier compID and outputs the result as the transform coefficient Coeff_Phor that has undergone the primary horizontal transform.
  • Step S 426 the primary vertical transform unit 444 executes the primary vertical transform Pver defined by the transform type TrTypeIdxV of the primary vertical transform and the picture height of the transform block on the transform coefficient Coeff_Phor of the transform block that has undergone the primary horizontal transform for each transform block specified by the color signal identifier compID and outputs the result as the transform coefficient Coeff_P that has undergone the primary transform.
  • Step S 427 the secondary transform unit 433 performs the secondary transform on the input primary transform coefficient Coeff_P on the basis of the secondary transform identifier st_idx and derives and outputs the transform coefficient Coeff.
  • the transform process ends and the process returns to FIG. 54 .
  • Step S 424 of FIG. 55 An example of the flow of the primary transform selection process executed in Step S 424 of FIG. 55 will be described with reference to the flowchart to FIG. 56 .
  • the primary transform selection unit 442 determines whether or not the adaptive primary transform flag apt_flag[compID] of the color signal corresponding to the color signal identifier compID is 1 (true) in Step S 431 . In a case in which the adaptive primary transform flag apt_flag[compID] is determined to be 1 (true), the process proceeds to Step S 432 .
  • Step S 432 the primary transform selection unit 442 selects a transform set TrSetV (primary horizontal transform set) of the primary vertical transform and a transform set TrSetH (primary vertical transform set) of the primary horizontal transform on the basis of the prediction mode information PInfo.
  • TrSetV primary horizontal transform set
  • TrSetH primary vertical transform set
  • Step S 433 the primary transform selection unit 442 derives a primary horizontal transform specifying flag pt_hor_flag and a primary vertical transform specifying flag pt_ver_flag from the primary transform identifier pt_idx[compID] of the color signal corresponding to the color signal identifier compID.
  • Step S 434 the primary transform selection unit 442 selects the transform type TrTypeIdxH of an orthogonal transform applied as the primary horizontal transform IPThor with reference to the primary horizontal transform set TrSetH and the primary horizontal transform specifying flag pt_hor_flag.
  • Step S 435 the primary transform selection unit 442 selects the transform type TrTypeIdxV of an orthogonal transform applied as the inverse primary vertical transform IPTver with reference to the primary vertical transform set TrSetV and the primary vertical transform specifying flag pt_ver_flag.
  • the primary transform selection process ends, and the process returns to FIG. 55 .
  • Step S 431 in a case in which it is determined that the adaptive primary transform flag apt_flag[compID] is 0 (false), the process proceeds to Step S 436 .
  • the primary transform unit 432 derives the transform type TrTvpeIdxH of the inverse primary horizontal transform IPhor and the transform type TrTypeIdxV of the inverse primary vertical transform IPTver using the method in accordance with the value of the adaptive primary transform flag apt_flag[compID] of the chrominance.
  • the encoding unit 415 encodes the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx generated by the transform unit 413 . However, the encoding unit 415 may encode the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx of luminance and cause the flag and the identifier to be signaled, and may not encode the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx of the chrominance.
  • the encoding unit 415 encodes the primary transform identifier pt_idx[COMPONENT_Y] of luminance (causes the flag to be signaled) in Step S 443 .
  • the primary transform information encoding process ends.
  • the encoding unit 415 determines whether or not the processing object is a chrominance in Step S 441 .
  • the processes of Step S 442 and Step S 443 are omitted, that is, the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx are not signaled, and the primary transform information encoding process ends.
  • the adaptive primary transform selected for luminance can be applied to the transform block of the chrominance in the case in which the residual signal of the chrominance has a similar tendency to the residual signal of luminance. Therefore, the primary transform process with further improved coding efficiency can be performed on the residual signal of the chrominance in comparison to technologies of the related art.
  • the amount of the encoding process can be reduced more than in the case in which the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx are explicitly encoded for each of luminance and the chrominance.
  • the chrominance adaptive primary transform information derivation unit 441 has a similar configuration and performs similar processes to those of the chrominance adaptive primary transform information derivation unit 131 . That is, the chrominance adaptive primary transform information derivation unit 441 can execute the chrominance adaptive primary transform information derivation process, similarly to the chrominance adaptive primary transform information derivation unit 131 .
  • the image encoding apparatus 400 can make it possible to apply the adaptive primary transform also to the chrominance and further infer the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx of the chrominance from the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx of luminance at all times as indicated in the row of No. #1 in the table of FIG. 7 , and therefore similar effects to those in the case of the image decoding apparatus 100 can be obtained.
  • the image encoding apparatus 400 can make it possible to apply the adaptive primary transform also so the chrominance and further can set the value of the adaptive primary transform flag of apt_flag[Cb/Cr] of the chrominance to the value of adaptive primary transform flag apt_flag[Y] of luminance in the case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, for example, and can set the adaptive primary transform flag of apt_flag[Cb/Cr] of the chrominance to 0 (false) in the case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other as indicated in the row of No. #3 in the table of FIG. 7 . That is, similar effects to those in the case of the image decoding apparatus 100 can be obtained in that case as well.
  • the image encoding apparatus 400 can make it possible to apply the adaptive primary transform also to the chrominance, and further can set the value of the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance to the value of the adaptive primary transform flag apt_flag[Y] of luminance in the case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, for example, and can set the value of the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance to 0 (false) in the case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy (IntraBC; also referred to as in-screen motion compensation), as indicated in the row of No. #4 in the table of FIG. 7 . That is, similar effects to those in the case of the image decoding apparatus 100 can be obtained in that case as well.
  • the image encoding apparatus 400 can make it possible to apply the adaptive primary transform also to the chrominance, and further, can set the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance to the value of the adaptive primary transform flag apt_flag [Y] of luminance in a case in which a chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag indicating whether the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance is inferred on the basis of the adaptive primary transform flag apt_flag [Y] of luminance is 1 (true), for example, and can set the value of the adaptive primary transform flag apt_flag[Cb/Cr] of a chrominance to 0 (false) in a case in which the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag is 0 (false), as indicated in the row of No.
  • the encoding unit 415 determines whether or not a processing object is a chrominance in Step S 451 . Ina case in which the processing object is determined to be the chrominance, the process proceeds to Step S 452 . In Step S 452 , the encoding unit 415 determines whether or not the adaptive primary transform flag apt_flag[COMPONENT_Y] of luminance is true. In a case in which the flag is determined to be true, the process proceeds to Step S 453 . In Step S 453 , the encoding unit 415 determines whether or not the transform quantization bypass enabled flag transquant_bypass_enabled_flag is false.
  • Step S 454 the encoding unit 415 determines whether or not the transform skip flag ts_flag of the chrominance is false. In a case in which the transform skip flag ts_flag of the chrominance is determined to be false, the process proceeds to Step S 455 . In Step S 455 , the encoding unit 415 encodes the chrominance adaptive primary transform information inference flag chroma_apt_info_infer_flag. When the process of Step S 455 ends, the chrominance adaptive primary transform information inference flag encoding process ends, and the process returns to FIG. 54 .
  • the chrominance adaptive primary transform information inference flag encoding process ends, and the process returns to FIG. 54 .
  • the adaptive primary transform flag apt_flag[COMPONENT_Y] of luminance is determined to be false in Step S 452
  • the chrominance adaptive primary transform information inference flag encoding process ends, and the process returns to FIG. 54 .
  • the transform quantization bypass enabled flag transquant_bypass_enabled_flag is determined to be true in Step S 453
  • the chrominance adaptive primary transform information inference flag encoding process ends, and the process returns to FIG.
  • Step S 454 the chrominance adaptive primary transform information inference flag encoding process ends, and the process returns to FIG. 54 .
  • the image encoding apparatus 400 can make it possible to apply the adaptive primary transform also to the chrominance, and further can set the value of the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance to the value of the adaptive primary transform flag apt_flag[Y] of luminance in the case in which the size of the short side of the transform block to be processed of the chrominance is greater than or equal to a predetermined threshold value, for example, and can set the value of the adaptive primary transform flag apt_flag[Cb/Cr] of the chrominance to 0 (false) in a case in which the size of the short side of the transform block is smaller than the threshold value, as indicated in the row of No. #6 in the table of FIG. 7 . That is, similar effects to those in the case of the image decoding apparatus 100 can be obtained in that case as well.
  • the image encoding apparatus 400 can make it possible to apply the adaptive (inverse) primary transform also to the chrominance, and further, can set the transform type of the (inverse) primary horizontal transform to a predetermined transform type in a case in which the size of the picture width of a transform block with the chrominance to be processed is equal to or smaller than a predetermined threshold value, for example, and can set the transform type of the (inverse) primary horizontal transform on the basis of a horizontal transform set and a primary horizontal transform specifying flag in a case in which the size of the picture width of the transform block is greater than the threshold value, as indicated in the row of No. #7 in the table of FIG. 7 .
  • the transform type of the (inverse) primary vertical transform can be set to a predetermined transform type, and in a case in which the size of the picture height of the transform block is greater than the threshold value, the transform type of the (inverse) primary vertical transform can be set on the basis of a vertical transform set and a primary vertical transform specifying flag. That is, similar effects to those in the case of the image decoding apparatus 100 can be obtained in that case as well.
  • the plurality of above-described cases can be arbitrarily combined as well in the case of the image encoding apparatus 400 .
  • each of the above-described cases can also be combined with other cases that are not described above. That is, similar effects to those in the case of the image decoding apparatus 100 can be obtained in that case as well.
  • Information regarding an (inverse) orthogonal transform may include a transform skip flag ts_flag indicating whether an (inverse) orthogonal transform process is to be skipped, as in the case of decoding.
  • the transform skip flag ts_flag of a chrominance is inferred from the transform skip flag ts_flag of luminance at all times, for example, as indicated in the row of No. #1 in FIG. 27 .
  • a value of the transform skip flag ts_flag[compID] of a chrominance may be set to a value of the transform skip flag ts_flag[COMPONENT_Y] of luminance.
  • control unit 411 executes a transform skip flag encoding process and controls encoding of the transform skip flag ts_flag.
  • An example of the flow of the transform skip flag encoding process will be described with reference to the flowchart of FIG. 59 .
  • each of the processes of Step S 461 to Step S 464 of FIG. 59 is executed similarly to each of the processes of Step S 241 to Step S 244 of the transform skip flag derivation process of FIG. 29 .
  • Step S 464 the process proceeds to Step S 465 .
  • the transform skip flag encoding process ends.
  • the transform skip flag ts_flag[compID] of the chrominance can be derived from the transform skip flag ts_flag[COMPONENT_Y] of luminance likewise on the decoding side, and thus the flag is not signaled.
  • Step S 466 the control unit 411 sets an arbitrary value for the transform skip flag ts_flag[compID] of luminance. Then, the control unit 411 supplies the transform skip flag ts_flag[compID] of luminance to the encoding unit 415 to cause the flag to be encoded in Step S 467 .
  • the transform skip flag encoding process ends. That is, in this case, the transform skip flag ts_flag[compID] of luminance is signaled (transmitted).
  • the decoding side obtains the transform skip flag ts_flag[compID] of luminance through extraction and decoding using a bit stream.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in the case in which the prediction type of the coding block is intra prediction, the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • the transform skip flag encoding process ends.
  • the decoding side can similarly derive the transform skip flag ts_flag[compID] of the transform block in the case of the chrominance and inter prediction from the transform skip flag ts_flag[COMPONENT_Y] of luminance, and thus the flag is not signaled.
  • Step S 477 the control unit 411 sets an arbitrary value for the transform skip flag ts_flag[compID]. Then, in Step S 478 , the control unit 411 supplies the transform skip flag ts_flag[compID] to the encoding unit 415 to cause the flag to be encoded.
  • the transform skip flag encoding process ends.
  • the transform skip flag ts_flag[compID] of the transform block in the case of luminance or intra prediction is signaled (transmitted).
  • the decoding side can obtain the transform skip flag ts_flag[compID] through extraction and decoding using a bit stream.
  • the transform skip flag ts_flag of luminance and the transform skip flag ts_flag of the transform block of which the prediction mode is intra prediction are signaled, and the transform skip flag ts_flag of the chrominance of which the prediction mode is inter prediction is not signaled.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance
  • the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • Step S 487 the transform skip flag encoding process ends. That is, in this case, also in the case of intra prediction in which the prediction modes for luminance and the chrominance match each other as well as under the condition of ⁇ Modified example 1>, the tendencies of the residual signals are similar, and thus, the transform skip flag ts_flag[compID] is not signaled.
  • Step S 484 the control unit 411 sets an arbitrary value for the transform skip flag ts_flag[compID]. Then, in Step S 489 , the control unit 411 supplies the transform skip flag ts_flag[compID] to the encoding unit 415 to cause the flag to be encoded.
  • Step S 489 ends, the transform skip flag encoding process ends.
  • the transform skip flag ts_flag[compID] of luminance and the transform skip flag ts_flag[compID] of the transform block for intra prediction in which the prediction modes for luminance and the chrominance do not match each other are signaled (transmitted).
  • the decoding side can obtain the transform skip flag ts_flag[compID] through extraction and decoding using a bit stream.
  • the transform skip flag ts_flag of luminance and the transform skip flag ts_flag of the transform block for intra prediction in which the prediction modes for luminance and the chrominance do not match each other are signaled, and other transform skip flags ts_flag are not signaled.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[compID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance
  • the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • Step S 497 the transform skip flag encoding process ends. That is, in this case, also in the case of intra prediction of which the prediction mode is intra block copy as well as under the condition of ⁇ Modified example 1>, the tendencies of the residual signals are similar, and thus, the transform skip flag ts_flag[compID] is not signaled.
  • Step S 498 the control unit 411 sets an arbitrary value for the transform skip flag ts_flag[compID]. Then, in Step S 499 , the control unit 411 supplies the transform skip flag ts_flag[compID] to the encoding unit 415 to cause the flag to be encoded.
  • the transform skip flag encoding process ends.
  • the transform skip flag ts_flag[compID] of luminance and the transform skip flag ts_flag[compID] of the transform block for intra prediction of which the prediction mode is not intra block copy are signaled (transmitted).
  • the decoding side can obtain the transform skip flag ts_flag[compID] through extraction and decoding using a bit stream.
  • the transform skip flag ts_flag of luminance and the transform skip flag ts_flag of the transform block for intra prediction of which the prediction mode is not intra block copy are signaled, and other transform skip flags ts_flag are not signaled.
  • transmission (encoding and decoding) of the transform skip flag ts_flag of the chrominance can be omitted only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the transform skip flag ts_flag[copmID] of the chrominance may be set to the value of the transform skip flag ts_flag[COMPONENT_Y] of luminance, and in the case in which the chrominance transform skip information inference flag chroma_ts_info_infer_flag is 0 (false), the transform skip flag ts_flag[compID] of the chrominance may be signaled (i.e., the transform skip flag ts_flag[compID] of the chrominance can be decoded from the encoded data).
  • the transform skip flag encoding process ends.
  • Step S 506 the control unit 411 sets an arbitrary value for the transform skip flag ts_flag[compID]. Then, the control unit 411 supplies the transform skip flag ts_flag[compID] to the encoding unit 415 and causes the flag to be encoded in Step S 507 .
  • the transform skip flag encoding process ends.
  • a secondary transform identifier st_idx indicating which (inverse) secondary transform is to be applied may be included in information regarding an (inverse) orthogonal transform.
  • a secondary transform identifier chroma_st_idx of a chrominance common for Cb and Cr
  • a secondary transform identifier st_idx[Y] of luminance for example, an increase in overhead of the code amount of the syntax by which the chrominance (Cb or Cr) is signaled can be curbed as well.
  • a secondary transform identifier st_idx of a chrominance may be inferred from a secondary transform identifier st_idx of luminance at all times as indicated in the row of No. #1 of FIG. 39 .
  • the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of secondary transform identifier st_idx of luminance.
  • control unit 411 executes the secondary transform identifier encoding process for the chrominance and controls encoding of the secondary transform identifier chroma_st_idx for the chrominance.
  • An example of the flow of the secondary transform identifier encoding process for the chrominance will be described with reference to the flowchart of FIG. 64 .
  • each of the processes of Step S 511 to Step S 514 of FIG. 64 is executed similarly to each of the processes of Step S 291 to Step S 294 of the chrominance secondary transform identifier encoding process of FIG. 42 .
  • transmission (encoding and decoding) of the secondary transform identifier chroma_ts_flag of the chrominance can be omitted, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance as described with reference to the row of No. #2 of FIG. 39 , and in the case in which the prediction type of the coding block is intra prediction, the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • control unit 411 executes the secondary transform identifier encoding process for the chrominance and controls encoding of the secondary transform identifier chroma_st_idx of the chrominance.
  • An example of the flow of the secondary transform identifier encoding process for the chrominance will be described with reference to the flowchart of FIG. 65 .
  • each of the processes of Step S 521 to Step S 524 of FIG. 65 is executed similarly to each of the processes of Step S 301 to Step S 304 of the chrominance secondary transform identifier encoding process of FIG. 44 .
  • Step S 525 the secondary transform identifier encoding process for the chrominance ends.
  • the decoding side can similarly derive the secondary transform identifier chroma_st_idx of the transform block in the case of the chrominance and inter prediction from the secondary transform identifier st_idx of luminance, and thus the identifier is not signaled.
  • Step S 526 the control unit 411 sets an arbitrary value for the secondary transform identifier chroma_st_idx of the chrominance. Then, in Step S 527 , the control unit 411 supplies the secondary transform identifier chroma_st_idx of the chrominance to the encoding unit 415 to cause the identifier to be encoded.
  • Step S 527 the secondary transform identifier encoding process for the chrominance ends.
  • the secondary transform identifier chroma_st_idx of the chrominance of the transform block for intra prediction is signaled (transmitted).
  • the decoding side can obtain the secondary transform identifier chroma_st_idx of the chrominance through extraction and decoding using a bit stream.
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted only in the case of the inter prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance is set to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • Step S 531 to Step S 535 is executed similarly to each of the processes of Step S 311 to Step S 315 of FIG. 46 .
  • the tendencies of the residual signals are similar, and thus the secondary transform identifier chroma_st_idx of the chrominance may not be signaled.
  • Step S 537 the control unit 411 sets an arbitrary value for secondary transform identifier chroma_st_idx of the chrominance. Then, in Step S 538 , the control unit 411 supplies the secondary transform identifier chroma_st_idx of the chrominance to the encoding unit 415 to cause the identifier to be encoded.
  • Step S 538 the secondary transform identifier encoding process for the chrominance ends.
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance is set to the value of the secondary transform identifier st_idx of luminance.
  • the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • Step S 541 to Step S 545 is executed similarly to each of the processes of Step S 321 to Step S 325 of FIG. 48 .
  • the secondary transform identifier encoding process for the chrominance ends.
  • the tendencies of the residual signals are similar, and thus the secondary transform identifier chroma_st_idx of the chrominance may not be signaled.
  • Step S 547 the control unit 411 sets an arbitrary value for secondary transform identifier chroma_st_idx of the chrominance.
  • Step S 548 the control unit 411 supplies the secondary transform identifier chroma_st_idx of the chrominance to the encoding unit 415 to cause the identifier to be encoded.
  • transmission (encoding and decoding) of the secondary transform identifier chroma_st_idx of the chrominance can be omitted only in the case of the prediction mode in which the tendencies of the residual signals are similar, and thus deterioration in coding efficiency can be curbed.
  • an increase in a load of encoding and decoding can be curbed as well.
  • the value of the secondary transform identifier chroma_st_idx of the chrominance may be set to the value of the secondary transform identifier st_idx of luminance, and in the case in which the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is 0 (false), the secondary transform identifier chroma_st_idx of the chrominance may be signaled (i.e., the secondary transform identifier chroma_st_idx of the chrominance can be decoded from the encoded data).
  • Step S 551 to Step S 553 is executed similarly to each of the processes of Step S 331 to Step S 333 of FIG. 50 .
  • control unit 411 sets the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag in Step S 554 and supplies the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag to the encoding unit 415 so as to be encoded in Step S 555 .
  • Step S 556 the control unit 411 determines whether or not the value of the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is true. In a case in which the value of the chrominance secondary transform identifier inference flag chroma_st_idx_infer_flag is determined to be true, the process proceeds to Step S 557 . In Step S 557 , the control unit 411 applies the secondary transform identifier st_idx for luminance to the secondary transform identifier chroma_st_idx of the chrominance.
  • Step S 558 the control unit 411 sets the secondary transform identifier chroma_st_idx of the chrominance.
  • Step S 559 the control unit 411 supplies the identifier to the encoding unit 415 to cause the identifier to be encoded.
  • a data unit by which information regarding an image and information regarding encoding and decoding of the image described above are set is arbitrary in each case and not limited to the above-described examples.
  • the information may be set in each of a transform unit (TU), a transform block (TB), a prediction unit (PU), a prediction block (PB), a coding unit (CU), a largest coding unit (LCU), a sub-block, a block, a tile, a slice, a picture, a sequence, or a component, or data of the data units may be targeted.
  • a data unit is set for each piece of information. That is, all pieces of information may not be set (or targeted) in the same data unit.
  • a storage place of information is arbitrary, and information may be stored in the header of the above-described data unit, a parameter set, or the like. In addition, information may be stored in a plurality of places.
  • Control information related to the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side.
  • control information that controls whether or not to allow (or deny) the application of the present technology described above may be transmitted.
  • control information that specifies an upper limit, a lower limit, or both of a block size that allows (or denies) the application of the present technology described above may be transmitted.
  • the present technology can be applied to arbitrary image encoding and decoding in which a primary transform and a secondary transform (an inverse secondary change and an inverse primary transform) are performed. That is, specifications of a transform (inverse transform), quantization (inverse quantization), encoding (decoding), prediction, and the like are arbitrary and are not limited to the above-described examples. For example, with respect to a transform (inverse transform), (inverse) transforms (i.e., three or more (inverse) transforms) other than an (inverse) primary transform and an (inverse) secondary transform may be performed.
  • encoding (decoding) may be of a reversible method or an irreversible method.
  • quantization (inverse quantization), prediction, or the like may be omitted.
  • a process which is not described above such as a filtering process may be performed.
  • the system, apparatus, processor, and the like to which the present technology is applied can be used in any fields, such as traffic, medical care, crime prevention, agriculture, livestock industry, mining industry, beauty, factory, household appliance, weather, natural surveillance, or the like.
  • the present technology is applicable to systems and devices that transmit an image provided for viewing.
  • the present technology is applicable to systems and devices provided for traffic.
  • the present technology is applicable to systems and devices provided for security.
  • the present technology is applicable to systems and devices provided for sports.
  • the present technology is applicable to systems and devices provided for agriculture.
  • the present technology is applicable to systems and devices provided for livestock industry.
  • the present technology is also applicable to systems and devices for monitoring conditions of natural such as volcanoes, forests, oceans, and the like.
  • the present technology is applicable to weather observation systems and weather observation devices for observing weather, temperature, humidity, wind speed, sunshine duration, and the like. Furthermore, in one example, the present technology is also applicable to systems, devices, or the like for observing ecology of wildlife such as birds, fish, reptiles, amphibians, mammals, insects, plants, and the like.
  • the series of processes described above can be applied to a multi-view image encoding and decoding system that performs encoding and decoding of multi-view images including images with a plurality of views (view).
  • the present technology may be applied to encoding and decoding of each view (view).
  • the series of processes described above can be applied to a hierarchical image encoding (scalable encoding) and decoding system that performs encoding and decoding on hierarchical images that have been made into a plurality of layers (hierarchies) to have a scalability function with respect to a predetermined parameter.
  • the present technology may be applied to encoding and decoding of each of the hierarchies (layers).
  • the series of processes described above can be executed by hardware, and can also be executed in software.
  • a program forming the software is installed on a computer.
  • the term computer includes a computer built into special-purpose hardware, a computer able to execute various functions by installing various programs thereon, such as a general-purpose personal computer, for example, and the like.
  • FIG. 69 is a block diagram illustrating an exemplary hardware configuration of a computer that executes the series of processes described above according to a program.
  • a central processing unit (CPU) 801 read-only memory (ROM) 802 , and random access memory (RAM) 803 are interconnected through a bus 804 .
  • CPU central processing unit
  • ROM read-only memory
  • RAM random access memory
  • an input/output interface 810 is also connected to the bus 804 .
  • An input unit 811 , an output unit 812 , a storage unit 813 , a communication unit 814 , and a drive 815 are connected to the input/output interface 810 .
  • the input unit 811 includes a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like, for example.
  • the output unit 812 includes a display, a speaker, an output terminal, and the like, for example.
  • the storage unit 813 includes a hard disk, a RAM disk, non-volatile memory, and the like, for example.
  • the communication unit 814 includes a network interface, for example.
  • the drive 815 drives a removable medium 821 such as a magnetic disk, an optical disc, a magneto-optical disc, or semiconductor memory.
  • the series of processes described above are performed by having the CPU 801 load a program stored in the storage unit 813 into the RAM 803 via the input/output interface 810 and the bus 804 , and execute the program, for example. Additionally, data required for the CPU 801 to execute various processes and the like is also stored in the RAM 803 as appropriate.
  • the program executed by the computer may be applied by being recorded onto the removable medium 821 as an instance of packaged media or the like, for example.
  • the program may be installed in the storage unit 813 via the input/output interface 810 by inserting the removable medium 821 into the drive 815 .
  • the program may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program may be received by the communication unit 814 and installed in the storage unit 813 .
  • the program may also be preinstalled in the ROM 802 or the storage unit 813 .
  • the image encoding apparatus 400 and the image decoding apparatus 100 can be applied to, for example, various electronic apparatuses such as: transmitters or receivers for satellite broadcasting, wired broadcasting such as cable TV, distribution on the Internet and distribution to terminals through cellular communication; recording devices which record images on media such as optical discs, magnetic disks, and flash memories; or reproduction devices which reproduce images from the foregoing storage media.
  • various electronic apparatuses such as: transmitters or receivers for satellite broadcasting, wired broadcasting such as cable TV, distribution on the Internet and distribution to terminals through cellular communication; recording devices which record images on media such as optical discs, magnetic disks, and flash memories; or reproduction devices which reproduce images from the foregoing storage media.
  • FIG. 70 illustrates an example of a schematic configuration of a television apparatus to which the above-described embodiment is applied.
  • the television apparatus 900 has an antenna 901 , a tuner 902 , a demultiplexer 903 , a decoder 904 , a video signal processing unit 905 , a display unit 906 , an audio signal processing unit 907 , a speaker 908 , an external interface (I/F) unit 909 , a control unit 910 , a user interface (I/F) unit 911 , and a bus 912 .
  • I/F external interface
  • control unit 910 control unit 910
  • I/F user interface
  • the tuner 902 extracts a signal of a desired channel from a broadcasting signal received via the antenna 901 and demodulates the extracted signal. Then, the tuner 902 outputs an encoded bit stream obtained from the demodulation to the demultiplexer 903 . That is, the tuner 902 plays a role as a transmission section of the television apparatus 900 which receives an encoded stream in which images are encoded.
  • the demultiplexer 903 demultiplexes a video stream and an audio stream of a program to be viewed from the encoded stream and outputs the demultiplexed streams to the decoder 904 .
  • the demultiplexer 903 extracts auxiliary data such as an electronic program guide (EPG) from the encoded bit stream and supplies the extracted data to the control unit 910 .
  • EPG electronic program guide
  • the demultiplexer 903 may perform descrambling.
  • the decoder 904 decodes the video stream and the audio stream input from the demultiplexer 903 . Then, the decoder 904 outputs video data generated from the decoding process to the video signal processing unit 905 . In addition, the decoder 904 outputs audio data generated from the decoding process to the audio signal processing unit 907 .
  • the video signal processing unit 905 reproduces the video data input from the decoder 904 to cause the display unit 906 to display a video.
  • the video signal processing unit 905 may cause the display unit 906 to display an application screen supplied via a network.
  • the video signal processing unit 905 may perform an additional process, for example, noise reduction, on the video data in accordance with a setting.
  • the video signal processing unit 905 may generate an image of a graphical user interface (GUI), for example, a menu, a button, or a cursor and superimpose the generated image on an output image.
  • GUI graphical user interface
  • the display unit 906 is driven with a driving signal supplied from the video signal processing unit 905 and displays a video or an image on a video plane of a display device (e.g., a liquid crystal display, a plasma display, an organic electroluminescence display (OLED), etc.).
  • a display device e.g., a liquid crystal display, a plasma display, an organic electroluminescence display (OLED), etc.
  • the audio signal processing unit 907 performs a reproduction process including D/A conversion and amplification on the audio data input from the decoder 904 and causes the speaker 908 to output a sound.
  • the audio signal processing unit 907 may perform an additional process such as noise removal on the audio data.
  • the external interface unit 909 is an interface for connecting the television apparatus 900 to an external apparatus or a network.
  • a video stream or an audio stream received via the external interface unit 909 may be decoded by the decoder 904 .
  • the external interface unit 909 also plays the role as a transmission sections of the television apparatus 900 which receives an encoded stream in which images are encoded.
  • the control unit 910 has a processor such as a CPU and a memory such as a RAM and a ROM.
  • the memory stores a program executed by the CPU, program data, EPG data, and data acquired via a network.
  • the program stored in the memory is read and executed by the CPU at the time of, for example, start-up of the television apparatus 900 .
  • the CPU controls operations of the television apparatus 900 by executing the program in response to, for example, operation signals input from the user interface section 911 .
  • the user interface section 911 is connected to the control unit 910 .
  • the user interface section 911 includes, for example, buttons and switches with which a user operates the television apparatus 900 , a reception unit for remote control signals, and the like.
  • the user interface section 911 generates an operation signal by detecting an operation by a user via any aforementioned constituent element and outputs the generated operation signal to the control unit 910 .
  • the bus 912 connects the tuner 902 , the demultiplexer 903 , the decoder 904 , the video signal processing unit 905 , the audio signal processing unit 907 , the external interface unit 909 , and the control unit 910 to one another.
  • the decoder 904 may also include the functions of the image decoding apparatus 100 described above. In other words, the decoder 904 may be configured to decode encoded data according to the method described in each of the above embodiments. In this manner, the television apparatus 900 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • the video signal processing unit 905 may be able to encode image data provided from the decoder 904 , and cause the obtained encoded data to be output externally to the television apparatus 900 through external interface unit 909 .
  • the video signal processing unit 905 may also include the functions of the image encoding apparatus 400 described above.
  • the video signal processing unit 905 may be configured to encode image data provided from the decoder 904 according to the method described in each of the above embodiments.
  • the television apparatus 900 becomes able to further decrease the amount of transmission related to the transmission (transmission) of parameters. In this manner, the television apparatus 900 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • FIG. 71 illustrates an example of a schematic configuration of a mobile telephone to which the above-described embodiments are applied.
  • a mobile telephone 920 includes an antenna 921 , a communication unit 922 , an audio codec 923 , a speaker 924 , a microphone 925 , a camera unit 926 , an image processing unit 927 , a multiplexing/demultiplexing unit 928 , a recording/reproducing unit 929 , a display unit 930 , a control unit 931 , an operation unit 932 , and a bus 933 .
  • the antenna 921 is connected to the communication unit 922 .
  • the speaker 924 and the microphone 925 are connected to the audio codec 923 .
  • the operation unit 932 is connected to the control unit 931 .
  • the bus 933 mutually connects the communication unit 922 , the audio codec 923 , the camera unit 926 , the image processing unit 927 , the multiplexing/demultiplexing unit 928 , the recording/reproducing unit 929 , the display unit 930 , and the control unit 931 .
  • the mobile telephone 920 performs actions such as transmitting/receiving an audio signal, transmitting/receiving an electronic mail or image data, capturing an image, and recording data in various operation modes including an audio call mode, a data communication mode, a photography mode, and a videophone mode.
  • an analog audio signal generated by the microphone 925 is supplied to the audio codec 923 .
  • the audio codec 923 then converts the analog audio signal into audio data, performs A/D conversion on the converted audio data, and compresses the data.
  • the audio codec 923 thereafter outputs the compressed audio data to the communication unit 922 .
  • the communication unit 922 encodes and modulates the audio data to generate a transmission signal.
  • the communication unit 922 then transmits the generated transmission signal to a base station (not shown) through the antenna 921 .
  • the communication unit 922 amplifies a radio signal received through the antenna 921 , performs frequency conversion, and acquires a reception signal.
  • the communication unit 922 thereafter demodulates and decodes the reception signal to generate the audio data and output the generated audio data to the audio codec 923 .
  • the audio codec 923 expands the audio data, performs D/A conversion on the data, and generates the analog audio signal.
  • the audio codec 923 then supplies the generated audio signal to the speaker 924 to cause it to output the audio.
  • the control unit 931 In the data communication mode, for example, the control unit 931 generates character data configuring an electronic mail, in accordance with a user operation detected through the operation unit 932 .
  • the control unit 931 further displays characters on the display unit 930 .
  • the control unit 931 generates electronic mail data in accordance with an instruction to send it obtained from a user through the operation unit 932 and outputs the generated electronic mail data to the communication unit 922 .
  • the communication unit 922 encodes and modulates the electronic mail data to generate a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to the base station (not shown) through the antenna 921 .
  • the communication unit 922 further amplifies a radio signal received through the antenna 921 , performs frequency conversion, and acquires a reception signal.
  • the communication unit 922 thereafter demodulates and decodes the reception signal, restores the electronic mail data, and outputs the restored electronic mail data to the control unit 931 .
  • the control unit 931 displays the content of the electronic mail on the display unit 930 as well as supplies the electronic mail data to a storage medium of the recording/reproducing unit 929 to cause the data to be recorded in the medium.
  • the recording/reproducing unit 929 includes an arbitrary storage medium that is readable and writable.
  • the storage medium may be a built-in storage medium such as a RAM or a flash memory, or may be an externally-mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.
  • the camera unit 926 images an object to generate image data and outputs the generated image data to the image processing unit 927 .
  • the image processing unit 927 encodes the image data input from the camera unit 926 and supplies an encoded stream to the storage medium of the recording/reproducing unit 929 to cause the encoded stream to be recorded in the medium.
  • the recording/reproducing unit 929 reads out an encoded stream recorded on a storage medium, and outputs to the image processing unit 927 .
  • the image processing unit 927 decodes the encoded stream input from the recording/reproducing unit 929 , supplies image data to the display unit 930 , and causes the image to be displayed.
  • the multiplexing/demultiplexing unit 928 multiplexes a video stream encoded by the image processing unit 927 and an audio stream input from the audio codec 923 , and outputs the multiplexed stream to the communication unit 922 .
  • the communication unit 922 encodes and modulates the stream to generate a transmission signal.
  • the communication unit 922 then transmits the generated transmission signal to the base station (not shown) through the antenna 921 .
  • the communication unit 922 amplifies a radio signal received through the antenna 921 , performs frequency conversion, and acquires a reception signal.
  • the transmission signal and the reception signal can include an encoded bit stream.
  • the communication unit 922 thus demodulates and decodes the reception signal to restore the stream, and outputs the restored stream to the multiplexing/demultiplexing unit 928 .
  • the multiplexing/demultiplexing unit 928 demultiplexes the video stream and the audio stream from the input stream and outputs the video stream and the audio stream to the image processing unit 927 and the audio codec 923 , respectively.
  • the image processing unit 927 decodes the video stream to generate video data.
  • the video data is then supplied to the display unit 930 , which displays a series of images.
  • the audio codec 923 expands and performs D/A conversion on the audio stream to generate an analog audio signal.
  • the audio codec 923 then supplies the generated audio signal to the speaker 924 to cause it to output the audio.
  • the image processing unit 927 may include the functions of the image encoding apparatus 400 described above, for example. In other words, the image processing unit 927 may be configured to encode image data according to the method described in each of the above embodiments. In this manner, the mobile phone 920 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • the image processing unit 927 may include the functions of the image decoding apparatus 100 described above, for example.
  • the image processing unit 927 may be configured to decode encoded data according to the method described in each of the above embodiments. In this manner, the mobile phone 920 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • FIG. 72 illustrates an example of a schematic configuration of a recording/reproducing apparatus to which the above-described embodiments are applied.
  • the recording/reproducing apparatus 940 encodes audio data and video data of a received broadcast program and records the data into a recording medium, for example.
  • the recording/reproducing apparatus 940 may also encode audio data and video data acquired from another apparatus and record the data into the recording medium, for example.
  • the recording/reproducing apparatus 940 reproduces the data recorded in the recording medium on a monitor and a speaker, for example, in response to a user instruction. In this case, recording/reproducing apparatus 940 decodes the audio data and the video data.
  • the recording/reproducing apparatus 940 includes a tuner 941 , an external interface unit 942 , an encoder 943 , a hard disk drive (HDD) 944 , a disk drive 945 , a selector 946 , a decoder 947 , an on-screen display (OSD) unit 948 , a control unit 949 , and a user interface unit 950 .
  • a tuner 941 an external interface unit 942
  • an encoder 943 e.g., a hard disk drive (HDD) 944 , a disk drive 945 , a selector 946 , a decoder 947 , an on-screen display (OSD) unit 948 , a control unit 949 , and a user interface unit 950 .
  • HDD hard disk drive
  • OSD on-screen display
  • the tuner 941 extracts a signal of a desired channel from a broadcast signal received through an antenna (not shown) and demodulates the extracted signal.
  • the tuner 941 then outputs an encoded bit stream obtained by the demodulation to the selector 946 . That is, the tuner 941 has a role as a transmission unit in the recording/reproducing apparatus 940 .
  • the external interface unit 942 is an interface which connects the recording/reproducing apparatus 940 with an external device or a network.
  • the external interface unit 942 may be, for example, an institute of electrical and electronic engineers (IEEE) 1394 interface, a network interface, a USB interface, or a flash memory interface.
  • IEEE institute of electrical and electronic engineers
  • the video data and the audio data received through the external interface unit 942 are input to the encoder 943 , for example. That is, the external interface unit 942 has a role as a transmission unit in the recording/reproducing apparatus 940 .
  • the encoder 943 encodes the video data and the audio data in the case where the video data and the audio data input from the external interface unit 942 are not encoded.
  • the encoder 943 thereafter outputs an encoded bit stream to the selector 946 .
  • the HDD unit 944 records, into an internal hard disk, the encoded bit stream in which content data such as video and audio is compressed, various programs, and other data.
  • the HDD unit 944 reads these data from the hard disk when the video and the audio are reproduced.
  • the disk drive 945 records and reads data into/from a recording medium attached to the disk drive.
  • the recording medium attached to the disk drive 945 may be, for example, a digital versatile disc (DVD) disc (such as DVD-Video, DVD-random access memory (DVD-RAM), DVD-recordable (DVD-R), DVD-rewritable (DVD-RW), DVD+recordable (DVD+R), or DVD+rewritable (DVD+RW)) or a Blu-ray (Registered Trademark) disk.
  • DVD digital versatile disc
  • the selector 946 selects the encoded bit stream input from the tuner 941 or the encoder 943 when recording the video and audio, and outputs the selected encoded bit stream to the HDD unit 944 or the disk drive 945 .
  • the selector 946 outputs the encoded bit stream input from the HDD unit 944 or the disk drive 945 to the decoder 947 .
  • the decoder 947 decodes the encoded bit stream to generate the video data and the audio data.
  • the decoder 947 then outputs the generated video data to the OSD unit 948 and the generated audio data to an external speaker.
  • the OSD unit 948 reproduces the video data input from the decoder 947 and displays the video.
  • the OSD unit 948 may also superpose an image of a GUI such as a menu, buttons, or a cursor onto the displayed video.
  • the control unit 949 includes a processor such as a CPU and a memory such as a RAM and a ROM.
  • the memory stores a program executed by the CPU as well as program data.
  • the program stored in the memory is read by the CPU at the start-up of the recording/reproducing apparatus 940 and executed, for example.
  • the CPU controls the operation of the recording/reproducing apparatus 940 in accordance with an operation signal that is input from the user interface unit 950 , for example.
  • the user interface unit 950 is connected to the control unit 949 .
  • the user interface unit 950 includes a button and a switch for a user to operate the recording/reproducing apparatus 940 as well as a reception part which receives a remote control signal, for example.
  • the user interface unit 950 detects a user operation through these components to generate an operation signal, and outputs the generated operation signal to the control unit 949 .
  • the encoder 943 may include the functions of the image encoding apparatus 400 described above, for example. In other words, the encoder 943 may be configured to encode image data according to the method described in each of the above embodiments. In this manner, the recording/reproducing apparatus 940 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • the decoder 947 may include the functions of the image decoding apparatus 100 described above, for example. In other words, the decoder 947 may be configured to decode encoded data according to the method described in each of the above embodiments. In this manner, the recording/reproducing apparatus 940 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • FIG. 73 illustrates an example of a schematic configuration of an imaging apparatus to which the above-described embodiments are applied.
  • the imaging apparatus 960 images an object to generate an image, encodes image data, and records the data into a recording medium.
  • the imaging apparatus 960 includes an optical block 961 , an imaging unit 962 , a signal processing unit 963 , an image processing unit 964 , a display unit 965 , an external interface unit 96 , a memory unit 967 , a media drive 968 , an OSD unit 969 , a control unit 970 , a user interface unit 971 , and a bus 972 .
  • the optical block 961 is connected to the imaging unit 962 .
  • the imaging unit 962 is connected to the signal processing unit 963 .
  • the display unit 965 is connected to the image processing unit 964 .
  • the user interface unit 971 is connected to the control unit 970 .
  • the bus 972 mutually connects the image processing unit 964 , the external interface unit 966 , the memory unit 967 , the media drive 968 , the OSD unit 969 , and the control unit 970 .
  • the optical block 961 includes a focus lens and a diaphragm mechanism.
  • the optical block 961 forms an optical image of an object on an imaging plane of the imaging unit 962 .
  • the imaging unit 962 includes an image sensor such as a Charge Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS) and performs photoelectric conversion to convert the optical image formed on the imaging plane into an image signal as an electric signal. Then, the imaging unit 962 outputs the image signal to the signal processing unit 963 .
  • CCD Charge Coupled Device
  • CMOS Complementary Metal Oxide Semiconductor
  • the signal processing unit 963 performs various camera signal processes such as a knee correction, a gamma correction and a color correction on the image signal input from the imaging unit 962 .
  • the signal processing unit 963 outputs the image data, on which the camera signal processes have been performed, to the image processing unit 964 .
  • the image processing unit 964 encodes the image data input from the signal processing unit 963 and generates the encoded data.
  • the image processing unit 964 then outputs the generated encoded data to the external interface unit 966 or the media drive 968 .
  • the image processing unit 964 also decodes the encoded data input from the external interface unit 966 or the media drive 968 to generate image data.
  • the image processing unit 964 then outputs the generated image data to the display unit 965 .
  • the image processing unit 964 may output to the display unit 965 the image data input from the signal processing unit 963 to cause the display unit 965 to display the image.
  • the image processing unit 964 may superpose display data acquired from the OSD unit 969 onto the image that is output on the display unit 965 .
  • the OSD unit 969 generates an image of a GUI such as a menu, buttons, or a cursor and outputs the generated image to the image processing unit 964 .
  • the external interface unit 966 is configured as a USB input/output terminal, for example.
  • the external interface unit 966 connects the imaging apparatus 960 with a printer when printing an image, for example.
  • a drive is connected to the external interface unit 96 as needed.
  • a removable medium such as a magnetic disk or an optical disk is attached to the drive, for example, so that a program read from the removable medium can be installed to the imaging apparatus 960 .
  • the external interface unit 966 may also be configured as a network interface that is connected to a network such as a LAN or the Internet. That is, the external interface unit 966 has a role as a transmission unit in the imaging apparatus 960 .
  • the recording medium attached to the media drive 968 may be an arbitrary removable medium that is readable and writable such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Furthermore, the recording medium may be attached to the media drive 968 in a fixed manner so that a non-transportable storage unit such as a built-in hard disk drive or a solid state drive (SSD) is configured, for example.
  • a non-transportable storage unit such as a built-in hard disk drive or a solid state drive (SSD) is configured, for example.
  • the control unit 970 includes a processor such as a CPU and a memory such as a RAM and a ROM.
  • the memory stores a program executed by the CPU as well as program data.
  • the program stored in the memory is read by the CPU at the start-up of the imaging apparatus 960 and then executed.
  • the CPU controls the operation of the imaging apparatus 960 in accordance with an operation signal that is input from the user interface unit 971 , for example.
  • the user interface unit 971 is connected to the control unit 970 .
  • the user interface unit 971 includes buttons and switches for a user to operate the imaging apparatus 960 , for example.
  • the user interface unit 971 detects a user operation through these components to generate an operation signal, and outputs the generated operation signal to the control unit 970 .
  • the image processing unit 964 may include the functions of the image encoding apparatus 400 described above, for example. In other words, the image processing unit 964 may be configured to encode image data according to the method described in each of the above embodiments. In this manner, the imaging apparatus 960 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • the image processing unit 964 may include the functions of the image decoding apparatus 100 described above, for example.
  • the image processing unit 964 may be configured to decode encoded data according to the method described in each of the above embodiments. In this manner, the imaging apparatus 960 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • the present technology may also be implemented as any kind of configuration installed in any apparatus or an apparatus included in a system, such as a processor provided as a large-scale integration (LSI) chip or the like, a module that uses multiple processors or the like, a unit that uses multiple modules or the like, a set that further adds other functions to a unit (i.e., a configuration of a part of an apparatus), or the like.
  • FIG. 74 illustrates one example of a schematic configuration of a video set applying the present technology.
  • the video set 1300 illustrated in FIG. 74 is such a multifunctional configuration, and is a combination of a device that includes functions related to image encoding and decoding (either one, or both) with a device that includes other functions related to such functions.
  • the video set 1300 includes a module group such as a video module 1311 , external memory 1312 , a power management module 1313 , and a front-end module 1314 , and a device that includes related functions such as connectivity 1321 , a camera 1322 , and a sensor 1323 .
  • a module group such as a video module 1311 , external memory 1312 , a power management module 1313 , and a front-end module 1314 , and a device that includes related functions such as connectivity 1321 , a camera 1322 , and a sensor 1323 .
  • a module is a part that collects several interrelated partial functions into a unified function.
  • the specific physical configuration may be any configuration, but for example, it is conceivable to dispose and integrate multiple processors with respective functions, electronic circuit elements such as resistors and capacitors, other devices, and the like onto a circuit board or the like. It is also conceivable to combine a module with another module, processor, or the like to create a new module.
  • the video module 1311 is a combination of configurations that include functions related to image processing, and includes an application processor, a video processor, a broadband modem 1333 , and an RF module 1334 .
  • the processor is an integration of configurations having predetermined functions into a semiconductor chip as a system on a chip (SoC), and may also be designated a large-scale integration (LSI) chip or the like, for example.
  • the configurations having predetermined functions may be logic circuits (hardware configurations), but may also be a CPU, ROM, RAM, and the like as well as a program executed using these (software configurations), and may also be a combination of both.
  • a processor may include logic circuits and CPU, ROM, RAM, and the like, and may be configured to realize a subset of the functions with the logic circuits (hardware configurations) while realizing other functions with programs (software configurations) executed on the CPU.
  • the application processor 1331 in FIG. 74 is a processor that executes an application related to image processing. To realize a predetermined function, the application executed in the application processor 1331 is able to not only execute computational processing, but is also able to control configurations inside and outside the video module 1311 , such as the video processor 1332 , for example, as necessary.
  • the video processor 1332 is a processor that includes functions related to image encoding/decoding (either one, or both).
  • the broadband modem 1333 performs digital modulation and the like to convert data (a digital signal) transmitted by wired or wireless (or both) broadband communication performed over a broadband connection such as the Internet or the public telephone network into an analog signal, and also performs demodulation to convert an analog signal received by such broadband communication into data (a digital signal).
  • the broadband modem 1333 processes any kind of information, such as image data processed by the video processor 1332 , a stream in which image data is encoded, application programs, and settings data, for example.
  • the RF module 1334 is a module that performs frequency conversion, modulation/demodulation, amplification, filter processing, and the like on radio frequency (RF) signals transmitted and received through an antenna.
  • the RF module 1334 generates an RF signal by performing frequency conversion and the like on a baseband signal generated by the broadband modem 1333 .
  • the RF module 1334 generates a baseband signal by performing frequency conversion and the like on an RF signal received via the front-end module 1314 .
  • the application processor 1331 and the video processor 1332 may also be unified and configured as a single processor.
  • the external memory 1312 is a module provided externally to the video module 1311 that includes a storage device utilized by the video module 1311 .
  • the storage device of the external memory 1312 may be realized by any kind of physical configuration, but since the storage device typically is used to store large amounts of data such as image data in units of frames, it is desirable to realize the storage device with relatively inexpensive and high-capacity semiconductor memory such as dynamic random access memory (DRAM), for example.
  • DRAM dynamic random access memory
  • the power management module 1313 manages and controls the supply of power to the video module 1311 (each configuration inside the video module 1311 ).
  • the front-end module 1314 is a module that provides a front-end function (a circuit on the antenna-side transmit/receive port) to the RF module 1334 . As illustrated in FIG. 74 , the front-end module 1314 includes an antenna unit 1351 , a filter 1352 , and an amplification unit 1353 , for example.
  • the antenna unit 1351 includes an antenna that transmits and receives wireless signals, and a peripheral configuration thereof.
  • the antenna unit 1351 transmits a signal supplied from the amplification unit 1353 as a wireless signal, and supplies a received wireless signal to the filter 1352 as an electric signal (RF signal).
  • the filter 1352 performs filter processing and the like on the RF signal received through the antenna unit 1351 , and supplies the processed RF signal to the RF module 1334 .
  • the amplification unit 1353 amplifies and supplies the RF signal supplied from the RF module 1334 to the antenna unit 1351 .
  • the connectivity 1321 is a module that includes functions related to external connections.
  • the physical configuration of the connectivity 1321 may be any configuration.
  • the connectivity 1321 includes a configuration having a communication function other than the communication standard supporting by the broadband modem 1333 , an external input/output terminal, or the like.
  • the connectivity 1321 may include a module having a communication function conforming to a wireless communication standard such as Bluetooth (registered trademark), EEE 802.11 (for example, Wireless Fidelity (Wi-Fi (registered trademark))), near field communication (NFC), or Infrared Data Association (IrDA), and an antenna or the like that transmits and receives signals conforming to the standard.
  • the connectivity 1321 may include a module having a communication function conforming to a wired communication function such as Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) (registered trademark), and a port conforming to the standard.
  • the connectivity 1321 may include a function of transmitting another kind of data (signal), such as an analog input/output terminal.
  • the connectivity 1321 may include the transmission destination device of the data (signal).
  • the connectivity 1321 may include a drive (not only a drive for removable media, but also including a hard disk, a solid-state drive (SSD), network-attached storage (NAS), and the like) that reads and writes data with respect to a recording medium such as a magnetic disk, an optical disc, a magneto-optical disc, or semiconductor memory.
  • the connectivity 1321 may include devices (such as a monitor and a speaker) that output images and sound.
  • the camera 1322 is a module that has a function of imaging a subject and obtaining image data of the subject.
  • the image data obtained by the imaging by the camera 1322 is supplied to the video processor 1332 and encoded, for example.
  • the sensor 1323 is a module having any type of sensor function, such as a sound sensor, an ultrasonic sensor, a light sensor, an illumination sensor, an infrared sensor, an image sensor, a rotation sensor, an angle sensor, an angular velocity sensor, a speed sensor, an acceleration sensor, an inclination sensor, a magnetic field sensor, a shock sensor, or a temperature sensor, for example.
  • Data detected by the sensor 1323 is supplied to the application processor 1331 and utilized by an application and the like, for example.
  • configurations described as a module above may also be realized as a processor, while conversely, the configurations described as a processor may also be realized as a module.
  • the present technology can be applied to the video processor 1332 as described later. Consequently, the video set 1300 may be carried out as a set applying the present technology.
  • FIG. 75 illustrates one example of a schematic configuration of the video processor 1332 ( FIG. 74 ) applying the present technology.
  • the video processor 1332 includes a function of receiving the input of a video signal and an audio signal and encoding these signals according to a predetermined method, and a function of decoding encoded video data and audio data, and reproducing and outputting a video signal and an audio signal.
  • the video processor 1332 includes a video input processing unit 1401 , a first image enlargement/reduction unit 1402 , a second image enlargement/reduction unit 1403 , a video output processing unit 1404 , frame memory 1405 , and a memory control unit 1406 . Also, the video processor 1332 includes an encode/decode engine 1407 , video elementary stream (ES) buffers 1408 A and 1408 B, and audio ES buffers 1409 A and 1409 B. Additionally, the video processor 1332 includes an audio encoder 1410 , an audio decoder 1411 , a multiplexer (MUX) 1412 , a demultiplexer (DMUX) 1413 , and a stream buffer 1414 .
  • MUX multiplexer
  • DMUX demultiplexer
  • the video input processing unit 1401 acquires a video signal input from the connectivity 1321 ( FIG. 74 ) or the like, for example, and converts the video signal into digital image data.
  • the first image enlargement/reduction unit 1402 performs format conversion, image enlargement/reduction processing, and the like on the image data.
  • the second image enlargement/reduction unit 1403 performs a process of enlarging or reducing the image according to the format at the destination to which to output through the video output processing unit 1404 , format conversion and image enlargement/reduction processing similar to the first image enlargement/reduction unit 1402 , and the like on the image data.
  • the video output processing unit 1404 performs format conversion, conversion to an analog signal, and the like on the image data, and outputs the result to the connectivity 1321 for example as a reproduced video signal.
  • the frame memory 1405 is memory for image data shared by the video input processing unit 1401 , the first image enlargement/reduction unit 1402 , the second image enlargement/reduction unit 1403 , the video output processing unit 1404 , and the encode/decode engine 1407 .
  • the frame memory 1405 is realized as semiconductor memory such as DRAM, for example.
  • the memory control unit 1406 receives a synchronization signal from the encode/decode engine 1407 , and controls the access and writes and reads to the frame memory 1405 in accordance with an access schedule of access to the frame memory 1405 written in an access management table 1406 A.
  • the access management table 1406 A is updated by the memory control unit 1406 according to processes executed by the encode/decode engine 1407 , the first image enlargement/reduction unit 1402 , the second image enlargement/reduction unit 1403 , and the like.
  • the encode/decode engine 1407 executes a process of encoding image data as well as a process of decoding a video stream, which is data in which image data is encoded. For example, the encode/decode engine 1407 encodes image data read from the frame memory 1405 , and successively writes the encoded data to the video ES buffer 1408 A as a video stream. Also, for example, the encode/decode engine 1407 successively reads and decodes a video stream from the video ES buffer 1408 B, and writes the decoded data to the frame memory 1405 as image data. During this encoding and decoding, the encode/decode engine 1407 uses the frame memory 1405 as a work area. Also, the encode/decode engine 1407 outputs a synchronization signal to the memory control unit 1406 at the timing of starting the process for each macroblock, for example.
  • the video ES buffer 1408 A buffers and supplies a video stream generated by the encode/decode engine 1407 to the multiplexer (MUX) 1412 .
  • the video ES buffer 1408 B buffers and supplies a video stream supplied from the demultiplexer (DMUX) 1413 to the encode/decode engine 1407 .
  • the audio ES buffer 1409 A buffers and supplies an audio stream generated by the audio encoder 1410 to the multiplexer (MUX) 1412 .
  • the audio ES buffer 1409 B buffers and supplies an audio stream supplied from the demultiplexer (DMUX) 1413 to the audio decoder 1411 .
  • the audio encoder 1410 for example digitally converts an audio signal input from the connectivity 1321 or the like, for example, and encodes the audio signal according to a predetermined method such as the MPEG Audio method or the AudioCode number 3 (AC3) method, for example.
  • the audio encoder 1410 successively writes an audio stream, which is data in which an audio signal is encoded, to the audio ES buffer 1409 A.
  • the audio decoder 1411 decodes an audio stream supplied from the audio ES buffer 1409 B, performs conversion to an analog signal and the like, for example, and supplies the result to the connectivity 1321 and the like for example as a reproduced audio signal.
  • the multiplexer (MUX) 1412 multiplexes a video stream and an audio stream.
  • the multiplexing method i.e., the format of the bit stream generated by multiplexing
  • the multiplexer (MUX) 1412 is also able to add predetermined header information or the like to the bit stream.
  • the multiplexer (MUX) 1412 is able to convert the format of the streams by multiplexing. For example, by multiplexing a video stream and an audio stream, the multiplexer (MUX) 1412 converts the streams to a transport stream, which is a bit stream in a format for transmission. Also, for example, by multiplexing a video stream and an audio stream, the multiplexer (MUX) 1412 converts the streams to data (file data) in a file format for recording.
  • the demultiplexer (DMUX) 1413 demultiplexes a bit stream in which a video stream and an audio stream are multiplexed, according to a method corresponding to the multiplexed by the multiplexer (MUX) 1412 .
  • the demultiplexer (DMUX) 1413 extracts the video stream and the audio stream (separates the video stream and the audio stream) from a bit stream read out from the stream buffer 1414 .
  • the demultiplexer (DMUX) 1413 is able to convert the format of the stream by demultiplexing (an inverse conversion of the conversion by the multiplexer (MUX) 1412 ).
  • the demultiplexer (DMUX) 1413 is able to acquire a transport stream supplied from the connectivity 1321 , the broadband modem 1333 , or the like for example via the stream buffer 1414 , and by demultiplexing, is able to convert the transport stream into a video stream and an audio stream.
  • the demultiplexer (DMUX) 1413 is able to acquire file data read out from any of various types of recording media by the connectivity 1321 , for example via the stream buffer 1414 , and by demultiplexing, is able to convert the file data into a video stream and an audio stream.
  • the stream buffer 1414 buffers a bit stream.
  • the stream buffer 1414 buffers a transport stream supplied from the multiplexer (MUX) 1412 , and at a predetermined timing, or on the basis of an external request or the like, supplies the transport stream to the connectivity 1321 , the broadband modem 1333 , or the like, for example.
  • MUX multiplexer
  • the stream buffer 1414 buffers file data supplied from the multiplexer (MUX) 1412 , and at a predetermined timing, or on the basis of an external request or the like, supplies the file data to the connectivity 1321 or the like, for example, and causes the file data to be recorded on any of various types of recording media.
  • MUX multiplexer
  • the stream buffer 1414 buffers a transport stream acquired via the connectivity 1321 , the broadband modem 1333 , and the like, for example, and at a predetermined timing, or on the basis of an external request or the like, supplies the transport stream to the demultiplexer (DMUX) 1413 .
  • DMUX demultiplexer
  • the stream buffer 1414 buffers file data read out from any of various types of recording media in the connectivity 1321 or the like, for example, and at a predetermined timing, or on the basis of an external request or the like, supplies the file data to the demultiplexer (DMUX) 1413 .
  • DMUX demultiplexer
  • a video signal input into the video processor 1332 from the connectivity 1321 or the like is converted to digital image data of a predetermined format such as 4:2:2 Y/Cb/Cr format in the video input processing unit 1401 , and is successively written to the frame memory 1405 .
  • the digital image data is read out to the first image enlargement/reduction unit 1402 or the second image enlargement/reduction unit 1403 , subjected to a format conversion to a predetermined format such as 4:2:0 Y/Cb/Cr or the like and an enlargement/reduction process, and again written to the frame memory 1405 .
  • the image data is encoded by the encode/decode engine 1407 , and written to the video ES buffer 1408 A as a video stream.
  • an audio signal input into the video processor 1332 from the connectivity 1321 or the like is encoded by the audio encoder 1410 , and written to the audio ES buffer 1409 A as an audio stream.
  • the video stream in the video ES buffer 1408 A and the audio stream in the audio ES buffer 1409 A are read out and multiplexed by the multiplexer (MUX) 1412 , and converted to a transport stream, file data, or the like.
  • the transport stream generated by the multiplexer (MUX) 1412 is buffered in the stream buffer 1414 , and then output to an external network via the connectivity 1321 , the broadband modem 1333 , or the like, for example.
  • the file data generated by the multiplexer (MUX) 1412 is buffered in the stream buffer 1414 , and then output to the connectivity 1321 or the like, for example, and recorded to any of various types of recording media.
  • a transport stream input into the video processor 1332 from an external network via the connectivity 1321 , the broadband modem 1333 , or the like for example is buffered in the stream buffer 1414 , and then demultiplexed by the demultiplexer (DMUX) 1413 .
  • file data read out from any of various types of recording media in the connectivity 1321 or the like, for example, and input into the video processor 1332 is buffered in the stream buffer 1414 , and then demultiplexed by the demultiplexer (DMUX) 1413 .
  • a transport stream or file data input into the video processor 1332 is separated into a video stream and an audio stream by the demultiplexer (DMUX) 1413 .
  • the audio stream is supplied to the audio decoder 1411 via the audio ES buffer 1409 B and decoded, and an audio signal is reproduced.
  • the video stream after being written to the video ES buffer 1408 B, is successively read out and decoded by the encode/decode engine 1407 , and written to the frame memory 1405 .
  • the decoded image data is subjected to an enlargement/reduction process by the second image enlargement/reduction unit 1403 , and written to the frame memory 1405 .
  • the decoded image data is read out to the video output processing unit 1404 , format-converted to a predetermined format such as 4:2:2 Y/Cb/Cr format, additionally converted to an analog signal, and a video signal is reproduced and output.
  • a predetermined format such as 4:2:2 Y/Cb/Cr format
  • the encode/decode engine 1407 may include the functions of the image encoding apparatus 400 or the functions of the image decoding apparatus 100 described above, or both.
  • the video processor 1332 is able to obtain effects similar to each of the embodiments described above with reference to FIGS. 1 to 68 .
  • the present technology i.e., the functions of the image encoding apparatus 400 , the functions of the image decoding apparatus 100 , or both
  • the present technology may be realized by hardware such as a logic circuit or the like, may be realized by software such as an embedded program, or may be realized by both of the above.
  • FIG. 76 illustrates another example of a schematic configuration of the video processor 1332 applying the present technology.
  • the video processor 1332 includes a function of encoding/decoding video data according to a predetermined method.
  • the video processor 1332 includes a control unit 1511 , a display interface 1512 , a display engine 1513 , an image processing engine 1514 , and internal memory 1515 . Also, the video processor 1332 includes a codec engine 1516 , a memory interface 1517 , a multiplexer/demultiplexer (MUX DMUX) 1518 , a network interface 1519 , and a video interface 1520 .
  • a codec engine 1516 a memory interface 1517 , a multiplexer/demultiplexer (MUX DMUX) 1518 , a network interface 1519 , and a video interface 1520 .
  • MUX DMUX multiplexer/demultiplexer
  • the control unit 1511 controls the operation of each processing unit in the video processor 1332 , such as the display interface 1512 , the display engine 1513 , the image processing engine 1514 , and the codec engine 1516 .
  • the control unit 1511 includes a main CPU 1531 , a sub CPU 1532 , and a system controller 1533 , for example.
  • the main CPU 1531 executes a program or the like for controlling the operation of each processing unit in the video processor 1332 .
  • the main CPU 1531 generates control signals in accordance with the program or the like, and supplies the control signals to each processing unit (in other words, controls the operation of each processing unit).
  • the sub CPU 1532 fulfills a supplementary role to the main CPU 1531 .
  • the sub CPU 1532 executes child processes, subroutines, and the like of the program or the like executed by the main CPU 1531 .
  • the system controller 1533 controls the operations of the main CPU 1531 and the sub CPU 1532 , such as specifying programs to be executed by the main CPU 1531 and the sub CPU 1532 .
  • the display interface 1512 under control by the control unit 1511 , outputs image data to the connectivity 1321 and the like, for example.
  • the display interface 1512 converts digital image data to an analog signal and outputs an analog signal, or outputs the digital image data directly, as a reproduced video signal to a monitor apparatus or the like of the connectivity 1321 .
  • the display engine 1513 under control by the control unit 1511 , performs various conversion processes such as format conversion, size conversion, and gamut conversion on the image data to match the hardware specs of the monitor apparatus or the like that is to display the image.
  • the image processing engine 1514 under control by the control unit 1511 performs predetermined image processing on the image data, such as filter processing for improving image quality, for example.
  • the internal memory 1515 is memory provided inside the video processor 1332 , and shared by the display engine 1513 , the image processing engine 1514 , and the codec engine 1516 .
  • the internal memory 1515 is used to exchange data between the display engine 1513 , the image processing engine 1514 , and the codec engine 1516 .
  • the internal memory 1515 stores data supplied from the display engine 1513 , the image processing engine 1514 , or the codec engine 1516 , and as necessary (for example, in response to a request), supplies the data to the display engine 1513 , the image processing engine 1514 , or the codec engine 1516 .
  • the internal memory 1515 may be realized by any kind of storage device, but since the storage device typically is used to store small amounts of data such as image data in units of blocks, parameters, and the like, it is desirable to realize the storage device with semiconductor memory that is relatively (for example, compared to the external memory 1312 ) small in capacity but has a fast response speed, such as static random access memory (SRAM), for example.
  • SRAM static random access memory
  • the codec engine 1516 executes processes related to the encoding and decoding of image data.
  • the encoding/decoding method supported by the codec engine 1516 may be any method, and there may be one or multiple such methods.
  • the codec engine 1516 may be provided with a codec function for multiple encoding/decoding methods, and may be configured to encode or decode image data by selecting from among the multiple methods.
  • the codec engine 1516 includes MPEG-2 Video 1541 , AVC/H.264 1542 , HEVC/H.265 1543 , HEVC/H.265 (Scalable) 1544 , HEVC/H.265 (Multi-view) 1545 , and MPEG-DASH 1551 as function blocks of codec-related processing, for example.
  • the MPEG-2 Video 1541 is a function block that encodes and decodes image data according to the MPEG-2 method.
  • the AVC/H.264 1542 is a function block that encodes and decodes image data according to the AVC method.
  • the HEVC/H.265 1543 is a function block that encodes and decodes image data according to the HEVC method.
  • the HEVC/H.265 (Scalable) 1544 is a function block that scalably encodes and scalably decodes image data according to the HEVC method.
  • the HEVC/H.265 (Multi-view) 1545 is a function block that multi-view encodes and multi-view decodes image data according to the HEVC method.
  • the MPEG-DASH 1551 is a function block that transmits and receives image data according to the MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) method.
  • MPEG-DASH is a technology that uses the Hypertext Transfer Protocol (HTTP) to stream video, one feature of which being that appropriate encoded data is selected and transmitted in units of segments from among multiple sets of encoded data having different resolutions or the like prepared in advance.
  • HTTP Hypertext Transfer Protocol
  • the MPEG-DASH 1551 executes the generation, transmission control, and the like of a stream conforming to the standard, while for the encoding/decoding of image data, the MPEG-2 Video 1541 to the HEVC/H.265 (Multi-view) 1545 are used.
  • the memory interface 1517 is an interface for the external memory 1312 .
  • Data supplied from the image processing engine 1514 and the codec engine 1516 is supplied to the external memory 1312 through the memory interface 1517 .
  • data read out from the external memory 1312 is supplied to the video processor 1332 (the image processing engine 1514 or the codec engine 1516 ) through the memory interface 1517 .
  • the multiplexer/demultiplexer (MUX DMUX) 1518 multiplexes and demultiplexes various image-related data, such as a bit stream of encoded data, image data, a video signal, and the like.
  • the multiplexing/demultiplexing method may be any method.
  • the multiplexer/demultiplexer (MUX DMUX) 1518 is not only able to collect multiple pieces of data into a single piece of data, but also add predetermined header information and the like to the data.
  • the multiplexer/demultiplexer (MUX DMUX) 1518 is not only able to divide a single piece of data into multiple pieces of data, but also add predetermined header information and the like to each divided piece of data.
  • the multiplexer/demultiplexer (MUX DMUX) 1518 is able to convert the format of data by multiplexing/demultiplexing.
  • the multiplexer/demultiplexer (MUX DMUX) 1518 is able to convert the bit stream to a transport stream, which is a bit stream in a format for transmission, or to data in a file format (file data) for recording.
  • the inverse conversion is also possible.
  • the network interface 1519 is an interface for the broadband modem 1333 , the connectivity 1321 , and the like, for example.
  • the video interface 1520 is an interface for the connectivity 1321 , the camera 1322 , and the like, for example.
  • a transport stream is received from an external network through the connectivity 1321 , the broadband modem 1333 , or the like
  • the transport stream is supplied to the multiplexer/demultiplexer (MUX DMUX) 1518 through the network interface 1519 and demultiplexed, and decoded by the codec engine 1516 .
  • the image data obtained by the decoding of the codec engine 1516 is, for example, subjected to predetermined image processing by the image processing engine 1514 , subjected to a predetermined conversion by the display engine 1513 , supplied to the connectivity 1321 or the like for example through the display interface 1512 , and the image is displayed on a monitor.
  • the image data obtained by the decoding of the codec engine 1516 is re-encoded by the codec engine 1516 , multiplexed and converted to file data by the multiplexer/demultiplexer (MUX DMUX) 1518 , output to the connectivity 1321 or the like for example through the video interface 1520 , and recorded on any of various types of recording media.
  • MUX DMUX multiplexer/demultiplexer
  • file data of encoded data in which image data is encoded that is read out from a recording medium not illustrated by the connectivity 1321 or the like is supplied to the multiplexer/demultiplexer (MUX DMUX) 1518 through the video interface 1520 and demultiplexed, and decoded by the codec engine 1516 .
  • the image data obtained by the decoding of the codec engine 1516 is subjected to predetermined image processing by the image processing engine 1514 , subjected to a predetermined conversion by the display engine 1513 , supplied to the connectivity 1321 or the like for example through the display interface 1512 , and the image is displayed on a monitor.
  • the image data obtained by the decoding of the codec engine 1516 is re-encoded by the codec engine 1516 , multiplexed and converted to a transport stream by the multiplexer/demultiplexer (MUX DMUX) 1518 , supplied to the connectivity 1321 , the broadband modem 1333 , or the like for example through the network interface 1519 , and transmitted to another apparatus not illustrated.
  • MUX DMUX multiplexer/demultiplexer
  • the exchange of image data and other data between each of the processing units inside the video processor 1332 is performed by utilizing the internal memory 1515 and the external memory 1312 , for example. Additionally, the power management module 1313 controls the supply of power to the control unit 1511 , for example.
  • the video processor 1332 configured in this way, it is sufficient to apply the present technology according to the embodiments described above to the codec engine 1516 .
  • the codec engine 1516 it is sufficient for the codec engine 1516 to include the functions of the image encoding apparatus 400 or the functions of the image decoding apparatus 100 described above, or both.
  • the video processor 1332 is able to obtain effects similar to each of the embodiments described above with reference to FIGS. 1 to 68 .
  • the present technology i.e., the functions of the image encoding apparatus 400
  • the present technology may be realized by hardware such as a logic circuit or the like, may be realized by software such as an embedded program, or may be realized by both of the above.
  • the configuration of the video processor 1332 may be any configuration, and may be a configuration other than the two examples described above.
  • the video processor 1332 may be configured as a single semiconductor chip, but may also be configured as multiple semiconductor chips. For example, a three-dimensionally stacked LSI chip in which multiple semiconductors are stacked is possible. Also, a configuration realized by multiple LSI chips is possible.
  • the video set 1300 can be embedded into any of various types of apparatus that process image data.
  • the video set 1300 can be embedded into the television apparatus 900 ( FIG. 70 ), the mobile telephone 920 ( FIG. 71 ), the recording/reproducing apparatus 940 ( FIG. 72 ), the imaging apparatus 960 ( FIG. 73 ), and the like.
  • the apparatus is able to obtain effects similar to each of the embodiments described above with reference to FIGS. 1 to 68 .
  • the video processor 1332 is included, even a part of each configuration of the video set 1300 described above can be carried out as a configuration applying the present technology. For example, it is possible to carry out only the video processor 1332 as a video processor applying the present technology. Also, for example, the processor illustrated by the dashed line 1341 as described above, the video module 1311 , and the like can be carried out as a processor, module, or the like applying the present technology. Furthermore, for example, the video module 1311 , the external memory 1312 , the power management module 1313 , and the front-end module 1314 can also be combined and carried out as a video unit 1361 applying the present technology. With any of these configurations, it is possible to obtain effects similar to each of the embodiments described above with reference to FIGS. 1 to 68 .
  • any type of configuration can be embedded into any of various types of apparatus that process image data, similarly to the case of the video set 1300 .
  • the video processor 1332 the processor illustrated by the dashed line 1341 , the video module 1311 , or the video unit 1361 can be embedded into the television apparatus 900 (FIG. 70 ), the mobile telephone 920 ( FIG. 71 ), the recording/reproducing apparatus 940 ( FIG. 72 ), the imaging apparatus 960 ( FIG. 73 ), and the like.
  • the apparatus is able to obtain effects similar to each of the embodiments described above with reference to FIGS. 1 to 68 , similarly to the video set 1300 .
  • FIG. 77 illustrates one example of a schematic configuration of a network system applying the present technology.
  • the network system 1600 illustrated in FIG. 77 is a system in which devices exchange information regarding images (moving images) with each other over a network.
  • the cloud service 1601 of the network system 1600 is a system that provides a service related to images (moving images) to terminals such as a computer 1611 , audio-visual (AV) equipment 1612 , a mobile information processing terminal 1613 , and an Internet of Things (IoT) device 1614 communicably connected to the cloud service 1601 .
  • the cloud service 1601 provides a service of supplying image (moving image) content to terminals, like what is called video streaming (on-demand or live streaming).
  • the cloud service 1601 provides a backup service that receives and stores image (moving image) content from terminals.
  • the cloud service 1601 provides a service of mediating the exchange of image (moving image) content between terminals.
  • the physical configuration of the cloud service 1601 may be any configuration.
  • the cloud service 1601 may include various servers, such as a server that saves and manages moving images, a server that delivers moving images to terminals, a server that acquires moving images from terminals, and a server that manages users (terminals) and payments, as well as any type of network, such as the Internet or a LAN.
  • the computer 1611 includes an information processing apparatus such as a personal computer, server, or workstation, for example.
  • the AV equipment 1612 includes image processing apparatus such as a television receiver, a hard disk recorder, a game console, or a camera, for example.
  • the mobile information processing terminal 1613 includes a mobile information processing apparatus such as a notebook personal computer, a tablet terminal, a mobile telephone, or a smartphone, for example.
  • the IoT device 1614 includes any object that executes image-related processing, such as a machine, an electric appliance, a piece of furniture, some other thing, an IC tag, or a card-shaped device, for example.
  • terminals all include a communication function, and are able to connect to (establish a session with) the cloud service 1601 and exchange information with (i.e., communicate with) the cloud service 1601 . Also, each terminal is also able to communicate with another terminal. Communication between terminals may be performed by going through the cloud service 1601 , or may be performed without going through the cloud service 1601 .
  • the image data may be encoded/decoded as described above in each of the embodiments.
  • the terminals (from the computer 1611 to the IoT device 1614 ) and the cloud service 1601 each may include the functions of the image encoding apparatus 400 and the image decoding apparatus 100 described above. In this manner, the terminals (from the computer 1611 to the IoT device 1614 ) that exchange image data and the cloud service 1601 can obtain similar effects to those of each of the embodiments described above with reference to FIG. 1 to FIG. 68 .
  • a “flag” is information for identifying each of a plurality of states including not only information to be used to identify two state of true (1) or false (0) but also information that can identify three or more states. Therefore, a value that the “flag” can have may be, for example, a binary value of 1 or 0, or a ternary or higher value. That is, the number of bits constituting the “flag” is arbitrary, and may be one or more bits.
  • identification information (also including a flag) is assumed to have a form in which the identification information is included in a bit stream and a form in which differential information between certain reference information and the identification information is included in a bit stream, and thus in the present specification, a “flag” and “identification information” include not only the information but also differential information serving as a reference.
  • various kinds of information (metadata, etc.) with respect to encoded data may be transmitted or recorded in any form as long as they are associated with encoded dat.
  • the term “associated with” used herein means, in one example, that when one data is processed, other data can be used (linkable). In other words, the data associated with each other may be collected as one data or may be individual data.
  • information associated with encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image).
  • the information associated with encoded data (image) may be recorded on a recording medium (or other recording areas of the same recording medium) different from that of the encoded data (image).
  • this term “associated with” may be a part of data, not the entire data.
  • an image and information corresponding to the image may be associated with each other in any units such as a plurality of frames, one frame, a part within a frame, or the like.
  • the terms “combine”, “multiplex”, “attach”, “integrate”, “include”, “store”, “push into”, “put into, “insert”, and the like mean combining a plurality of objects into one, for example, combining encoded data and metadata into a single data item, which means one usage of “associated with” described above.
  • the present technology can also be implemented by any of configurations constituting an apparatus or a system, for example, a processor as a large scale integration (LSI) and the like, a module using a plurality of processors and the like, a unit using a plurality of modules and the like, a set in which other functions are further added to a set, or the like (i.e., a partial configuration of an apparatus).
  • LSI large scale integration
  • modules using a plurality of processors and the like
  • a unit using a plurality of modules and the like i.e., a partial configuration of an apparatus.
  • a system means a set of a plurality of constituent elements (e.g., devices or modules (parts)), regardless of whether or not all the constituent elements are in the same housing. Accordingly, a plurality of devices that is contained in different housings and connected via a network and one device in which a plurality of modules is contained in one housing are both systems.
  • constituent elements e.g., devices or modules (parts)
  • an element described as a single device may be divided and configured as a plurality of devices (or processing units).
  • elements described as a plurality of devices (or processing units) above may be configured collectively as a single device (or processing unit).
  • an element other than those described above may be added to the configuration of each device (or processing unit).
  • a part of the configuration of a given device (or processing unit) may be included in the configuration of another device (or another processing unit) as long as the configuration or operation of the system as a whole is substantially the same.
  • the present technology can adopt a configuration of cloud computing which performs processing by allocating and sharing one function by a plurality of devices through a network.
  • the program described above can be executed in any device.
  • the device has a necessary function (functional block or the like) and can obtain necessary information.
  • each step described by the above-described flowcharts can be executed by one device or executed by being allocated to a plurality of devices.
  • the plurality of processes included in this one step can be executed by one device or executed by being allocated to a plurality of devices.
  • a plurality of processes included in one step can also be executed as a process of a plurality of steps.
  • a process described as a plurality of steps can be collectively executed in one step.
  • processing in steps describing the program may be executed chronologically along the order described in this specification, or may be executed concurrently, or individually at necessary timing such as when a call is made. In other words, unless a contradiction arises, processing in the steps may be executed in an order different from the order described above. Furthermore, processing in steps describing the program may be executed concurrently with processing of another program, or may be executed in combination with processing of another program.
  • any plurality of the present technologies described in this specification can be performed alone independently of each other, unless a contradiction arises.
  • any plurality of the present technologies can be performed in combination.
  • a part or all of the present technology described in any of the embodiments can be performed in combination with a part or all of the present technology described in another embodiment.
  • any of a part or all of the present technologies described above can be performed in combination with another technology that is not described above.
  • present technology may also be configured as below.
  • An image processing apparatus including:
  • an inverse orthogonal transform unit configured to perform an inverse orthogonal transform of chrominance using information regarding the inverse orthogonal transform of chrominance derived on the basis of information regarding an inverse orthogonal transform of luminance.
  • the image processing apparatus in which the information regarding the inverse orthogonal transform includes an adaptive primary transform flag indicating whether an adaptive inverse primary transform of adaptively selecting one from a plurality of different inverse orthogonal transforms and using the selected transform as an inverse primary transform is to be applied in a transform block to be processed.
  • the image processing apparatus in which the information regarding the inverse orthogonal transform includes a primary transform identifier indicating which inverse primary transform is to be applied to inverse primary transforms in a vertical direction and a horizontal direction.
  • a value of the primary transform identifier of chrominance is set to a value of the primary transform identifier of luminance in a case in which the adaptive primary transform flag of chrominance is true, and is set to a predetermined value in a case in which the adaptive primary transform flag of chrominance is false.
  • the image processing apparatus in which the inverse orthogonal transform unit performs the inverse primary transform as the inverse orthogonal transform.
  • the image processing apparatus in which the inverse orthogonal transform unit performs an inverse primary horizontal transform which is the inverse primary transform in the horizontal direction and an inverse primary vertical transform which is the inverse primary transform in the vertical direction as the inverse orthogonal transform.
  • the image processing apparatus in which the transform type of the inverse primary horizontal transform of the chrominance and the transform type of the inverse primary vertical transform of the chrominance are each derived using a method in accordance with a value of the adaptive primary transform flag of the chrominance.
  • a transform set of the inverse primary horizontal transform and a transform set of the inverse primary vertical transform are selected on the basis of prediction mode information
  • a primary horizontal transform specifying flag and a primary vertical transform flag are derived from the primary transform identifier of the chrominance
  • a transform type of the inverse primary horizontal transform of the chrominance is selected on the basis of the primary horizontal transform set and the primary horizontal transform specifying flag
  • a transform type of the inverse primary vertical transform of the chrominance is selected on the basis of the primary vertical transform set and the primary vertical transform.
  • a predetermined transform type is selected as a transform type of the inverse primary horizontal transform of the chrominance and a predetermined transform type is selected as a transform type of the inverse primary vertical transform of the chrominance.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction, and the flag is set to be false in a case in which the prediction type of the coding block is intra prediction.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or a case in which a prediction type is intra prediction of which the prediction mode is intra block copy, and the flag is set to be false in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy.
  • a value of the adaptive primary transform flag of chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction or a case in which the prediction type is intra prediction in which prediction modes for luminance and chrominance match each other, and is set to false in a case in which the prediction type of the coding block is intra prediction in which prediction modes for luminance and chrominance do not match each other.
  • a value of the adaptive primary transform flag of chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a chrominance adaptive primary transform information inference flag indicating whether the adaptive primary transform flag of chrominance is to be inferred on the basis of the adaptive primary transform flag of luminance is true, and is set to false in a case in which the chrominance adaptive primary transform information inference flag is false.
  • the image processing apparatus in which the chrominance adaptive primary transform information inference flag is decoded from a bit stream in a case in which a processing object component is the chrominance, the adaptive primary transform flag of luminance is true, a transform quantization bypass flag is false, and a transform skip flag is false, and a value of the chrominance adaptive primary transform information inference is set to be false in other cases.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a size of a short side of a transform block to be processed of the chrominance is greater than or equal to a predetermined threshold value, and the value is set to be false in a case in which the size of the short side of the transform block is shorter than the threshold value.
  • the inverse orthogonal transform unit is configured to perform the inverse primary horizontal transform which is the inverse primary transform in the horizontal direction and the inverse primary vertical transform which is the inverse primary transform in the vertical direction as the inverse orthogonal transform,
  • the transform type of the inverse primary horizontal transform is set to a predetermined transform type in a case in which a size of a picture width of the transform block to be processed of the chrominance is equal to or smaller than a predetermined threshold value, and the transport type is set on the basis of a horizontal transform set and a primary horizontal transform specifying flag in a case in which the size of the picture width of the transform block is greater than the threshold value, and
  • the transform type of the inverse primary vertical transform is set to a predetermined transform type in a case in which a size of a picture height of the transform block to be processed of the chrominance is equal to or smaller than a threshold value, and the transform type is set on the basis of a vertical transform set and a primary vertical transform specifying flag in a case in which the size of the picture height of the transform block is greater than the threshold value.
  • the image processing apparatus according to any of (1) to (17), in which the information regarding the inverse orthogonal transform includes a transform skip flag indicating whether an inverse orthogonal transform process is to be skipped.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction, and the value is set to a value of the transform skip flag of the chrominance that is obtained by decoding a bit stream in a case in which the prediction type of the coding block is intra prediction.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, and the value is set to a value of the transform skip flag of the chrominance obtained by decoding a bit stream in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, and the value is set to a value of the transform skip flag of the chrominance in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which a chrominance transform skip information inference flag indicating whether the transform skip flag of the chrominance is to be inferred on the basis of the transform skip flag of luminance is true, and the value is set to a value of the transform skip flag of the chrominance obtained by decoding a bit stream in a case in which the chrominance transform skip information inference flag is false.
  • the image processing apparatus according to any of (1) to (23), in which the information regarding the inverse orthogonal transform includes a secondary transform identifier indicating which inverse secondary transform is to be applied.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction, and the value is set to a value of the secondary transform identifier of the chrominance obtained by decoding a bit stream in a case in which the prediction type of the coding block is intra prediction.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, and the value is set to a value of the secondary transform identifier of the chrominance obtained by decoding a bit stream in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, and the value is set to a value of the secondary transform identifier of the chrominance obtained by decoding a bit stream in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which a chrominance secondary transform identification information inference flag indicating whether the secondary transform identifier of the chrominance is to be inferred on the basis of the secondary transform identifier of luminance is true, and the value is set to a value of the secondary transform identifier of the chrominance obtained by decoding a bit stream in a case in which the chrominance secondary transform identification information inference flag is false.
  • An image processing method including:
  • An image processing apparatus including:
  • an orthogonal transform unit configured to perform an orthogonal transform of chrominance using information regarding the orthogonal transform of chrominance derived on the basis of information regarding an orthogonal transform of luminance.
  • information regarding the orthogonal transform includes an adaptive primary transform flag indicating whether an adaptive primary transform of adaptively selecting one from a plurality of different orthogonal transforms and using the selected transform as a primary transform is to be applied to a transform block to be processed.
  • the image processing apparatus in which the information regarding the orthogonal transform includes a primary transform identifier indicating which primary transform is to be applied to primary transforms in a vertical direction and a horizontal direction.
  • a value of the primary transform identifier of chrominance is set to a value of the primary transform identifier of luminance in a case in which the adaptive primary transform flag of chrominance is true, and is set to a predetermined value in a case in which the adaptive primary transform flag of chrominance is false.
  • the image processing apparatus in which the orthogonal transform unit performs a primary transform as the orthogonal transform.
  • the image processing apparatus in which the orthogonal transform unit performs a primary horizontal transform which is the primary transform in a horizontal direction and a primary vertical transform which is the primary transform in a vertical direction as the orthogonal transform.
  • a transform type of the primary horizontal transform of a chrominance and a transform type of the primary vertical transform of the chrominance are each derived using a method in accordance with a value of the adaptive primary transform flag of the chrominance.
  • a transform set of a primary horizontal transform and a transform set of a primary vertical transform are selected on the basis of prediction mode information.
  • a primary horizontal transform specifying flag and a primary vertical transform flag are derived from the primary transform identifier of the chrominance
  • a transform type of the primary horizontal transform of the chrominance is selected on the basis of the primary horizontal transform set and the primary horizontal transform specifying flag
  • a transform type of the primary vertical transform of the chrominance is selected on the basis of the primary vertical transform set and the primary vertical transform specifying flag.
  • a predetermined transform type is selected of a transform type of the primary horizontal transform of the chrominance
  • a predetermined transform type is selected as a transform type of the primary vertical transform of the chrominance
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a prediction type of a coding block to which a transform block to be processed belongs is inter prediction, and the value is set to be false in a case in which the prediction type of the coding block is intra prediction.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, and the value is set to be false in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, and the value is set to be false in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a chrominance adaptive primary transform information inference flag indicating whether the adaptive primary transform flag of the chrominance is to be inferred on the basis of the adaptive primary transform flag of luminance is true, and the value is set to be false in a case in which the chrominance adaptive primary transform information inference flag is false.
  • the image processing apparatus in which, in a case in which a processing object component is a chrominance, the adaptive primary transform flag of luminance is true, a transform quantization bypass flag is false, and a transform skip flag is false, the chrominance adaptive primary transform information inference flag is encoded.
  • a value of the adaptive primary transform flag of the chrominance is set to a value of the adaptive primary transform flag of luminance in a case in which a size of a short side of a transform block to be processed of the chrominance is greater than or equal to a predetermined threshold value, and a value is set to be false in a case in which the size of the short side of the transform block is smaller than the threshold value.
  • the orthogonal transform unit is configured to perform the primary horizontal transform which is the primary transform in the horizontal direction and the primary vertical transform which is the primary transform in the vertical direction as the orthogonal transform,
  • a transform type of the primary horizontal transform is set to a predetermined transform type in a case in which a size of a picture width of the transform block to be processed of the chrominance is equal to or smaller than a predetermined threshold value, and the transform type is set on the basis of a horizontal transform set and a primary horizontal transform specifying flag in a case in which the size of the picture width of the transform block is greater than the threshold value, and
  • a transform type of the primary vertical transform is set to a predetermined transform type in a case in which a size of a picture height of the transform block to be processed of the chrominance is equal to or smaller than a predetermined threshold value, and the transform type is set on the basis of a vertical transform set and a primary vertical transform specifying flag in a case in which the size of the picture height of the transform block is greater than the threshold value.
  • the image processing apparatus according to any of (31) to (47), in which the information regarding the inverse orthogonal transform includes a transform skip flag indicating whether an inverse orthogonal transform process is to be skipped.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block is intra prediction.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block intra prediction of which the prediction mode is not intra block copy.
  • a value of the transform skip flag of the chrominance is set to a value of the transform skip flag of luminance in a case in which a chrominance transform skip information inference flag indicating whether the transform skip flag of the chrominance is to be inferred on the basis of the transform skip flag of luminance is true, and the value is set to an arbitrary value and encoded in a case in which the chrominance transform skip information inference flag is false.
  • the image processing apparatus according to any of (31) to (53), in which the information regarding the orthogonal transform includes a secondary transform identifier indicating which secondary transform is to be applied.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block is intra prediction.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction in which the prediction modes for luminance and the chrominance match each other, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block is intra prediction in which the prediction modes for luminance and the chrominance do not match each other.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which the prediction type of the coding block to which the transform block to be processed belongs is inter prediction or intra prediction of which the prediction mode is intra block copy, and the value is set to an arbitrary value and encoded in a case in which the prediction type of the coding block is intra prediction of which the prediction mode is not intra block copy.
  • a value of the secondary transform identifier of the chrominance is set to a value of the secondary transform identifier of luminance in a case in which a chrominance secondary transform identification information inference flag indicating whether the secondary transform identifier of the chrominance is to be inferred on the basis of the secondary transform identifier of luminance is true, and the value is set to an arbitrary value and encoded in a case in which the chrominance secondary transform identification information inference flag is false.
  • An image processing method including:
US16/468,451 2016-12-28 2017-12-15 Image processing apparatus and method Abandoned US20210243475A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2016-256732 2016-12-28
JP2016256732 2016-12-28
PCT/JP2017/045042 WO2018123644A1 (ja) 2016-12-28 2017-12-15 画像処理装置および方法

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/045042 A-371-Of-International WO2018123644A1 (ja) 2016-12-28 2017-12-15 画像処理装置および方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/888,943 Continuation US20220394299A1 (en) 2016-12-28 2022-08-16 Image processing apparatus and method

Publications (1)

Publication Number Publication Date
US20210243475A1 true US20210243475A1 (en) 2021-08-05

Family

ID=62708047

Family Applications (2)

Application Number Title Priority Date Filing Date
US16/468,451 Abandoned US20210243475A1 (en) 2016-12-28 2017-12-15 Image processing apparatus and method
US17/888,943 Pending US20220394299A1 (en) 2016-12-28 2022-08-16 Image processing apparatus and method

Family Applications After (1)

Application Number Title Priority Date Filing Date
US17/888,943 Pending US20220394299A1 (en) 2016-12-28 2022-08-16 Image processing apparatus and method

Country Status (10)

Country Link
US (2) US20210243475A1 (pt)
EP (1) EP3565249A4 (pt)
JP (1) JP7067487B2 (pt)
KR (1) KR102476148B1 (pt)
CN (1) CN110100438B (pt)
AU (1) AU2017387752B2 (pt)
BR (1) BR112019012707A2 (pt)
CA (1) CA3041856A1 (pt)
RU (1) RU2759052C2 (pt)
WO (1) WO2018123644A1 (pt)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200329242A1 (en) * 2017-12-28 2020-10-15 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US20210211727A1 (en) * 2018-09-02 2021-07-08 Lg Electronics Inc. Image coding method based on multiple transform selection and device therefor
US20220046241A1 (en) * 2017-07-03 2022-02-10 Panasonic Intellectual Property Corporation Of America Coding method, decoding method, encoder, and decoder
US20220124330A1 (en) * 2018-10-05 2022-04-21 Lg Electronics Inc. Method for coding transform coefficient and device therefor
US20220295061A1 (en) * 2019-04-23 2022-09-15 Beijing Bytedance Network Technology Co., Ltd. Context modeling and selection of multiple transform matrix

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102636267B1 (ko) 2018-08-16 2024-02-14 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 변형 행렬 선택의 계수에 따른 코딩
KR20200039591A (ko) * 2018-10-05 2020-04-16 한국전자통신연구원 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장한 기록 매체
EP3881546A4 (en) * 2018-12-04 2022-01-26 Huawei Technologies Co., Ltd. VIDEO ENCODER, VIDEO DECODER, AND CORRESPONDING METHOD
WO2020130569A1 (ko) 2018-12-20 2020-06-25 주식회사 엘지화학 디엔계 고무 라텍스, 이의 제조방법 및 이를 포함하는 코어-쉘 구조의 그라프트 공중합체
WO2020216177A1 (en) 2019-04-20 2020-10-29 Beijing Bytedance Network Technology Co., Ltd. Signaling of chroma and luma syntax elements in video coding
CN115567707A (zh) 2019-05-30 2023-01-03 抖音视界有限公司 色度分量的自适应环路滤波
AU2019232801A1 (en) * 2019-09-17 2021-04-01 Canon Kabushiki Kaisha Method, apparatus and system for encoding and decoding a block of video samples
CN114982232A (zh) * 2019-12-26 2022-08-30 日本放送协会 编码装置、解码装置和程序
US20220394298A1 (en) * 2019-12-27 2022-12-08 Interdigital Vc Holdings France Transform coding for inter-predicted video data

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1193977B1 (en) * 1997-06-09 2003-08-27 Hitachi, Ltd. Image sequence coding method
JP3887178B2 (ja) * 2001-04-09 2007-02-28 株式会社エヌ・ティ・ティ・ドコモ 信号符号化方法及び装置並びに復号方法及び装置
CN102648628B (zh) * 2009-12-04 2017-09-29 汤姆森许可贸易公司 视频编码和解码中针对帧间帧的改进色度变换的方法和装置
JP2011259205A (ja) * 2010-06-09 2011-12-22 Sony Corp 画像復号化装置と画像符号化装置およびその方法とプログラム
JP5741076B2 (ja) * 2010-12-09 2015-07-01 ソニー株式会社 画像処理装置及び画像処理方法
GB2492333B (en) * 2011-06-27 2018-12-12 British Broadcasting Corp Video encoding and decoding using transforms
JP5707261B2 (ja) * 2011-07-12 2015-04-22 Kddi株式会社 動画像符号化装置、動画像復号装置、動画像符号化方法、動画像復号方法、およびプログラム
CN103843345B (zh) * 2011-10-17 2017-05-10 株式会社东芝 编码设备、解码设备、编码方法和解码方法
CN107820078B (zh) * 2011-10-18 2021-08-10 株式会社Kt 视频信号解码方法
ES2864427T3 (es) * 2011-12-15 2021-10-13 Tagivan Ii Llc Señalización de banderas de bloque codificado (CBF) de luminancia-crominancia en codificación de vídeo
AU2012232992A1 (en) * 2012-09-28 2014-04-17 Canon Kabushiki Kaisha Method, apparatus and system for encoding and decoding the transform units of a coding unit
US20150016516A1 (en) * 2013-07-15 2015-01-15 Samsung Electronics Co., Ltd. Method for intra prediction improvements for oblique modes in video coding
AU2014202921B2 (en) * 2014-05-29 2017-02-02 Canon Kabushiki Kaisha Method, apparatus and system for de-blocking a block of video samples
US10405000B2 (en) * 2014-11-21 2019-09-03 Vid Scale, Inc. One-dimensional transform modes and coefficient scan order
CN107005692B (zh) * 2014-11-27 2020-11-17 株式会社Kt 对视频信号进行解码或编码的方法及设备
US10306229B2 (en) * 2015-01-26 2019-05-28 Qualcomm Incorporated Enhanced multiple transforms for prediction residual
US10841601B2 (en) * 2015-06-23 2020-11-17 Telefonaktiebolaget Lm Ericsson (Publ) Methods and arrangements for transcoding
EP3354029A4 (en) * 2015-09-23 2019-08-21 Nokia Technologies Oy METHOD, APPARATUS AND COMPUTER PROGRAM PRODUCT FOR ENCODING 360-DEGREE PANORAMIC VIDEO
US10455228B2 (en) * 2016-03-21 2019-10-22 Qualcomm Incorporated Determining prediction parameters for non-square blocks in video coding

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220046241A1 (en) * 2017-07-03 2022-02-10 Panasonic Intellectual Property Corporation Of America Coding method, decoding method, encoder, and decoder
US20200329242A1 (en) * 2017-12-28 2020-10-15 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US11729386B2 (en) * 2017-12-28 2023-08-15 Panasonic Intellectual Property Corporation Of America Mode selection based on adaptive transform basis
US20230345000A1 (en) * 2017-12-28 2023-10-26 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US20210211727A1 (en) * 2018-09-02 2021-07-08 Lg Electronics Inc. Image coding method based on multiple transform selection and device therefor
US20220124330A1 (en) * 2018-10-05 2022-04-21 Lg Electronics Inc. Method for coding transform coefficient and device therefor
US20220295061A1 (en) * 2019-04-23 2022-09-15 Beijing Bytedance Network Technology Co., Ltd. Context modeling and selection of multiple transform matrix
US11930176B2 (en) * 2019-04-23 2024-03-12 Beijing Bytedance Network Technology Co., Ltd Context modeling and selection of multiple transform matrix

Also Published As

Publication number Publication date
RU2759052C2 (ru) 2021-11-09
WO2018123644A1 (ja) 2018-07-05
CN110100438A (zh) 2019-08-06
KR20190099410A (ko) 2019-08-27
KR102476148B1 (ko) 2022-12-12
AU2017387752A1 (en) 2019-07-11
CN110100438B (zh) 2022-04-15
EP3565249A4 (en) 2019-12-18
AU2017387752B2 (en) 2022-08-25
RU2019119077A (ru) 2020-12-21
BR112019012707A2 (pt) 2019-11-19
RU2019119077A3 (pt) 2021-04-05
CA3041856A1 (en) 2018-07-05
JP7067487B2 (ja) 2022-05-16
US20220394299A1 (en) 2022-12-08
EP3565249A1 (en) 2019-11-06
JPWO2018123644A1 (ja) 2019-11-21

Similar Documents

Publication Publication Date Title
US20220394299A1 (en) Image processing apparatus and method
US11695941B2 (en) Image processing device and method
US11877008B2 (en) Image processing apparatus and method
US20200304790A1 (en) Image processing apparatus and method
US11812042B2 (en) Image decoding device and method for setting information for controlling decoding of coded data
KR102393150B1 (ko) 화상 처리 장치 및 방법
US11354824B2 (en) Image processing apparatus and method
US11272180B2 (en) Image processing apparatus and method
EP3528499A1 (en) Image processing device and image processing method
US20190313095A1 (en) Image processing apparatus and image processing method
JP2020174400A (ja) 画像復号装置および方法
WO2018131523A1 (ja) 画像処理装置および画像処理方法
JP6891895B2 (ja) 画像処理装置および画像処理方法
US20200162735A1 (en) Image processing apparatus and method
US10893269B2 (en) Image processing device and method
WO2018150934A1 (ja) 画像処理装置および方法

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TSUKUBA, TAKESHI;REEL/FRAME:049433/0914

Effective date: 20190516

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION