WO2021149812A1 - 画像処理装置および方法 - Google Patents

画像処理装置および方法 Download PDF

Info

Publication number
WO2021149812A1
WO2021149812A1 PCT/JP2021/002296 JP2021002296W WO2021149812A1 WO 2021149812 A1 WO2021149812 A1 WO 2021149812A1 JP 2021002296 W JP2021002296 W JP 2021002296W WO 2021149812 A1 WO2021149812 A1 WO 2021149812A1
Authority
WO
WIPO (PCT)
Prior art keywords
coefficient data
unit
inverse
conversion
coefficient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/002296
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
健史 筑波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to US17/782,173 priority Critical patent/US12309397B2/en
Priority to KR1020227020349A priority patent/KR20220128338A/ko
Priority to CN202180009380.3A priority patent/CN114982235B/zh
Priority to EP21744921.4A priority patent/EP4087242A4/en
Priority to JP2021572823A priority patent/JP7622652B2/ja
Publication of WO2021149812A1 publication Critical patent/WO2021149812A1/ja
Anticipated expiration legal-status Critical
Priority to JP2025004343A priority patent/JP7747236B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/10—Texturing; Colouring; Generation of textures or colours
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/98—Adaptive-dynamic-range coding [ADRC]

Definitions

  • the present disclosure relates to an image processing apparatus and method, and in particular, image processing capable of suppressing an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of coefficient data after the reverse adaptive color conversion.
  • image processing capable of suppressing an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of coefficient data after the reverse adaptive color conversion.
  • Non-Patent Document 1 a coding method has been proposed in which a predicted residual of a moving image is derived, coefficient-converted, quantized and encoded (see, for example, Non-Patent Document 1 and Non-Patent Document 2). Further, as a coding tool for improving the coding efficiency in RGB444, an adaptive color transformation (ACT (Adaptive Color Transform)) for converting the coefficient data of the RGB domain into the coefficient data of the YCgCo domain has been proposed. Further, a reversible method has been proposed as the adaptive color conversion (see, for example, Non-Patent Document 3).
  • ACT Adaptive Color Transform
  • the coefficient data of the YCgCo domain that is the input signal is [-2 ⁇ bitDepth, 2 ⁇ bitDepth. It was proposed to clip in -1] (see, for example, Non-Patent Document 4).
  • the range of the coefficient data of the YCgCo domain that has been adaptively color-converted by the reversible method is wider than the range of the coefficient data of the YCgCo domain that has been adaptively color-converted by the lossy method. Therefore, when the method described in Non-Patent Document 4 is applied to the reversible reverse adaptive color conversion disclosed in Non-Patent Document 3, the coefficient data of the YCgCo domain is changed by the clip processing, and the coefficient data of the YCgCo domain is changed after the reverse adaptive color conversion. There was a risk that the distortion of the coefficient data in the RGB domain would increase.
  • the present disclosure has been made in view of such a situation, and it is possible to suppress an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data after the reverse adaptive color conversion. It is something to do.
  • the image processing device on one aspect of the present technology has a clip processing unit that clips the coefficient data that has undergone adaptive color conversion by a reversible method at a level based on the bit depth of the coefficient data, and the clip processing unit that clips the coefficient data at the level.
  • the image processing method of one aspect of the present technology is to clip the coefficient data adaptively color-converted by the lossless method at a level based on the bit depth of the coefficient data, and to obtain the coefficient data clipped at the level by the lossless method.
  • This is an image processing method for lossless color conversion.
  • the coefficient data adaptively color-converted by the lossless method is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at that level is reversible. Lossless color conversion is performed by the method.
  • Non-Patent Document 1 (above)
  • Non-Patent Document 2 (above)
  • Non-Patent Document 3 (above)
  • Non-Patent Document 4 (above)
  • Non-Patent Document 5 Recommendation ITU-T H.264 (04/2017) "Advanced video coding for generic audiovisual services", April 2017
  • Non-Patent Document 6 Recommendation ITU-T H.265 (02/18) "High efficiency video coding", february 2018
  • the contents described in the above-mentioned non-patent documents are also the basis for determining the support requirements.
  • Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly described in the examples, they are within the disclosure range of the present technology. It shall meet the support requirements of the claims.
  • technical terms such as Parsing, Syntax, and Semantics are within the scope of disclosure of the present technology even if they are not directly described in the examples. Meet the support requirements in the range of.
  • a "block” (not a block indicating a processing unit) used in the description as a partial area of an image (picture) or a processing unit indicates an arbitrary partial area in the picture unless otherwise specified. Its size, shape, characteristics, etc. are not limited.
  • “block” includes TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), and CU described in the above-mentioned non-patent documents.
  • CodingUnit LCU (LargestCodingUnit), CTB (CodingTreeBlock), CTU (CodingTreeUnit), subblock, macroblock, tile, slice, etc., any partial area (processing unit) is included.
  • the block size may be specified using the identification information that identifies the size.
  • the block size may be specified by the ratio or difference with the size of the reference block (for example, LCU, SCU, etc.).
  • the designation of the block size also includes the designation of the range of the block size (for example, the designation of the range of the allowable block size).
  • coding includes not only the entire process of converting an image into a bitstream but also a part of the process. For example, it not only includes processing that includes prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also includes processing that collectively refers to quantization and arithmetic coding, prediction processing, quantization, and arithmetic coding. Including processing, etc.
  • decoding includes not only the entire process of converting a bitstream into an image, but also some processes.
  • processing not only includes processing that includes inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also processing that includes inverse arithmetic decoding and inverse quantization, inverse arithmetic decoding, inverse quantization, and prediction processing. Including processing that includes and.
  • ACT Adaptive Color Transform
  • This adaptive color conversion is also called YCgCo conversion.
  • the reverse processing of adaptive color conversion is also referred to as inverse adaptive color conversion (Inverse ACT).
  • the reverse processing of YCgCo conversion is also called reverse YCgCo conversion.
  • This YCgCo conversion (inverse YCgCo conversion) is an irreversible method. Therefore, in the YCgCo conversion (inverse YCgCo conversion), the conversion between the coefficient data of the RGB domain and the coefficient data of the YCgCo domain can be irreversible (lossy).
  • Non-Patent Document 3 a reversible method was proposed as the adaptive color conversion.
  • This reversible adaptive color conversion is also called YCgCo-R conversion.
  • the reverse processing of YCgCo-R conversion is also called reverse YCgCo-R conversion. That is, this YCgCo-R conversion (inverse YCgCo-R conversion) is a reversible method. Therefore, in the YCgCo-R conversion (inverse YCgCo-R conversion), lossless conversion between the coefficient data of the RGB domain and the coefficient data of the YCgCo domain can be realized.
  • lossless coding (lossless coding) can be realized by applying the YCgCo-R conversion and the inverse YCgCo-R conversion.
  • the YCgCo-R conversion converts an RGB component into a YCgCo component as shown in the following equations (1) to (4).
  • the coefficient data of the RGB domain can be converted into the coefficient data of the YCgCo domain equivalent to the YCbCr domain only by a simple shift operation and addition / subtraction, as in the case of the YCgCo conversion. Can be converted to data.
  • the inverse YCgCo-R conversion is carried out as in the following equations (5) to (8).
  • Non-Patent Document 4 in the lossy method of inverse adaptive color conversion (Inverse ACT), in order to suppress an increase in the load of the inverse adaptive color conversion process, the coefficient data of the YCgCo domain serving as an input signal is [-]. It was proposed to clip with 2 ⁇ bitDepth, 2 ⁇ bitDepth-1]. Note that [A, B] indicates a range in which A is the lower limit value and B is the upper limit value. Further, “clip at [A, B]” means that the value below the lower limit value A of the input signal is set to A, and the value above the upper limit value B is set to B. bitDepth indicates the bit depth of the coefficient data of the RGB domain before adaptive color conversion.
  • the range of the coefficient data of the bit depth bitDepth is [-2 ⁇ bitDepth, 2 ⁇ bitDepth-1].
  • the range of the coefficient data of each component of the YCgCo domain is theoretically within the range of [-2 ⁇ bitDepth, 2 ⁇ bitDepth-1].
  • the coefficient data of each component of the YCgCo domain may take a value outside this range due to the influence of some external factor or the like.
  • the reverse adaptive color conversion process if the range of the input signal is expanded in this way, the load of the reverse adaptive color conversion process may increase. In particular, when it is implemented in hardware, the cost may increase.
  • the coefficient data of each component of the YCgCo domain which is the input signal for the inverse YCgCo conversion, is clipped with [-2 ⁇ bitDepth, 2 ⁇ bitDepth-1]. That is, the coefficient data r Y [x] [y], r Cb [x] [y], and r Cr [x] [y], which are the input signals of the inverse YCgCo conversion, are converted into the following equations (9) to (11). Process as follows.
  • r Y [x] [y] indicates the coefficient data of the Y component.
  • r Cb [x] [y] shows the coefficient data of the Cg component.
  • r Cr [x] [y] shows the coefficient data of the Co component.
  • Clip3 (A, B, C) shows a clip function that clips C with the lower limit value A and the upper limit value B.
  • ⁇ indicates a bit shift (that is, a power of 2).
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion by a reversible method.
  • the coefficient data r Y [x] [y], r Cb [x] [y], and r Cr [x] [y] processed as in equations (9) to (11) are as follows. It is processed as in the formulas (12) to (15).
  • the range of the coefficient data after the conversion is wider than that in the case of the YCgCo conversion.
  • the range of the coefficient data of the Y component which is the luminance component
  • the range of the coefficient data of the Cg component and Co component which are color difference components, is [-2 ⁇ (bitDepth + 1), 2 ⁇ (bitDepth + 1) -1]. That is, the dynamic range of the coefficient data of the Y component is bitDepth + 1, and the dynamic range of the coefficient data of the Cg component and the Co component is bitDepth + 2.
  • Non-Patent Document 4 when the method described in Non-Patent Document 4 is applied to the reversible adaptive color conversion disclosed in Non-Patent Document 3, the coefficient data of the YCgCo domain is changed by the clip processing, and RGB after the inverse adaptive color conversion is performed. There was a risk of increased distortion of domain coefficient data. Note that this "distortion" indicates a discrepancy with respect to the coefficient data of the RGB domain before the inverse adaptive color conversion, that is, the difference in the coefficient data of the RGB domain before and after the inverse adaptive color conversion. That is, with such a method, it may be difficult to realize lossless conversion between the coefficient data of the RGB domain and the coefficient data of the YCgCo domain.
  • FIG. 1 is a diagram showing an example of the relationship between the input value and the output value when the residual signal of the RGB domain is converted to YCgCo or YCgCo-R.
  • FIG. 1 shows an example of the relationship between the G component (input value) and the Y component (output value).
  • FIG. 1B shows an example of the relationship between the B component (input value) and the Cg component (output value).
  • FIG. 1C shows an example of the relationship between the R component (input value) and the Co component (output value).
  • the gray circle shows an example of the relationship between the input value and the output value in the case of YCgCo conversion.
  • the white circle shows an example of the relationship between the input value and the output value in the case of YCgCo-R conversion.
  • the thick solid line shows an example of the upper limit of the clip.
  • the thick dotted line shows an example of the lower limit of the clip.
  • the coefficient data of the Cg component and the coefficient data of the Co component after the YCgCo-R conversion can exceed the upper limit value and the lower limit value of the clip.
  • the value above the upper limit value is clipped to the upper limit value
  • the value below the lower limit value is clipped to the lower limit value. That is, the values of some coefficient data change. Therefore, there is a risk that distortion (mismatch with that before YCgCo-R conversion) will increase in the coefficient data after the inverse YCgCo-R conversion.
  • the reverse adaptive color conversion becomes irreversible in this way, there is a risk that it becomes difficult to realize lossless coding in the coding to which the reverse adaptive color conversion is applied. Further, the increase in the distortion of the coefficient data of the RGB domain after the inverse adaptive color conversion increases the difference between the decoded image and the image before encoding, which may reduce the quality of the decoded image.
  • the coefficient data obtained by the lossless method of adaptive color conversion which is the input of the lossless method of inverse adaptive color conversion, is clipped at a level based on the bit depth of the coefficient data.
  • the coefficient data that has been adaptively color-converted by the lossless method is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at that level is inversely adaptive color-converted by the lossless method. To do so.
  • a clip processing unit that clips adaptive color-converted coefficient data by a lossless method at a level based on the bit depth of the coefficient data, and a coefficient data clipped at that level by the clip processing unit.
  • a reverse adaptive color conversion unit that performs reverse adaptive color conversion by a lossless method is provided.
  • the range between the upper and lower limits of the clip processing includes the range that the coefficient data of the adaptive color conversion by the reversible method can theoretically take (that is, the range that the coefficient data can theoretically take).
  • the luminance component and the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method may be clipped at the same level as each other.
  • the coefficient data that has undergone adaptive color conversion by the reversible method is composed of a Y component that is a luminance component, a Cg component that is a color component (color difference component), and a Co component that is a color component (color difference component). That is, all these components may be clipped at the same level as each other. By doing so, the clipping process can be easily performed and the increase in load can be suppressed as compared with the case where each component is clipped at a different level.
  • the brightness component and color component of the coefficient data are defined by subtracting 1 from the power of 2 with the value obtained by adding 1 to the bit depth of the coefficient data adaptively color-converted by the reversible method as the power index.
  • Color difference component may be clipped.
  • the brightness component and color of the coefficient data are defined by multiplying the power of 2 by -1 to the power of 2 with the value obtained by adding 1 to the bit depth of the coefficient data adaptively color-converted by the reversible method as the power index.
  • the component (color difference component) may be clipped. Further, both the upper limit clip and the lower limit clip may be performed.
  • the upper limit value actResMax of the clip processing performed on the coefficient data subjected to the adaptive color conversion by the reversible method is set as the following equation (16) using the value obtained by adding 1 to the bit depth of the coefficient data.
  • the lower limit value actResMin of the clip processing performed on the coefficient data subjected to the adaptive color conversion by the reversible method is set as the following equation (17) using the value obtained by adding 1 to the bit depth of the coefficient data. do.
  • the upper limit value actResMax is set to the value obtained by subtracting 1 from the power of 2 whose power index is the value obtained by adding 1 to the bit depth.
  • the lower limit value actResMin is set to a value obtained by multiplying the power of 2 having the value obtained by adding 1 to the bit depth as the power index by -1. Then, as shown in the following equations (18) to (20), the luminance component and the color component (color difference) of the coefficient data adaptively color-converted by the reversible method using the upper limit value actResMax and the lower limit value actResMin. Ingredients) are clipped.
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion as in the above equations (12) to (15) to derive the coefficient data of the RGB domain.
  • the range between the upper limit value and the lower limit value of the clip processing can be made wider than the range that can theoretically be obtained by the coefficient data of the adaptive color conversion by the reversible method. Therefore, by clipping with such an upper limit value and a lower limit value, it is possible to suppress the occurrence of distortion of the coefficient data after the inverse adaptive color conversion. That is, it is possible to suppress an increase in the load of the inverse adaptive color conversion process while realizing lossless conversion between the coefficient data of the RGB domain and the coefficient data of the YCgCo domain.
  • the clip processing may be performed in consideration of the dynamic range of the buffer.
  • the coefficient data that has undergone adaptive color conversion in a reversible manner is clipped at a level based on its bit depth and the dynamic range of the buffer that holds the coefficient data during inverse adaptive color conversion. It may be.
  • clipping the coefficient data in consideration of the dynamic range (hardware limitation) of the buffer in this way it is possible to suppress the occurrence of buffer overflow.
  • the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, it is possible to suppress an increase in the dynamic range of the buffer and suppress an increase in cost. ..
  • the clipping level is based on the bit depth of the adaptive color-converted coefficient data by the reversible method, and is not based on the dynamic range of the buffer that holds the coefficient data, and is not based on the bit depth.
  • the value derived using the smaller of the values based on the dynamic range of the buffer may be used. In this way, by setting the upper and lower limits using the smaller values (that is, the coefficient data is made into a narrower range by clipping), the hardware limit of the buffer is satisfied, and vice versa. It is possible to suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data after the adaptive color conversion.
  • the smaller of the value obtained by adding 1 to the bit depth of the coefficient data subjected to the adaptive color conversion by the reversible method and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data is called the exponentiation index.
  • the brightness component and the color component (color difference component) of the coefficient data may be clipped with the value obtained by subtracting 1 from the power of 2 to be the upper limit value.
  • the smaller of the value obtained by adding 1 to the bit depth of the coefficient data subjected to the adaptive color conversion by the reversible method and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data is called the exponentiation index.
  • the brightness component and the color component (color difference component) of the coefficient data may be clipped with the value obtained by multiplying the power of 2 by -1 as the lower limit value. Further, both the upper limit clip and the lower limit clip may be performed.
  • the upper limit value actResMax of the clip processing performed on the coefficient data that has been adaptively color-converted by the reversible method is 1 from the value obtained by adding 1 to the bit depth of the coefficient data and the dynamic range of the buffer that holds the coefficient data. Is set as in the following equation (21) using the value obtained by subtracting. Further, the lower limit value actResMin of the clip processing performed on the coefficient data subjected to the adaptive color conversion by the reversible method is set to 1 from the value obtained by adding 1 to the bit depth of the coefficient data and the dynamic range of the buffer holding the coefficient data. Is set as in the following equation (22) using the value obtained by subtracting.
  • the upper limit value actResMax is derived from the power of 2 with the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. The value is obtained by subtracting 1.
  • the lower limit value actResMin is set to the power of 2 with the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. It is a value multiplied by -1. Then, using the upper limit value actResMax and the lower limit value actResMin, as shown in the above equations (18) to (20), the luminance component and the color component (color difference) of the coefficient data adaptively color-converted by the reversible method are used. Ingredients) are clipped.
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion as in the above equations (12) to (15) to derive the coefficient data of the RGB domain.
  • ⁇ Method 2> As described above in ⁇ Clip processing in the reversible adaptive color conversion>, the range of the coefficient data of the Cg component and the Co component, which are the color components (color difference components), is larger than the range of the coefficient data of the Y component, which is the luminance component. wide.
  • the luminance component and the color difference component of the coefficient data may be clipped at each level.
  • the luminance component of the coefficient data may be clipped at the first level
  • the color component (color difference component) of the coefficient data may be clipped at the second level.
  • the width of the clip (between the upper limit value and the lower limit value) becomes unnecessarily wide for a component with a narrower range, and the reduction in the load of the inverse adaptive color conversion process is suppressed. Will be done.
  • clipping each component with a width corresponding to the component it is possible to further suppress an increase in the load of the inverse adaptive color conversion process.
  • the difference between the upper limit value and the lower limit value of the second level may be wider than the difference between the upper limit value and the lower limit value of the first level.
  • the range of the coefficient data of the Cg component and the Co component, which are the color difference components is wider than the range of the coefficient data of the Y component, which is the luminance component. Therefore, the difference between the upper and lower limits of the clip for the coefficient data of the Cg component and the Co component, which are the color difference components, is wider than the difference between the upper and lower limits of the clip for the coefficient data of the Y component, which is the brightness component.
  • the brightness component of the coefficient data may be clipped with the value obtained by subtracting 1 from the power of 2 having the bit depth of the coefficient data adaptively color-converted by the reversible method as the power index as the upper limit value. Further, the brightness component of the coefficient data may be clipped with the value obtained by multiplying the power of 2 having the bit depth of the coefficient data adaptively color-converted by the reversible method as the exponent by -1 as the lower limit value. In addition, both such upper and lower limit clipping for the luminance component may be performed.
  • the color component (color difference component) of the coefficient data is set to the upper limit value of the value obtained by subtracting 1 from the power of 2 whose power index is the value obtained by adding 1 to the bit depth of the coefficient data adaptively color-converted by the reversible method. ) May be clipped. Further, the value obtained by adding 1 to the bit depth of the coefficient data adaptively color-converted by the reversible method is used as the power index, and the value obtained by multiplying the power of 2 by -1 is set as the lower limit value, and the color component (color difference) of the coefficient data Ingredients) may be clipped. Further, both the clip of the upper limit value and the clip of the lower limit value for the color component (color difference component) may be performed. Further, as described above, the upper limit value clip and the lower limit value clip for each of the luminance component and the color component (color difference component) may be performed.
  • the upper limit value actResMaxY of the clip processing performed on the luminance component of the coefficient data adaptively color-converted by the reversible method is set as in the following equation (23) using the bit depth of the coefficient data.
  • the lower limit value actResMinY of the clip processing performed on the luminance component of the coefficient data adaptively color-converted by the reversible method is set by the following equation (24) using the bit depth of the coefficient data.
  • the upper limit value actResMaxC of the clip processing performed on the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method is set to the upper limit value actResMaxC of the coefficient data by using the bit depth of the coefficient data as in the following equation (25). Set.
  • the lower limit value actResMinC of the clip processing performed on the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method is set to the lower limit value actResMinC of the coefficient data as in the following equation (26) using the bit depth of the coefficient data. Set.
  • the upper limit value actResMaxY for the luminance component is set to a value obtained by subtracting 1 from the power of 2 with the bit depth as the power index.
  • the lower limit value actResMinY for the luminance component is set to a value obtained by multiplying the power of 2 with the bit depth as a power index by -1. Then, using the upper limit value actResMaxY and the lower limit value actResMinY, the luminance component of the coefficient data subjected to the adaptive color conversion by the reversible method is clipped as shown in the following equation (27).
  • the upper limit value actResMaxC for the color component (color difference component) is set to the value obtained by subtracting 1 from the power of 2 whose power index is the value obtained by adding 1 to the bit depth.
  • the lower limit value actResMinC for the color component (color difference component) is set to a value obtained by multiplying the power of 2 having the value obtained by adding 1 to the bit depth as the power index by -1. Then, using the upper limit value actResMaxC and the lower limit value actResMinC, as shown in the following equations (28) and (29), the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method is obtained. Clip.
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion as in the above equations (12) to (15) to derive the coefficient data of the RGB domain.
  • the coefficient data of each component can be clipped in the range corresponding to the component, so that the inverse adaptive color conversion process can be performed while suppressing the increase in distortion of the coefficient data after the inverse adaptive color conversion.
  • the increase in load can be further suppressed.
  • the clip processing may be performed in consideration of the dynamic range of the buffer.
  • the coefficient data that has undergone adaptive color conversion in a reversible manner is clipped at a level based on its bit depth and the dynamic range of the buffer that holds the coefficient data during inverse adaptive color conversion. It may be. That is, even if the above-mentioned first level and the second level are values based on the bit depth of the coefficient data that has been losslessly color-converted and the dynamic range of the buffer that stores the coefficient data, respectively. good.
  • By clipping the coefficient data in consideration of the dynamic range (hardware limitation) of the buffer in this way it is possible to suppress the occurrence of buffer overflow.
  • the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, it is possible to suppress an increase in the dynamic range of the buffer and suppress an increase in cost. ..
  • the level at which the brightness component is clipped is based on the bit depth of the coefficient data that has been adaptively color-converted by the reversible method, and is not based on the dynamic range of the buffer that holds the coefficient data and the bit depth. And, it may be a value derived by using the smaller of the values based on the dynamic range of the buffer. Also, the level at which the color component (color difference component) is clipped is based on the value obtained by adding 1 to the bit depth of the coefficient data that has been adaptively color-converted by the reversible method, and based on the dynamic range of the buffer that holds the coefficient data.
  • It may be a value derived by using the smaller of the non-existent value and the value based on the dynamic range of the buffer, which is not based on the value obtained by adding 1 to the bit depth.
  • the hardware limit of the buffer is satisfied, and vice versa. While suppressing the increase in distortion of each component of the coefficient data after the adaptive color conversion, it is possible to further suppress the increase in the load of the inverse adaptive color conversion process.
  • the brightness component of the coefficient data may be clipped with the value obtained by subtracting 1 as the upper limit value.
  • the smaller one is the power of 2.
  • the brightness component of the coefficient data may be clipped with the value multiplied by -1 as the lower limit value. In addition, both such upper and lower limit clipping for the luminance component may be performed.
  • the smaller of the value obtained by adding 1 to the bit depth of the coefficient data subjected to the adaptive color conversion by the reversible method and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data is called the exponentiation index.
  • the color component (color difference component) of the coefficient data may be clipped with the value obtained by subtracting 1 from the power of 2 to be the upper limit value.
  • the smaller of the value obtained by adding 1 to the bit depth of the coefficient data subjected to the adaptive color conversion by the reversible method and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data is called the exponentiation index.
  • the color component (color difference component) of the coefficient data may be clipped with the value obtained by multiplying the power of 2 by -1 as the lower limit value. Further, both the clip of the upper limit value and the clip of the lower limit value for the color component (color difference component) may be performed. Further, as described above, the upper limit value clip and the lower limit value clip for each of the luminance component and the color component (color difference component) may be performed.
  • the upper limit value actResMaxY of the clip processing performed on the brightness component of the coefficient data adaptively color-converted by the reversible method is subtracted by 1 from the bit depth of the coefficient data and the dynamic range of the buffer holding the coefficient data.
  • the lower limit value actResMinY of the clip processing performed on the brightness component of the coefficient data adaptively color-converted by the reversible method is subtracted by 1 from the bit depth of the coefficient data and the dynamic range of the buffer holding the coefficient data.
  • the upper limit value actResMaxC of the clip processing performed on the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method is held as the value obtained by adding 1 to the bit depth of the coefficient data and the coefficient data.
  • the setting is made as shown in the following equation (32).
  • the lower limit value actResMinC of the clip processing performed on the color component (color difference component) of the coefficient data adaptively color-converted by the reversible method is retained by adding 1 to the bit depth of the coefficient data and the coefficient data.
  • the setting is made as shown in the following equation (33).
  • the upper limit value actResMaxY for the brightness component is set to 1 from the power of 2 with the smaller one of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. Let it be the subtracted value.
  • the lower limit value actResMinY for the brightness component is -1 to the power of 2 with the smaller one of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. Is the value multiplied by.
  • the upper limit value actResMaxY and the lower limit value actResMinY the luminance component of the coefficient data adaptively color-converted by the reversible method is clipped as shown in the above equation (27).
  • the upper limit value actResMaxC for the color component (color difference component) is the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. The value is obtained by subtracting 1 from the exponent of 2 to the power of 2.
  • the lower limit value actResMinC for the color component (color difference component) is the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. It is a value obtained by multiplying the exponent of 2 by -1.
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion as in the above equations (12) to (15) to derive the coefficient data of the RGB domain.
  • the increase in the load of the inverse adaptive color conversion process is further suppressed while satisfying the hardware limitation of the buffer and suppressing the increase in the distortion of each component of the coefficient data after the inverse adaptive color conversion. be able to.
  • ⁇ Method 3> It may be clipped at a level based on the dynamic range of the buffer that holds the coefficient data during the inverse adaptive color conversion.
  • By clipping the coefficient data in consideration of the dynamic range (hardware limitation) of the buffer in this way it is possible to suppress the occurrence of buffer overflow.
  • the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, it is possible to suppress an increase in the dynamic range of the buffer and suppress an increase in cost. ..
  • the brightness component of the coefficient data is set to the upper limit of the value obtained by subtracting 1 from the power of 2 with the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data during inverse adaptive color conversion as the power index.
  • the color component (color difference component) may be clipped.
  • the luminance component and the color component (color difference component) of the coefficient data are clipped with the value obtained by subtracting 1 from the dynamic range of the buffer as the power index and multiplying the power of 2 by -1 as the lower limit value. You may.
  • the upper limit value actResMax of the clip processing performed on the coefficient data of the adaptive color conversion by the reversible method is set as the following equation (34) using the value obtained by subtracting 1 from the dynamic range of the buffer. ..
  • the lower limit value actResMin of the clip processing performed on the coefficient data subjected to the adaptive color conversion by the reversible method is set as the following equation (35) using the value obtained by subtracting 1 from the dynamic range of the buffer. ..
  • the upper limit value actResMax is set to the value obtained by subtracting 1 from the dynamic range of the buffer and subtracting 1 from the power of 2 which is the exponent.
  • the lower limit value actResMin is set to a value obtained by multiplying the power of 2 having the value obtained by subtracting 1 from the dynamic range of the buffer as the exponent by -1.
  • the coefficient data clipped in this way is subjected to inverse YCgCo-R conversion as in the above equations (12) to (15) to derive the coefficient data of the RGB domain.
  • the hardware limitation of the buffer can be satisfied, and the occurrence of buffer overflow can be suppressed. Moreover, the increase in cost can be suppressed. Further, since the range of the coefficient data is limited by the clip processing, it is possible to suppress an increase in the load of the inverse adaptive color conversion processing. Further, when the range of the coefficient data is narrower than the dynamic range of the buffer, it is possible to suppress an increase in distortion of the coefficient data after the inverse adaptive color conversion by doing so.
  • one or both of the upper limit value clip and the lower limit value clip may be selectively applied. For example, when the lower limit value of the clip is equal to or less than the lower limit value of the range of the coefficient data, the clip of the lower limit value may be omitted (skip). Further, when the upper limit value of the clip is equal to or larger than the upper limit value of the range of the coefficient data, the clip of the upper limit value may be omitted (skipped).
  • FIG. 2 is a block diagram showing an example of the configuration of a reverse adaptive color conversion device, which is an aspect of an image processing device to which the present technology is applied.
  • the inverse adaptive color conversion device 100 shown in FIG. 2 converts the coefficient data of the YCgCo domain in which the coefficient data of the RGB domain related to the image is adaptive color converted (YCgCo-R conversion) by the lossless method into the inverse adaptive color conversion (YCgCo-R conversion) of the lossless method. It is a device that performs inverse YCgCo-R conversion).
  • FIG. 2 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the inverse adaptive color conversion device 100, there may be a processing unit that is not shown as a block in FIG. 2, or there may be a processing or data flow that is not shown as an arrow or the like in FIG.
  • the inverse adaptive color conversion device 100 includes a selection unit 101, a clip processing unit 102, and an inverse YCgCo-R conversion unit 103.
  • the selection unit 101 acquires the coefficient data res_x'input to the inverse adaptive color conversion device 100. Further, the selection unit 101 acquires the cu_act_enabled_flag input to the inverse adaptive color conversion device 100.
  • cu_act_enabled_flag is flag information indicating whether or not adaptive color conversion (reverse adaptive color conversion) is applicable. When this cu_act_enabled_flag is true (for example, "1"), it indicates that adaptive color conversion (reverse adaptive color conversion) can be applied. When this cu_act_enabled_flag is false (for example, "0"), it indicates that the application of adaptive color conversion (reverse adaptive color conversion) is prohibited (that is, it is not applicable).
  • the selection unit 101 selects whether or not to perform inverse adaptive color conversion of the coefficient data res_x'based on this cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (for example, "1"), the selection unit 101 determines that the coefficient data res_x'is the coefficient data of the YCgCo domain obtained by converting the coefficient data of the RGB domain into YCgCo-R. Then, the selection unit 101 supplies the coefficient data res_x'to the clip processing unit 102 so as to execute the inverse YCgCo-R conversion on the coefficient data res_x'.
  • the selection unit 101 determines that the coefficient data res_x'is the coefficient data of the RGB domain. Then, the selection unit 101 outputs the coefficient data res_x'as the coefficient data res_x after the reverse application color conversion to the outside of the reverse adaptive color conversion device 100. That is, the coefficient data of the RGB domain is output to the outside of the inverse adaptive color conversion device 100.
  • the clip processing unit 102 acquires the coefficient data res_x'supplied from the selection unit 101. Further, the clip processing unit 102 acquires variables such as log2MaxDR and BitDepth input to the inverse adaptive color conversion device 100.
  • log2MaxDR indicates the dynamic range of the buffer that holds the coefficient data res_x'during the inverse adaptive color conversion process.
  • BitDepth indicates the bit depth of the coefficient data res_x'.
  • the clip processing unit 102 executes clip processing on the coefficient data res_x'using the upper limit value and the lower limit value derived based on variables such as log2MaxDR and BitDepth.
  • the clip processing unit 102 supplies the coefficient data res_x'after the clip processing to the inverse YCgCo-R conversion unit 103.
  • the inverse YCgCo-R conversion unit 103 acquires the coefficient data res_x'supplied from the clip processing unit 102 to which the clip processing has been executed.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the acquired coefficient data res_x'and generates the inverse YCgCo-R-converted coefficient data res_x.
  • the inverse YCgCo-R conversion unit 103 outputs the generated coefficient data res_x to the outside of the inverse adaptive color conversion device 100. That is, the coefficient data of the RGB domain is output to the outside of the inverse adaptive color conversion device 100.
  • the clip processing unit 102 clips the coefficient data res_x'at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R conversion unit 103 performs reverse adaptive color conversion of the coefficient data res_x'clipped at that level by the clip processing unit 102 in a reversible manner.
  • the clip processing unit 102 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color conversion does not increase. Therefore, the reverse adaptive color conversion device 100 can suppress an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the reverse adaptive color conversion.
  • the reverse adaptive color converter 100 is ⁇ 1. Clip processing of reverse adaptive color conversion of reversible method> Various methods of the present technology described above ("Method 1", “Method 1-1”, “Method 1-2”, “Method 2", “Method 2-1” , “Method 2-2", and “Method 3”) may be applied.
  • the clip processing unit 102 may clip the luminance component and the color component (color difference component) of the coefficient data res_x'at the same level as each other, as described above in ⁇ Method 1>.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the clip processing unit 102 can easily perform the clip process as compared with the case where each component is clipped at a different level, and the increase in the load can be suppressed.
  • the clip processing unit 102 uses the value obtained by adding 1 to the bit depth of the coefficient data res_x'as the exponentiation, and the upper limit is the value obtained by subtracting 1 from the power of 2.
  • the brightness component and the color component (color difference component) of the coefficient data res_x' may be clipped.
  • the clip processing unit 102 sets the value obtained by adding 1 to the bit depth of the coefficient data res_x'as the exponent, and multiplying the power of 2 by -1 as the lower limit value, and sets the brightness component of the coefficient data res_x'.
  • the color component (color difference component) may be clipped.
  • the clip processing unit 102 may perform both the clip of the upper limit value and the clip of the lower limit value.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the clip processing unit 102 can make the range between the upper limit value and the lower limit value of the clip processing wider than the range that the coefficient data res_x'can theoretically take. Therefore, the inverse adaptive color conversion device 100 can suppress the occurrence of distortion of the coefficient data res_x after the inverse adaptive color conversion. That is, the inverse adaptive color conversion device 100 can suppress an increase in the load of the inverse adaptive color conversion process while realizing lossless conversion between the coefficient data of the RGB domain and the coefficient data of the YCgCo domain.
  • the clip processing unit 102 holds the coefficient data res_x'in the bit depth and the coefficient data res_x' in the case of inverse adaptive color conversion. It may be clipped at a level based on the dynamic range.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow.
  • the dynamic range of the buffer can be set without considering the bit depth of the coefficient data res_x'to be held, an increase in the dynamic range of the buffer can be suppressed, and the inverse adaptive color converter 100 can be used. The increase in cost can be suppressed.
  • the clip processing unit 102 has a value that is based on the bit depth of the coefficient data res_x'and is not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value that is not based on the bit depth and of the buffer. Clipping may be performed at a level derived using the smaller of the values based on the dynamic range.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color conversion device 100 is buffered. It is possible to suppress an increase in the load of the inverse adaptive color conversion process while satisfying the hardware limitation of the above and suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.
  • the clip processing unit 102 sets the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x'.
  • the brightness component and color component (color difference component) of the coefficient data res_x' may be clipped with the value obtained by subtracting 1 from the power of 2 as the exponent as the upper limit value.
  • the clip processing unit 102 sets the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x'.
  • the brightness component and color component (color difference component) of the coefficient data res_x' may be clipped with the value obtained by multiplying the power of 2 as the exponent by -1 as the lower limit value. Further, the clip processing unit 102 may perform both the clip of the upper limit value and the clip of the lower limit value.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color conversion device 100 satisfies the hardware limitation of the buffer and suppresses the increase in the distortion of the coefficient data res_x after the inverse adaptive color conversion, while loading the inverse adaptive color conversion process. Can be suppressed.
  • the clip processing unit 102 may clip the luminance component and the color difference component of the coefficient data res_x'at their respective levels (first level and second level) as described above in ⁇ Method 2>. .. By doing so, the clip processing unit 102 can clip the coefficient data of each component in the range corresponding to the component.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x. Therefore, the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.
  • the difference between the upper limit value and the lower limit value of the second level may be wider than the difference between the upper limit value and the lower limit value of the first level. ..
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the difference between the upper and lower limits of the clip for the coefficient data res_x'of the Cg component and Co component, which is the color difference component, is calculated from the difference between the upper and lower limits of the clip for the coefficient data res_x' of the Y component, which is the brightness component.
  • the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.
  • the clip processing unit 102 may clip the brightness component of the coefficient data res_x'with the value obtained by subtracting 1 from the power of 2 having the bit depth of the coefficient data res_x' as the power index as the upper limit value. Further, the brightness component of the coefficient data res_x'may be clipped with the value obtained by multiplying the power of 2 having the bit depth of the coefficient data res_x' as the exponent by -1 as the lower limit value. Further, the clip processing unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value with respect to the luminance component.
  • the clip processing unit 102 sets the value obtained by subtracting 1 from the power of 2 having the value obtained by adding 1 to the bit depth of the coefficient data res_x'as the exponent, as the upper limit value, and the color component of the coefficient data res_x'( The color difference component) may be clipped. Further, the value obtained by adding 1 to the bit depth of the coefficient data res_x'is used as the exponent, and the value obtained by multiplying the power of 2 by -1 is set as the lower limit, and the color component (color difference component) of the coefficient data res_x' is clipped. You may. Further, the clip processing unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value with respect to the color component (color difference component). Further, the clip processing unit 102 may perform all the clipping of the upper limit value and the clipping of the lower limit value for each of the luminance component and the color component (color difference component) as described above.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the luminance component and the color component (color difference component) of the coefficient data res_x' clipped in this way, and derives the coefficient data res_x. By doing so, the clip processing unit 102 can clip the coefficient data res_x'of each component in the range corresponding to the component. Therefore, the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.
  • the clip processing unit 102 holds the coefficient data res_x'in the bit depth and the coefficient data res_x' in the case of inverse adaptive color conversion. Clipping may be done at levels based on the dynamic range (first and second levels).
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the luminance component and the color component (color difference component) of the coefficient data res_x' clipped in this way, and derives the coefficient data res_x. By doing so, the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data res_x'to be held, an increase in the dynamic range of the buffer can be suppressed, and the inverse adaptive color converter 100 can be used. The increase in cost can be suppressed.
  • the clip processing unit 102 has a value that is based on the bit depth of the coefficient data res_x'and is not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value that is not based on the bit depth and of the buffer. Clipping may be performed on the brightness component of the coefficient data res_x'at the level derived using the smaller of the values based on the dynamic range. Further, the clip processing unit 102 sets a value based on the value obtained by adding 1 to the bit depth of the coefficient data res_x'and not based on the dynamic range of the buffer holding the coefficient data res_x', and 1 for the bit depth. Clip processing is executed for the color component (color difference component) of the coefficient data res_x'at a level derived using the smaller of the values based on the dynamic range of the buffer and not based on the added value. You may.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color conversion device 100 can be used as a buffer hardware. It is possible to further suppress an increase in the load of the inverse adaptive color conversion process while satisfying the wear limitation and suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color conversion.
  • the clip processing unit 102 uses the smaller of the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x', which is the power of 2.
  • the brightness component of the coefficient data res_x' may be clipped with the value obtained by subtracting 1 from the power as the upper limit value.
  • the clip processing unit 102 uses the smaller of the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x', which is the power of 2.
  • the brightness component of the coefficient data res_x' may be clipped with the value obtained by multiplying the power by -1 as the lower limit value.
  • the clip processing unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value with respect to the luminance component.
  • the clip processing unit 102 sets the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x'.
  • the color component (color difference component) of the coefficient data res_x' may be clipped with the value obtained by subtracting 1 from the power of 2 as the exponent as the upper limit value.
  • the clip processing unit 102 sets the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x'and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x'.
  • the color component (color difference component) of the coefficient data res_x' may be clipped with the value obtained by multiplying the power of 2 as the exponent by -1 as the lower limit value. Further, the clip processing unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value with respect to the color component (color difference component). Further, the clip processing unit 102 may perform all the clipping of the upper limit value and the clipping of the lower limit value for each of the luminance component and the color component (color difference component) as described above.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color conversion device 100 satisfies the hardware limitation of the buffer and suppresses an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color conversion, while performing the inverse adaptive color conversion. The increase in processing load can be further suppressed.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x. By doing so, the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, an increase in the dynamic range of the buffer can be suppressed, and the cost of the inverse adaptive color converter 100 can be set. The increase can be suppressed.
  • the upper limit value of the clip processing unit 102 is the value obtained by subtracting 1 from the power of 2 whose power index is the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x'during the inverse adaptive color conversion.
  • the brightness component and the color component (color difference component) of the coefficient data res_x' may be clipped.
  • the clip processing unit 102 sets the value obtained by subtracting 1 from the dynamic range of the buffer as the exponent of 2 multiplied by -1 as the lower limit, and sets the brightness component and color of the coefficient data res_x'.
  • the component (color difference component) may be clipped.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clip-processed in this way, and derives the coefficient data res_x.
  • the inverse adaptive color converter 100 can satisfy the hardware limitation of the buffer and suppress the occurrence of buffer overflow. Further, it is possible to suppress an increase in the cost of the reverse adaptive color conversion device 100. Further, since the range of the coefficient data is limited by the clip processing, the inverse adaptive color conversion device 100 can suppress an increase in the load of the inverse adaptive color conversion processing. Further, when the range of the coefficient data is narrower than the dynamic range of the buffer, the inverse adaptive color conversion device 100 can suppress an increase in distortion of the coefficient data after the inverse adaptive color conversion by doing so.
  • the reverse adaptive color conversion device 100 can apply the various application examples described above in ⁇ Application Examples>.
  • the selection unit 101 of the reverse adaptive color conversion device 100 determines in step S101 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S102.
  • step S102 the clip processing unit 102 clips the coefficient data res_x'using a predetermined upper limit value and lower limit value.
  • step S103 the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clipped in step S102, and derives the coefficient data res_x after the inverse adaptive color conversion.
  • step S104 the inverse YCgCo-R conversion unit 103 outputs the derived coefficient data res_x to the outside of the inverse adaptive color conversion device 100.
  • the inverse adaptive color conversion process is completed.
  • step S101 If it is determined in step S101 that cu_act_enabled_flag is false, the process proceeds to step S105.
  • step S105 the selection unit 101 outputs the coefficient data res_x'as the coefficient data res_x after the inverse adaptive color conversion to the outside of the inverse adaptive color conversion device 100.
  • the inverse adaptive color conversion process is completed.
  • step S102 the clip processing unit 102 clips the coefficient data res_x'at a level based on the bit depth of the coefficient data.
  • step S103 the inverse YCgCo-R conversion unit 103 reverse-adaptive color-converts the coefficient data res_x'clipped at that level by the clip processing unit 102 in a reversible manner.
  • the clip processing unit 102 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color conversion does not increase. Therefore, the reverse adaptive color conversion device 100 can suppress an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the reverse adaptive color conversion.
  • FIG. 4 is a block diagram showing an example of the configuration of an inverse quantization inverse transformation apparatus, which is an aspect of an image processing apparatus to which the present technology is applied.
  • the coefficient data of the RGB domain related to the image is subjected to adaptive color conversion (YCgCo-R conversion) by a reversible method, orthogonally converted, and the quantized quantization coefficient is obtained.
  • Such a process (a process executed by the inverse quantization inverse transform device 200) is also referred to as an inverse quantization inverse transform process.
  • FIG. 4 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the inverse quantization inverse conversion device 200, there may be a processing unit not shown as a block in FIG. 4, or there may be a processing or data flow not shown as an arrow or the like in FIG. ..
  • the inverse quantization inverse conversion device 200 has an inverse quantization unit 201, an inverse orthogonal transformation unit 202, and an inverse adaptive color conversion unit 203.
  • the inverse quantization unit 201 acquires the quantization coefficient qcoef_x.
  • This quantization coefficient qcoef_x is derived by quantizing the orthogonal transformation coefficient coef_x by a predetermined method.
  • the inverse quantization unit 201 acquires parameters necessary for inverse quantization, such as quantization parameters qP, cu_act_enabled_flag, and transform_skip_flag.
  • the transform_skip_flag is flag information indicating whether or not the inverse orthogonal transform process is skipped (omitted). For example, if transform_skip_flag is true, the inverse orthogonal transform process is skipped. If transform_skip_flag is false, the inverse orthogonal transform process is executed.
  • the inverse quantization unit 201 inversely quantizes the quantization coefficient qcoef_x by a predetermined method corresponding to the above-mentioned quantization using these parameters, and derives the orthogonal transformation coefficient coef_x.
  • the inverse quantization unit 201 supplies the derived orthogonal transformation coefficient coef_x to the inverse orthogonal transform unit 202.
  • the inverse orthogonal transform unit 202 acquires the orthogonal transform coefficient coef_x supplied from the inverse quantization unit 201.
  • This orthogonal transformation coefficient coef_x is derived by orthogonally transforming the coefficient data res_x'by a predetermined method. Further, the inverse orthogonal transform unit 202 acquires parameters necessary for inverse quantization such as conversion information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx.
  • mts_idx is an identifier of MTS (Multiple Transform Selection).
  • lfnst_idx is the mode information related to the low frequency secondary conversion.
  • the inverse orthogonal transform unit 202 uses these parameters to inversely transform the orthogonal transform coefficient coef_x by a predetermined method corresponding to the above-mentioned orthogonal transform, and derives the coefficient data res_x'applied color-converted by a reversible method. do.
  • the inverse orthogonal transform unit 202 supplies the derived coefficient data res_x'to the inverse adaptive color converter 203.
  • the inverse adaptive color conversion unit 203 acquires the coefficient data res_x'supplied from the inverse orthogonal transform unit 202. This coefficient data res_x'is as described in the first embodiment. Further, the inverse adaptive color conversion unit 203 acquires the cu_act_enabled_flag. The inverse adaptive color conversion unit 203 performs inverse adaptive color conversion (inverse YCgCo-R conversion) of the coefficient data res_x'based on cu_act_enabled_flag as appropriate by a reversible method, and derives the coefficient data res_x after the inverse adaptive color conversion process. This coefficient data res_x is as described in the first embodiment, and is the coefficient data of the RGB domain.
  • the inverse adaptive color conversion unit 203 outputs the derived coefficient data res_x to the outside of the inverse quantization inverse conversion device 200.
  • ⁇ Application of this technology to inverse quantization inverse converter> In such an inverse quantization inverse converter 200, ⁇ 1.
  • the above-mentioned technique can be applied in the clip processing of the reverse adaptive color conversion of the reversible method. That is, the inverse quantization inverse conversion device 200 can apply the inverse adaptive color conversion apparatus 100 described in the first embodiment as the inverse adaptive color conversion unit 203. In that case, the reverse adaptive color conversion unit 203 has the same configuration as the reverse adaptive color conversion device 100, and performs the same processing.
  • the clip processing unit 102 clips the coefficient data res_x'supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clipped at that level by the clip processing unit 102, and derives the coefficient data res_x after the inverse adaptive color conversion processing. Then, the inverse YCgCo-R conversion unit 103 outputs the derived coefficient data res_x to the outside of the inverse quantization inverse conversion device 200.
  • the inverse adaptive color conversion unit 203 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color conversion does not increase. Therefore, the reverse adaptive color conversion unit 203 suppresses an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the reverse adaptive color conversion, as in the case of the reverse adaptive color conversion device 100. can do. Therefore, the inverse quantization inverse transformation device 200 can suppress an increase in the load of the inverse quantization inverse transformation processing while suppressing an increase in distortion of the output coefficient data res_x.
  • the reverse adaptive color conversion unit 203 has ⁇ 1.
  • Clip processing of reverse adaptive color conversion of reversible method> Various methods of the present technology described above (“Method 1", “Method 1-1”, “Method 1-2”, “Method 2", “Method 2-1” , “Method 2-2", and “Method 3") may be applied. That is, the inverse quantization inverse converter 200 has ⁇ 1.
  • Various methods of the present technology described above can be applied in the clip processing of the reverse adaptive color conversion of the reversible method. By applying any of these methods, the inverse quantization inverse converter 200 can obtain the same effect as described in ⁇ Application of the present technology to the inverse adaptive color converter>.
  • the quantization coefficient qcoef_x is inversely quantized to derive the orthogonal conversion coefficient coef_x.
  • step S202 the inverse orthogonal transform unit 202 refers to the conversion information Tinfo corresponding to each component identifier, inversely transforms the orthogonal transform coefficient coef_x, and derives the coefficient data res_x'.
  • step S203 the inverse adaptive color conversion unit 203 executes the inverse adaptive color conversion process, converts the coefficient data res_x'inverse YCgCo-R, and derives the coefficient data res_x after the inverse adaptive color conversion process.
  • step S203 When the process of step S203 is completed, the inverse quantization inverse transformation process is completed.
  • the inverse adaptive color conversion unit 203 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color conversion does not increase. Therefore, the inverse adaptive color conversion unit 203 can suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.
  • the inverse quantization inverse transform device 200 increases the load of the inverse quantization inverse transformation process while suppressing the increase in the distortion of the output coefficient data res_x. Can be suppressed.
  • FIG. 6 is a block diagram showing an example of the configuration of an image decoding device, which is an aspect of an image processing device to which the present technology is applied.
  • the image decoding device 400 shown in FIG. 6 is a device that decodes the coded data of the moving image.
  • the image decoding apparatus 400 decodes the encoded data of the moving image encoded by the encoding method such as VVC, AVC, HEVC, etc. described in the above-mentioned non-patent document.
  • the image decoding device 400 can decode the coded data (bit stream) generated by the image coding device 500 (FIG. 8) described later.
  • FIG. 6 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the image decoding apparatus 400, there may be a processing unit that is not shown as a block in FIG. 6, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. This also applies to other figures illustrating the processing unit and the like in the image decoding apparatus 400.
  • the image decoding device 400 includes a control unit 401, a storage buffer 411, a decoding unit 412, an inverse quantization inverse conversion unit 413, an arithmetic unit 414, an in-loop filter unit 415, a sorting buffer 416, and a frame memory. It includes 417 and a prediction unit 418.
  • the prediction unit 418 includes an intra prediction unit (not shown) and an inter prediction unit.
  • the control unit 401 executes a process related to decoding control. For example, the control unit 401 acquires the coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, residual information Rinfo, filter information Finfo, etc.) included in the bit stream via the decoding unit 412. Further, the control unit 401 can estimate the coding parameters not included in the bit stream. Further, the control unit 401 controls decoding by controlling each processing unit (accumulation buffer 411 to prediction unit 418) of the image decoding device 400 based on the acquired (or estimated) coding parameter.
  • the control unit 401 controls decoding by controlling each processing unit (accumulation buffer 411 to prediction unit 418) of the image decoding device 400 based on the acquired (or estimated) coding parameter.
  • control unit 401 supplies the header information Hinfo to the inverse quantization inverse conversion unit 413, the prediction unit 418, and the in-loop filter unit 415. Further, the control unit 401 supplies the prediction mode information Pinfo to the inverse quantization inverse conversion unit 413 and the prediction unit 418. Further, the control unit 401 supplies the conversion information Tinfo to the inverse quantization inverse conversion unit 413. Further, the control unit 401 supplies the residual information Rinfo to the decoding unit 412. Further, the control unit 401 supplies the filter information Finfo to the in-loop filter unit 415.
  • each coding parameter may be supplied to an arbitrary processing unit.
  • other information may be supplied to an arbitrary processing unit.
  • Header information Hinfo may include information such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (picture header), SH (slice header) and the like.
  • the header information Hinfo may include information that defines the image size (for example, parameters such as PicWidth indicating the width of the image and PicHeight indicating the height of the image). Further, the header information Hinfo may include information defining the bit depth (for example, parameters such as bitDepthY indicating the bit depth of the luminance component and bitDepthC indicating the bit depth of the color difference component). Further, the header information Hinfo may include ChromaArrayType, which is a parameter indicating the color difference array type. Further, the header information Hinfo may include parameters such as MaxCUSize indicating the maximum value of the CU size and MinCUSize indicating the minimum value of the CU size.
  • the header information Hinfo may include parameters such as MaxQTDepth indicating the maximum depth of the quadtree division (also referred to as Quad-tree division) and MinQTDepth indicating the minimum depth. Further, the header information Hinfo may include parameters such as MaxBTDepth indicating the maximum depth of the binary-tree division and MinBTDepth indicating the minimum depth. Further, the header information Hinfo may include parameters such as MaxTSSize indicating the maximum value of the conversion skip block (also referred to as the maximum conversion skip block size).
  • the header information Hinfo may include information that defines an on / off flag (also referred to as a valid flag) of each coding tool.
  • the header information Hinfo may include on / off flags for orthogonal transformation processing and quantization processing.
  • the on / off flag of the coding tool can also be interpreted as a flag indicating whether or not the syntax related to the coding tool exists in the coded data. Further, when the value of the on / off flag is 1 (true), it indicates that the coding tool can be used, and when the value of the on / off flag is 0 (false), it indicates that the coding tool cannot be used. It may be. The interpretation of the flag value (true or false) may be reversed.
  • the prediction mode information Pinfo may include, for example, parameters such as PBSize indicating the size (prediction block size) of the PB (prediction block) to be processed, and information such as intra prediction mode information IPinfo and motion prediction information MVinfo.
  • Intra prediction mode information IPinfo may include, for example, information such as prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and IntraPredModeY indicating the brightness intra prediction mode derived from the syntax.
  • the intra prediction mode information IPinfo may include information such as ccp_flag (cclmp_flag), mclm_flag, chroma_sample_loc_type_idx, chroma_mpm_idx, and IntraPredModeC indicating the brightness intra prediction mode derived from these syntaxes.
  • Chroma_sample_loc_type_idx is a color difference sample position type identifier, and is an identifier that identifies the type of pixel position of the color difference component (also referred to as the color difference sample position type).
  • the color difference sample position type identifier (chroma_sample_loc_type_idx) is transmitted as chroma_sample_loc_info () (that is, stored in chroma_sample_loc_info ()). This chroma_sample_loc_info () is information about the pixel position of the color difference component.
  • Chroma_mpm_idx is a color difference MPM identifier, and is an identifier indicating which prediction mode candidate in the color difference intra prediction mode candidate list (intraPredModeCandListC) is designated as the color difference intra prediction mode.
  • the information included in the prediction mode information Pinfo is arbitrary, and information other than this information may be included in the prediction mode information Pinfo.
  • the conversion information Tinfo may include, for example, information such as TBWSize, TBHSize, ts_flag, scanIdx, quantization parameter qP, quantization matrix scaling_matrix (see, for example, JCTVC-W1005, 7.3.4 Scaling list data syntax).
  • TBWSize is a parameter that indicates the width size of the conversion target conversion block to be processed.
  • the conversion information Tinfo may include the logarithmic log2TBWSize of TBWSize with a base of 2 instead of this TBWSize.
  • TBHSize is a parameter indicating the vertical width size of the conversion target conversion block to be processed.
  • the conversion information Tinfo may include the logarithmic log2TBHSize of TBHSize having a base of 2 instead of this TBHSize.
  • Ts_flag is a conversion skip flag, and is flag information indicating whether to skip the (reverse) primary conversion and the (reverse) secondary conversion.
  • scanIdx is a scan identifier.
  • the information included in the conversion information Tinfo is arbitrary, and information other than these information may be included in the conversion information Tinfo.
  • the residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) may include, for example, the following information.
  • cbf (coded_block_flag): Residual data presence / absence flag last_sig_coeff_x_pos: Last non-zero coefficient X coordinate last_sig_coeff_y_pos: Last non-zero coefficient Y coordinate coded_sub_block_flag: Subblock non-zero coefficient presence / absence flag sig_coeff_flag: Non-zero coefficient presence / absence flag gr1_flag: Non-zero coefficient level Flag indicating whether it is greater than 1 (also called GR1 flag)
  • gr2_flag Flag indicating whether the level of non-zero coefficient is greater than 2 (also called GR2 flag) sign_flag: A sign indicating the positive or negative of the non-zero coefficient (also called a sign code) coeff_abs_level_remaining: Non-zero coefficient residual level (also called non-zero coefficient residual level)
  • the information included in the residual information Rinfo is arbitrary, and information other than this information may be included.
  • the filter information Finfo may include, for example, control information regarding each of the following filtering processes.
  • Control information for deblocking filter (DBF) Control information for pixel adaptive offset (SAO)
  • Control information for adaptive loop filter (ALF) Control information for other linear and nonlinear filters
  • a picture to which each filter is applied information for specifying an area in the picture, filter on / off control information for each CU, filter on / off control information for slice and tile boundaries, and the like may be included.
  • the information included in the filter information Finfo is arbitrary, and information other than this information may be included.
  • the storage buffer 411 acquires and holds (stores) the bit stream input to the image decoding device 400.
  • the storage buffer 411 extracts the coded data included in the stored bit stream at a predetermined timing or when a predetermined condition is satisfied, and supplies the coded data to the decoding unit 412.
  • the decoding unit 412 acquires the coded data supplied from the storage buffer 411, entropy-decodes (reversibly decodes) the syntax value of each syntax element from the bit string according to the definition of the syntax table, and encodes the code. Derivation of syntax parameters.
  • This coding parameter may include information such as header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 412 decodes and parses (analyzes and acquires) this information from the bit stream.
  • the decoding unit 412 executes such processing (decoding, parsing, etc.) according to the control of the control unit 401, and supplies the obtained information to the control unit 401.
  • the decoding unit 412 decodes the encoded data with reference to the residual information Rinfo. At that time, the decoding unit 412 applies entropy decoding (reversible decoding) such as CABAC or CAVLC. That is, the decoding unit 412 decodes the coded data by a decoding method corresponding to the coding method of the coding process executed by the coding unit 514 of the image coding device 500.
  • entropy decoding reversible decoding
  • the decoding unit 412 performs arithmetic decoding using a context model on the coded data, and derives the quantization coefficient level of each coefficient position in each conversion block.
  • the decoding unit 412 supplies the derived quantization coefficient level to the inverse quantization inverse conversion unit 413.
  • the inverse quantization inverse transformation unit 413 acquires the quantization coefficient level supplied from the decoding unit 412. Inverse quantization Inverse conversion unit 413 acquires coding parameters such as prediction mode information Pinfo and conversion information Tinfo supplied from control unit 401.
  • the inverse quantization inverse transformation unit 413 executes the inverse quantization inverse transformation process for the quantization coefficient level based on the coding parameters such as the prediction mode information Pinfo and the transformation information Tinfo, and derives the residual data D'. do.
  • This inverse quantization inverse transformation process is an inverse process of the transformation quantization process executed in the transformation quantization unit 513 (FIG. 8) described later. That is, in the inverse quantization inverse transformation process, for example, processes such as inverse quantization, inverse orthogonal transformation, and inverse adaptive color transformation are executed.
  • the inverse quantization is an inverse process of quantization executed in the transformation quantization unit 513.
  • the inverse orthogonal transform is an inverse process of the orthogonal transform executed in the transform quantization unit 513.
  • the inverse adaptive color conversion is an inverse process of the adaptive color conversion executed in the conversion quantization unit 513.
  • the processing included in the inverse quantization inverse transformation processing is arbitrary, and some of the above-mentioned processing may be omitted, or processing other than the above-mentioned processing may be included.
  • the inverse quantization inverse transformation unit 413 supplies the derived residual data D'to the arithmetic unit 414.
  • the calculation unit 414 acquires the residual data D'supplied from the inverse quantization inverse conversion unit 413 and the prediction image supplied from the prediction unit 418.
  • the calculation unit 414 adds the residual data D'and the predicted image P corresponding to the residual data D'to derive a locally decoded image.
  • the calculation unit 414 supplies the derived locally decoded image to the in-loop filter unit 415 and the frame memory 417.
  • the in-loop filter unit 415 acquires a locally decoded image supplied from the calculation unit 414.
  • the in-loop filter unit 415 acquires the filter information Finfo supplied from the control unit 401.
  • the information input to the in-loop filter unit 415 is arbitrary, and information other than these information may be input.
  • the in-loop filter unit 415 appropriately performs a filter process on the locally decoded image based on the filter information Finfo.
  • the in-loop filter unit 415 may apply a bilateral filter as its filter processing.
  • the in-loop filter unit 415 may apply a deblocking filter (DBF (DeBlocking Filter)) as its filter processing.
  • the in-loop filter unit 415 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as its filter processing.
  • SAO Sample Adaptive Offset
  • the in-loop filter unit 415 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as its filter processing.
  • ALF adaptive Loop Filter
  • the in-loop filter unit 415 can apply a plurality of filters among these in combination as a filter process.
  • the in-loop filter unit 415 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as filter processing.
  • the in-loop filter unit 415 executes the filter processing corresponding to the filter processing executed by the coding side device.
  • the in-loop filter unit 415 executes a filter process corresponding to the filter process executed by the in-loop filter unit 518 (FIG. 8) of the image coding apparatus 500 described later.
  • the filter processing executed by the in-loop filter unit 415 is arbitrary and is not limited to the above example.
  • the in-loop filter unit 415 may apply a Wiener filter or the like.
  • the in-loop filter unit 415 supplies the filtered locally decoded image to the sorting buffer 416 and the frame memory 417.
  • the sorting buffer 416 receives the locally decoded image supplied from the in-loop filter unit 415 as an input, and holds (stores) it.
  • the sorting buffer 416 reconstructs and holds (stores in the buffer) the decoded image for each picture unit using the locally decoded image.
  • the sorting buffer 416 sorts the obtained decoded images from the decoding order to the reproduction order.
  • the rearrangement buffer 416 outputs the decoded image group sorted in the order of reproduction as moving image data to the outside of the image decoding apparatus 400.
  • the frame memory 417 acquires a locally decoded image supplied from the arithmetic unit 414, reconstructs the decoded image for each picture unit, and stores the decoded image in the buffer in the frame memory 417. Further, the frame memory 417 acquires the in-loop filtered locally decoded image supplied from the in-loop filter unit 415, reconstructs the decoded image for each picture unit, and stores it in the buffer in the frame memory 417. do.
  • the frame memory 417 appropriately supplies the stored decoded image (or a part thereof) to the prediction unit 418 as a reference image.
  • the frame memory 417 may store header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc. related to the generation of the decoded image.
  • the prediction unit 418 acquires the prediction mode information Pinfo supplied from the control unit 401. Further, the prediction unit 418 acquires a decoded image (or a part thereof) read from the frame memory 417. The prediction unit 418 executes the prediction process in the prediction mode adopted at the time of coding based on the prediction mode information Pinfo, and generates the prediction image P by referring to the decoded image as a reference image. The prediction unit 418 supplies the generated prediction image P to the calculation unit 414.
  • the image decoding apparatus 400 can apply the inverse quantization inverse transform device 200 described in the second embodiment as the inverse quantization inverse transform unit 413.
  • the inverse quantization inverse conversion unit 413 has the same configuration as the inverse quantization inverse conversion device 200 (FIG. 4), and performs the same processing.
  • the inverse quantization unit 201 acquires the quantization coefficient level supplied from the decoding unit 412 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 dequantizes the quantization coefficient qcoef_x using information such as the quantization parameters qP, cu_act_enabled_flag, and transform_skip_flag, and derives the orthogonal transformation coefficient coef_x.
  • the inverse orthogonal transform unit 202 uses information such as conversion information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx to perform inverse orthogonal transform of the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201, and derives the coefficient data res_x'.
  • the inverse adaptive color conversion unit 203 Based on the cu_act_enabled_flag, the inverse adaptive color conversion unit 203 performs inverse adaptive color conversion (inverse YCgCo-R conversion) of the coefficient data res_x'derived by the inverse orthogonal transform unit 202 by a reversible method as appropriate, and reverse adaptive color conversion. Derivation of the coefficient data res_x after processing. The inverse adaptive color conversion unit 203 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.
  • inverse adaptive color conversion inverse YCgCo-R conversion
  • the reverse adaptive color conversion unit 203 has the same configuration as the reverse adaptive color conversion device 100, and performs the same processing.
  • the clip processing unit 102 clips the coefficient data res_x'supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clipped at that level by the clip processing unit 102, and derives the coefficient data res_x after the inverse adaptive color conversion processing.
  • the inverse YCgCo-R conversion unit 103 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.
  • the inverse quantization inverse transformation unit 413 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color transformation does not increase. Therefore, the inverse quantization inverse transformation unit 413 performs the inverse quantization inverse transformation process while suppressing an increase in distortion of the output coefficient data res_x (residual data D'), as in the case of the inverse quantization inverse transform device 200. It is possible to suppress an increase in the load of. That is, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.
  • the inverse quantization inverse transform unit 413 has ⁇ 1.
  • Clip processing of reverse adaptive color conversion of reversible method> Various methods of the present technology described above (“Method 1”, “Method 1-1”, “Method 1-2”, “Method 2", “Method 2-1” , “Method 2-2", and “Method 3") may be applied. That is, the image decoding device 400 has ⁇ 1.
  • Various methods of the present technology described above can be applied in the clip processing of the reverse adaptive color conversion of the reversible method.
  • the image decoding apparatus 400 has the same effect as described in ⁇ Application of the present technology to the inverse quantization inverse converter> (that is, to the inverse adaptive color converter). The same effect as described in the application of this technology> can be obtained.
  • the storage buffer 411 acquires (stores) a bit stream (encoded data) supplied from the outside of the image decoding device 400 in step S401.
  • the decoding unit 412 executes the decoding process.
  • the decoding unit 412 parses (analyzes and acquires) various coding parameters (for example, header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, etc.) from the bit stream.
  • the control unit 401 sets the various coding parameters by supplying the acquired various coding parameters to the various processing units.
  • control unit 401 sets the processing unit based on the obtained coding parameter. Further, the decoding unit 412 decodes the bit stream and derives the quantization coefficient level according to the control of the control unit 401.
  • step S403 the inverse quantization inverse conversion unit 413 executes the inverse quantization inverse transformation process and derives the residual data D'.
  • step S404 the prediction unit 418 generates a prediction image P.
  • the prediction unit 418 executes the prediction process by the prediction method specified by the coding side based on the coding parameters and the like set in step S402, and refers to the reference image stored in the frame memory 417. And so on, the predicted image P is generated.
  • step S405 the calculation unit 414 adds the residual data D'obtained in step S403 and the predicted image P obtained in step S404 to derive a locally decoded image.
  • step S406 the in-loop filter unit 415 executes the in-loop filter process on the locally decoded image obtained by the process of step S405.
  • step S407 the sorting buffer 416 derives the decoded image using the locally decoded image filtered by the process of step S406, and rearranges the order of the decoded image group from the decoding order to the reproduction order.
  • the decoded image group sorted in the order of reproduction is output as a moving image to the outside of the image decoding device 400.
  • step S408 the frame memory 417 stores at least one of the locally decoded image obtained by the process of step S405 and the locally decoded image filtered by the process of step S406.
  • step S408 When the process of step S408 is completed, the image decoding process is completed.
  • the inverse quantization inverse transformation unit 413 clips the coefficient data res_x'at a level so that the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation does not increase. It can be performed. Therefore, the inverse quantization inverse transformation unit 413 can suppress the increase in the load of the inverse quantization inverse transformation processing while suppressing the increase in the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation. can.
  • the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.
  • FIG. 8 is a block diagram showing an example of the configuration of an image coding device, which is an aspect of an image processing device to which the present technology is applied.
  • the image coding device 500 shown in FIG. 8 is a device that encodes image data of a moving image.
  • the image coding device 500 uses a coding method such as VVC (Versatile Video Coding), AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding) described in the above-mentioned non-patent document to obtain image data of a moving image.
  • VVC Very Video Coding
  • AVC Advanced Video Coding
  • HEVC High Efficiency Video Coding
  • the image coding device 500 can generate coded data (bitstream) that can be decoded by the image decoding device 400 (FIG. 6) described above.
  • FIG. 8 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the image coding apparatus 500, there may be a processing unit that is not shown as a block in FIG. 8, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. This also applies to other figures for explaining the processing unit and the like in the image coding apparatus 500.
  • the image coding device 500 includes a control unit 501, a sorting buffer 511, a calculation unit 512, a conversion quantization unit 513, a coding unit 514, and a storage buffer 515. Further, the image coding device 500 includes an inverse quantization inverse conversion unit 516, an arithmetic unit 517, an in-loop filter unit 518, a frame memory 519, a prediction unit 520, and a rate control unit 521.
  • the control unit 501 divides the moving image data held by the sorting buffer 511 into blocks (CU, PU, TU, etc.) of the processing unit based on the block size of the external or predetermined processing unit. Further, the control unit 501 determines the coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization). do. For example, the control unit 501 can set a conversion skip flag or the like.
  • the control unit 501 determines the coding parameters as described above, the control unit 501 supplies them to each block.
  • the header information Hinfo is supplied to each block.
  • the prediction mode information Pinfo is supplied to the coding unit 514 and the prediction unit 520.
  • the conversion information Tinfo is supplied to the coding unit 514, the conversion quantization unit 513, and the inverse quantization inverse conversion unit 516.
  • the filter information Finfo is supplied to the coding unit 514 and the in-loop filter unit 518.
  • the supply destination of each coding parameter is arbitrary and is not limited to this example.
  • ⁇ Sort buffer> Each field (input image) of the moving image data is input to the image coding device 500 in the reproduction order (display order).
  • the sorting buffer 511 acquires and retains (stores) each input image in its reproduction order (display order).
  • the sorting buffer 511 sorts the input images in the coding order (decoding order) or divides the input images into blocks of processing units based on the control of the control unit 501.
  • the sorting buffer 511 supplies each input image after processing to the calculation unit 512.
  • the calculation unit 512 subtracts the prediction image P supplied from the prediction unit 520 from the image corresponding to the block of the processing unit supplied from the sorting buffer 511, derives the residual data D, and converts it into a conversion quantum. It is supplied to the chemical unit 513.
  • the conversion quantization unit 513 acquires the residual data D supplied from the calculation unit 512. Further, the conversion quantization unit 513 acquires the prediction mode information Pinfo and the conversion information Tinfo supplied from the control unit 501. The conversion quantization unit 513 executes the conversion quantization process on the residual data D based on the prediction mode information Pinfo and the conversion information Tinfo, and derives the quantization coefficient level. In the conversion quantization process, for example, processing such as adaptive color conversion, orthogonal transformation, and quantization is executed. Of course, the processing included in the conversion quantization processing is arbitrary, and some of the above-mentioned processing may be omitted, or processing other than the above-mentioned processing may be included. The conversion quantization unit 513 supplies the derived quantization coefficient level to the coding unit 514 and the inverse quantization inverse conversion unit 516.
  • the coding unit 514 acquires the quantization coefficient level supplied from the conversion quantization unit 513. Further, the coding unit 514 acquires various coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) supplied from the control unit 501. Further, the coding unit 514 acquires information about the filter such as the filter coefficient supplied from the in-loop filter unit 518. In addition, the coding unit 514 acquires information on the optimum prediction mode supplied from the prediction unit 520.
  • header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc. supplied from the control unit 501. Further, the coding unit 514 acquires information about the filter such as the filter coefficient supplied from the in-loop filter unit 518. In addition, the coding unit 514 acquires information on the optimum prediction mode supplied from the prediction unit 520.
  • the coding unit 514 entropy-codes (losslessly codes) the quantization coefficient level to generate a bit string (encoded data).
  • the coding unit 514 may apply, for example, CABAC (Context-based Adaptive Binary Arithmetic Code) as this entropy coding.
  • the coding unit 514 may apply, for example, CAVLC (Context-based Adaptive Variable Length Code) as this entropy coding.
  • CABAC Context-based Adaptive Binary Arithmetic Code
  • CAVLC Context-based Adaptive Variable Length Code
  • the content of this entropy coding is arbitrary and is not limited to these examples.
  • the coding unit 514 derives the residual information Rinfo from the quantization coefficient level, encodes the residual information Rinfo, and generates a bit string.
  • the coding unit 514 includes the information about the filter supplied from the in-loop filter unit 518 in the filter information Finfo, and includes the information about the optimum prediction mode supplied from the prediction unit 520 in the prediction mode information Pinfo. Then, the coding unit 514 encodes the various coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.) described above to generate a bit string.
  • the coding unit 514 multiplexes the bit strings of the various information generated as described above to generate the coded data.
  • the coding unit 514 supplies the coded data to the storage buffer 515.
  • the storage buffer 515 temporarily holds the coded data obtained in the coding unit 514.
  • the storage buffer 515 outputs the held coded data as, for example, a bit stream or the like to the outside of the image coding device 500 at a predetermined timing.
  • this coded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, or the like. That is, the storage buffer 515 is also a transmission unit that transmits coded data (bit stream).
  • Inverse quantization and inverse transformation> Inverse quantization Inverse conversion unit 516 acquires the quantization coefficient level supplied from the conversion quantization unit 513. Further, the inverse quantization inverse conversion unit 516 acquires the conversion information Tinfo supplied from the control unit 501.
  • the inverse quantization inverse transformation unit 516 executes the inverse quantization inverse transformation process for the quantization coefficient level based on the transformation information Tinfo, and derives the residual data D'.
  • This inverse quantization inverse transformation process is an inverse process of the transformation quantization process executed by the transformation quantization unit 513, and is an inverse quantization inverse process executed by the inverse quantization inverse transform unit 413 of the image decoding apparatus 400 described above. This is the same process as the conversion process.
  • inverse quantization is an inverse process of quantization executed in the transformation quantization unit 513, and is a process similar to the inverse quantization executed in the inverse quantization inverse quantization unit 413.
  • inverse orthogonal transformation is an inverse process of the orthogonal transformation executed by the conversion quantization unit 513, and is the same process as the inverse orthogonal transformation executed by the conversion quantization unit 513.
  • the inverse adaptive color conversion is an inverse process of the adaptive color conversion executed by the conversion quantization unit 513, and is the same process as the inverse adaptive color conversion executed by the conversion quantization unit 513.
  • the processing included in this inverse quantization inverse transformation processing is arbitrary, and some of the above-mentioned processing may be omitted, or processing other than the above-mentioned processing may be included.
  • the inverse quantization inverse transformation unit 516 supplies the derived residual data D'to the arithmetic unit 517.
  • the calculation unit 517 acquires the residual data D'supplied from the inverse quantization inverse conversion unit 516 and the prediction image P supplied from the prediction unit 520.
  • the calculation unit 517 adds the residual data D'and the predicted image P corresponding to the residual data D'to derive a locally decoded image.
  • the calculation unit 517 supplies the derived locally decoded image to the in-loop filter unit 518 and the frame memory 519.
  • the in-loop filter unit 518 acquires a locally decoded image supplied from the calculation unit 517. Further, the in-loop filter unit 518 acquires the filter information Finfo supplied from the control unit 501. Further, the in-loop filter unit 518 acquires an input image (original image) supplied from the sorting buffer 511.
  • the information input to the in-loop filter unit 518 is arbitrary, and information other than these information may be input. For example, if necessary, information such as prediction mode, motion information, code amount target value, quantization parameter qP, picture type, block (CU, CTU, etc.) may be input to the in-loop filter unit 518. good.
  • the in-loop filter unit 518 appropriately performs a filter process on the locally decoded image based on the filter information Finfo.
  • the in-loop filter unit 518 also uses an input image (original image) and other input information for the filter processing, if necessary.
  • the in-loop filter unit 518 may apply a bilateral filter as its filter processing.
  • the in-loop filter unit 518 may apply a deblocking filter (DBF (DeBlocking Filter)) as its filter processing.
  • the in-loop filter unit 518 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as its filter processing.
  • the in-loop filter unit 518 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as its filter processing.
  • ALF adaptive Loop Filter
  • the in-loop filter unit 518 can apply a plurality of filters among them in combination as a filter process. It should be noted that which filter is applied and which order is applied is arbitrary and can be appropriately selected.
  • the in-loop filter unit 518 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as filter processing.
  • the filter processing executed by the in-loop filter unit 518 is arbitrary and is not limited to the above example.
  • the in-loop filter unit 518 may apply a Wiener filter or the like.
  • the in-loop filter unit 518 supplies the filtered locally decoded image to the frame memory 519.
  • the in-loop filter unit 518 supplies information about the filter to the coding unit 514.
  • the frame memory 519 executes a process related to storage of data related to an image. For example, the frame memory 519 acquires a locally decoded image supplied from the arithmetic unit 517 and a filtered locally decoded image supplied from the in-loop filter unit 518, and holds (stores) it. Further, the frame memory 519 reconstructs and holds the decoded image for each picture unit using the locally decoded image (stored in the buffer in the frame memory 519). The frame memory 519 supplies the decoded image (or a part thereof) to the prediction unit 520 in response to the request of the prediction unit 520.
  • the prediction unit 520 executes a process related to the generation of the prediction image. For example, the prediction unit 520 acquires the prediction mode information Pinfo supplied from the control unit 501. For example, the prediction unit 520 acquires an input image (original image) supplied from the sorting buffer 511. For example, the prediction unit 520 acquires a decoded image (or a part thereof) read from the frame memory 519.
  • the prediction unit 520 executes prediction processing such as inter-prediction and intra-prediction using the prediction mode information Pinfo and the input image (original image). That is, the prediction unit 520 refers to the decoded image as a reference image, executes prediction and motion compensation, and generates a prediction image P. The prediction unit 520 supplies the generated prediction image P to the calculation unit 512 and the calculation unit 517. Further, the prediction unit 520 supplies information regarding the prediction mode selected by the above processing, that is, the optimum prediction mode, to the coding unit 514 as needed.
  • the rate control unit 521 executes a process related to rate control. For example, the rate control unit 521 controls the rate of the quantization operation of the conversion quantization unit 513 based on the code amount of the coded data stored in the storage buffer 515 so that overflow or underflow does not occur.
  • FIG. 9 is a block diagram showing a main configuration example of the conversion quantization unit 513 of FIG.
  • the conversion quantization unit 513 includes an adaptive color conversion unit 541, an orthogonal transformation unit 542, and a quantization unit 543.
  • the adaptive color conversion unit 541 acquires the residual data D (FIG. 8) supplied from the calculation unit 512 as the coefficient data res_x. Further, the adaptive color conversion unit 541 acquires the cu_act_enabled_flag supplied from the control unit 501. The adaptive color conversion unit 541 executes adaptive color conversion for the coefficient data res_x based on the value of cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (for example, "1"), the adaptive color conversion unit 541 performs a reversible adaptive color conversion (YCgCo-R conversion) to convert the RGB domain coefficient data res_x and the YCgCo domain coefficient data res_x. Convert to'.
  • YCgCo-R conversion reversible adaptive color conversion
  • This adaptive color transformation is an inverse process of the inverse adaptive color transformation executed by the inverse quantization inverse transform unit 413 and the inverse quantization inverse transform unit 516.
  • the adaptive color conversion unit 541 supplies the derived coefficient data res_x'to the orthogonal transformation unit 542.
  • the orthogonal transform unit 542 acquires the coefficient data res_x'supplied from the adaptive color converter 541.
  • the orthogonal transform unit 542 acquires information such as conversion information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501.
  • the orthogonal transformation unit 542 orthogonally transforms the coefficient data res_x'using the acquired information, and derives the orthogonal transformation coefficient coef_x.
  • This orthogonal transformation is an inverse process of the inverse orthogonal transformation executed by the inverse quantization inverse transform unit 413 and the inverse quantization inverse transform unit 516.
  • the orthogonal transform unit 542 supplies the derived orthogonal transform coefficient coef_x to the quantization unit 543.
  • the quantization unit 543 acquires the orthogonal transformation coefficient coeff_x supplied from the orthogonal transform unit 542. Further, the quantization unit 543 acquires information such as the quantization parameters qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501. The quantization unit 543 quantizes the orthogonal transformation coefficient coef_x using the acquired information, and derives the quantization coefficient qcoef_x. This quantization is an inverse process of inverse quantization executed in the inverse quantization inverse transform unit 413 and the inverse quantization inverse transform unit 516. The quantization unit 543 supplies the derived quantization coefficient qcoef_x to the coding unit 514 and the inverse quantization inverse conversion unit 516 (FIG. 8) as the quantization coefficient level.
  • FIG. 10 is a block diagram showing a main configuration example of the adaptive color conversion unit 541 of FIG. As shown in FIG. 10, the adaptive color conversion unit 541 has a selection unit 571 and a YCgCo-R conversion unit 572.
  • the selection unit 571 acquires the residual data D (FIG. 8) supplied from the calculation unit 512 as the coefficient data res_x. Further, the selection unit 571 acquires the cu_act_enabled_flag supplied from the control unit 501. The selection unit 571 selects whether or not to execute the adaptive color conversion for the acquired coefficient data res_x based on the value of the cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (for example, "1"), the selection unit 571 determines that the adaptive color conversion can be applied, and supplies the coefficient data res_x to the YCgCo-R conversion unit 572.
  • the selection unit 571 determines that the application of adaptive color conversion (reverse adaptive color conversion) is prohibited (that is, it is not applicable), and the coefficient data res_x Is supplied to the orthogonal transform unit 542 (FIG. 9) as the coefficient data res_x'after the applied color conversion.
  • the YCgCo-R conversion unit 572 acquires the coefficient data res_x supplied from the selection unit 101.
  • the YCgCo-R conversion unit 572 converts the acquired coefficient data res_x into YCgCo-R, and derives the YCgCo-R-converted coefficient data res_x'.
  • This YCgCo-R transformation is an inverse process of the inverse YCgCo-R transformation executed by the inverse quantization inverse transform unit 413 and the inverse quantization inverse transform unit 516.
  • the YCgCo-R conversion unit 572 supplies the derived coefficient data res_x'to the orthogonal transformation unit 542 (FIG. 9).
  • the image coding apparatus 500 can apply the inverse quantization inverse transform device 200 described in the second embodiment as the inverse quantization inverse transform unit 516.
  • the inverse quantization inverse conversion unit 516 has the same configuration as the inverse quantization inverse conversion device 200 (FIG. 4), and performs the same processing.
  • the inverse quantization unit 201 acquires the quantization coefficient level supplied from the conversion quantization unit 513 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 dequantizes the quantization coefficient qcoef_x using information such as the quantization parameters qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501, and derives the orthogonal transformation coefficient coef_x.
  • the inverse orthogonal transform unit 202 reverse-orthogonally transforms the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201 using information such as conversion information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501, and the coefficient. Derive the data res_x'.
  • the inverse adaptive color conversion unit 203 converts the coefficient data res_x'derived by the inverse orthogonal transform unit 202 based on the cu_act_enabled_flag supplied from the control unit 501 by an inverse adaptive color conversion (inverse YCgCo-R conversion) as appropriate. ), And derive the coefficient data res_x after the inverse adaptive color conversion process.
  • the inverse adaptive color conversion unit 203 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.
  • the reverse adaptive color conversion unit 203 has the same configuration as the reverse adaptive color conversion device 100, and performs the same processing.
  • the clip processing unit 102 clips the coefficient data res_x'supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data.
  • the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion of the coefficient data res_x'clipped at that level by the clip processing unit 102, and derives the coefficient data res_x after the inverse adaptive color conversion processing.
  • the inverse YCgCo-R conversion unit 103 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.
  • the inverse quantization inverse transformation unit 516 can clip the coefficient data res_x'at a level at which the distortion of the coefficient data res_x after the inverse adaptive color transformation does not increase. Therefore, the inverse quantization inverse transformation unit 516 performs the inverse quantization inverse transformation process while suppressing an increase in distortion of the output coefficient data res_x (residual data D'), as in the case of the inverse quantization inverse transform device 200. It is possible to suppress an increase in the load of. That is, the image coding device 500 can suppress an increase in the load of the image coding process while suppressing a decrease in the quality of the decoded image.
  • the inverse quantization inverse transform unit 516 has ⁇ 1.
  • Clip processing of reverse adaptive color conversion of reversible method> Various methods of the present technology described above (“Method 1”, “Method 1-1”, “Method 1-2”, “Method 2", “Method 2-1” , “Method 2-2", and “Method 3") may be applied. That is, the image coding device 500 has ⁇ 1.
  • Various methods of the present technology described above can be applied in the clip processing of the reverse adaptive color conversion of the reversible method.
  • the image coding apparatus 500 has the same effect as described in ⁇ Application of the present technology to the inverse quantization inverse converter> (that is, ⁇ Inverse adaptive color transforming apparatus>.
  • the same effect as described in Applying the present technology to> can be obtained.
  • step S501 the sorting buffer 511 is controlled by the control unit 501 to sort the frame order of the input moving image data from the display order to the coding order.
  • step S502 the control unit 501 sets a processing unit (divides a block) for the input image held by the sorting buffer 511.
  • step S503 the control unit 501 sets coding parameters (for example, header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, etc.) for the input image held by the sorting buffer 511.
  • coding parameters for example, header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, etc.
  • the prediction unit 520 executes the prediction process and generates a prediction image or the like of the optimum prediction mode. For example, in this prediction process, the prediction unit 520 executes intra-prediction to generate a prediction image or the like of the optimum intra-prediction mode, and executes inter-prediction to generate a prediction image or the like of the optimum inter-prediction mode. From them, the optimum prediction mode is selected based on the cost function value and the like.
  • step S505 the calculation unit 512 calculates the difference between the input image and the prediction image of the optimum mode selected by the prediction processing in step S504. That is, the calculation unit 512 derives the residual data D between the input image and the predicted image. The amount of residual data D derived in this way is reduced as compared with the original image data. Therefore, the amount of data can be compressed as compared with the case where the image is encoded as it is.
  • step S506 the conversion quantization unit 513 executes the conversion quantization process on the residual data D derived by the process of step S505 by using the coding parameters such as the conversion information Tinfo set in step S503. Then, the quantization coefficient level is derived.
  • step S507 the inverse quantization inverse conversion unit 516 uses the coding parameters such as the conversion information Tinfo set in step S503 to perform the inverse quantization inverse transformation with respect to the quantization coefficient level derived in step S506. Execute the process and derive the residual data D'.
  • This inverse quantization inverse transformation process is the inverse process of the transformation quantization process of step S506, and is the same process as the inverse quantization inverse transformation process of step S403 of the image decoding process of FIG. 7.
  • step S508 the calculation unit 517 locally decodes the residual data D'derived by the inverse quantization inverse transformation process of step S507 by adding the predicted image generated by the prediction process of step S504. Generate the decoded image.
  • step S509 the in-loop filter unit 518 executes the in-loop filter process on the locally decoded decoded image derived by the process of step S508.
  • step S510 the frame memory 519 stores the locally decoded decoded image derived by the process of step S508 and the locally decoded decoded image filtered in step S509.
  • step S511 the coding unit 514 encodes the quantization coefficient level derived by the conversion quantization process of step S506, and derives the coded data. At this time, the coding unit 514 encodes various coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo). Further, the coding unit 514 derives the residual information RInfo from the quantization coefficient level and encodes the residual information RInfo.
  • step S512 the storage buffer 515 stores the coded data derived in this way, and outputs it, for example, as a bit stream to the outside of the image coding device 500.
  • This bit stream is transmitted to a decoding side device (for example, an image decoding device 400) via, for example, a transmission line or a recording medium.
  • the rate control unit 521 controls the rate as needed.
  • the adaptive color conversion unit 541 adapts the coefficient data res_x derived by the process of step S505 based on the cu_act_enabled_flag set in step S503 (FIG. 11) in step S541. Convert and derive the coefficient data res_x'.
  • step S542 the orthogonal transform unit 542 orthogonally transforms the coefficient data res_x'derived in step S541 using the conversion information Tinfo, the prediction mode information Pinfo, etc. set in step S503 (FIG. 11), and performs orthogonal transform. Derivation of the coefficient coef_x.
  • step S543 the quantization unit 543 quantizes the orthogonal transformation coefficient coef_x derived in step S542 and derives the quantization coefficient qcoef_x by using the conversion information Tinfo and the like set in step S503 (FIG. 11).
  • step S543 When the process of step S543 is completed, the process returns to FIG.
  • the selection unit 571 of the adaptive color conversion unit 541 determines in step S571 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S572.
  • step S572 the YCgCo-R conversion unit 572 converts the coefficient data res_x of the three components into YCgCo-R, and derives the coefficient data res_x'after the adaptive color conversion.
  • step S573 the YCgCo-R conversion unit 572 supplies the derived coefficient data res_x'to the orthogonal transformation unit 542.
  • step S573 the adaptive color conversion process is completed.
  • step S571 If it is determined in step S571 that the cu_act_enabled_flag is false, the process proceeds to step S574.
  • step S574 the selection unit 571 supplies the coefficient data res_x to the orthogonal transform unit 542 as the coefficient data res_x'after the adaptive color conversion.
  • the adaptive color conversion process is completed.
  • the inverse quantization inverse transformation unit 516 clips the coefficient data res_x'at a level so that the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation does not increase. It can be performed. Therefore, the inverse quantization inverse transformation unit 516 can suppress the increase in the load of the inverse quantization inverse transformation processing while suppressing the increase in the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation. can.
  • the image coding device 500 can suppress an increase in the load of the image coding process while suppressing a decrease in the quality of the decoded image.
  • FIG. 14 is a block diagram showing a configuration example of computer hardware that executes the above-mentioned series of processes programmatically.
  • the CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • the input / output interface 810 is also connected to the bus 804.
  • An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.
  • the input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like.
  • the output unit 812 includes, for example, a display, a speaker, an output terminal, and the like.
  • the storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, or the like.
  • the communication unit 814 is composed of, for example, a network interface.
  • the drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the CPU 801 loads the program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804, and executes the above-described series. Processing is executed.
  • the RAM 803 also appropriately stores data and the like necessary for the CPU 801 to execute various processes.
  • the program executed by the computer can be recorded and applied to the removable media 821 as a package media or the like, for example.
  • the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable media 821 to the drive 815.
  • This program can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.
  • the program can be received by the communication unit 814 and installed in the storage unit 813.
  • this program can be installed in advance in ROM 802 or storage unit 813.
  • This technique can be applied to any image coding method or decoding method. That is, as long as it does not contradict the above-mentioned technology, the specifications of various processes related to image coding / decoding such as conversion (inverse transformation), quantization (inverse quantization), coding (decoding), and prediction are arbitrary. It is not limited to the example. In addition, some of these processes may be omitted as long as they do not contradict the present technology described above.
  • this technology can be applied to a multi-viewpoint image coding system that encodes a multi-viewpoint image including images of a plurality of viewpoints (views). Further, the present technology can be applied to a multi-viewpoint image decoding system that decodes coded data of a multi-viewpoint image including images of a plurality of viewpoints (views). In that case, the present technology may be applied to the coding and decoding of each viewpoint (view).
  • this technique can be applied to a hierarchical image coding (scalable coding) system that encodes a hierarchical image that is layered (layered) so as to have a scalability function for a predetermined parameter.
  • this technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes the encoded data of a hierarchical image that has been layered (layered) so as to have a scalability function for a predetermined parameter. can.
  • the present technology may be applied to the coding and decoding of each layer.
  • the inverse adaptive color converter 100, the inverse quantization inverse converter 200, the image decoding apparatus 400, and the image coding apparatus 500 have been described, but the present technology is arbitrary. Can be applied to the configuration.
  • this technology is a transmitter or receiver (for example, a television receiver or mobile phone) for satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication, or It can be applied to various electronic devices such as devices (for example, hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and reproduce images from these storage media.
  • devices for example, hard disk recorders and cameras
  • a processor as a system LSI (Large Scale Integration) or the like (for example, a video processor), a module using a plurality of processors (for example, a video module), a unit using a plurality of modules (for example, a video unit)
  • a processor as a system LSI (Large Scale Integration) or the like
  • a module using a plurality of processors for example, a video module
  • a unit using a plurality of modules for example, a video unit
  • it can be implemented as a configuration of a part of the device, such as a set (for example, a video set) in which other functions are added to the unit.
  • this technology can be applied to a network system composed of a plurality of devices.
  • the present technology may be implemented as cloud computing that is shared and jointly processed by a plurality of devices via a network.
  • this technology is implemented in a cloud service that provides services related to images (moving images) to arbitrary terminals such as computers, AV (AudioVisual) devices, portable information processing terminals, and IoT (Internet of Things) devices. You may try to do it.
  • the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..
  • Systems, devices, processing units, etc. to which this technology is applied can be used in any field such as transportation, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, home appliances, weather, nature monitoring, etc. .. Moreover, the use is arbitrary.
  • this technology can be applied to systems and devices used for providing ornamental contents and the like.
  • the present technology can be applied to systems and devices used for traffic such as traffic condition supervision and automatic driving control.
  • the present technology can be applied to systems and devices used for security purposes.
  • the present technology can be applied to a system or device used for automatic control of a machine or the like.
  • the present technology can be applied to systems and devices used for agriculture and livestock industry.
  • the present technology can also be applied to systems and devices for monitoring natural conditions such as volcanoes, forests and oceans, and wildlife. Further, for example, the present technology can be applied to systems and devices used for sports.
  • the "flag” is information for identifying a plurality of states, and is not only information used for identifying two states of true (1) or false (0), but also three or more states. It also contains information that can identify the state. Therefore, the value that this "flag” can take may be, for example, 2 values of 1/0 or 3 or more values. That is, the number of bits constituting this "flag” is arbitrary, and may be 1 bit or a plurality of bits.
  • the identification information (including the flag) is assumed to include not only the identification information in the bitstream but also the difference information of the identification information with respect to a certain reference information in the bitstream. In, the "flag” and “identification information” include not only the information but also the difference information with respect to the reference information.
  • various information (metadata, etc.) related to the coded data may be transmitted or recorded in any form as long as it is associated with the coded data.
  • the term "associate" means, for example, to make the other data available (linkable) when processing one data. That is, the data associated with each other may be combined as one data or may be individual data.
  • the information associated with the coded data (image) may be transmitted on a transmission path different from the coded data (image).
  • the information associated with the coded data (image) may be recorded on a recording medium (or another recording area of the same recording medium) different from the coded data (image). good.
  • this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part within the frame.
  • the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.
  • the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units).
  • the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit).
  • a configuration other than the above may be added to the configuration of each device (or each processing unit).
  • a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). ..
  • the above-mentioned program may be executed in any device.
  • the device may have necessary functions (functional blocks, etc.) so that necessary information can be obtained.
  • each step of one flowchart may be executed by one device, or may be shared and executed by a plurality of devices.
  • the plurality of processes may be executed by one device, or may be shared and executed by a plurality of devices.
  • a plurality of processes included in one step can be executed as processes of a plurality of steps.
  • the processes described as a plurality of steps can be collectively executed as one step.
  • the program executed by the computer may have the following characteristics.
  • the processing of the steps of writing a program may be performed in chronological order in the order described herein.
  • the processes of the steps for writing the program may be executed in parallel.
  • the processing of the step for writing the program may be executed individually at a required timing such as when it is called. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-mentioned order.
  • the processing of the step for writing this program may be executed in parallel with the processing of other programs.
  • the processing of the step of writing this program may be executed in combination with the processing of another program.
  • a plurality of technologies related to this technology can be independently implemented independently as long as there is no contradiction.
  • any plurality of the present technologies can be used in combination.
  • some or all of the techniques described in any of the embodiments may be combined with some or all of the techniques described in other embodiments. It is also possible to carry out a part or all of any of the above-mentioned techniques in combination with other techniques not described above.
  • the present technology can also have the following configurations.
  • a clip processing unit that clips coefficient data that has undergone adaptive color conversion by a lossless method at a level based on the bit depth of the coefficient data.
  • An image processing apparatus including a reverse adaptive color conversion unit that performs reverse adaptive color conversion of the coefficient data clipped at the level by the clip processing unit by the reversible method.
  • the clip processing unit is The brightness component and the color component of the coefficient data are clipped with the value obtained by subtracting 1 from the power of 2 having the value obtained by adding 1 to the bit depth as the power index as the upper limit value.
  • the image according to (2) which clips the luminance component and the color component of the coefficient data with the value obtained by multiplying the power of 2 having the value obtained by adding 1 to the bit depth as the exponent by -1 as the lower limit value.
  • Processing equipment (4)
  • the image processing apparatus according to (2) wherein the level is a value based on the bit depth and the dynamic range of the buffer that stores the coefficient data.
  • the level is a value among a value based on the bit depth and not based on the dynamic range of the buffer and a value not based on the bit depth and based on the dynamic range of the buffer.
  • the image processing apparatus according to (4) which is a value derived using the smaller one.
  • the clip processing unit is Of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, the value obtained by subtracting 1 from the power of 2 having the smaller one as the power index is set as the upper limit value.
  • Clip the brightness component and the color component of the coefficient data Of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, the value obtained by multiplying the power of 2 with the smaller one as the power index by -1 is used as the lower limit value.
  • the image processing apparatus according to (5) which clips the brightness component and the color component of the coefficient data.
  • the clip processing unit is Clip the luminance component of the coefficient data at the first level and The image processing apparatus according to (1), which clips the color component of the coefficient data at the second level.
  • the clip processing unit is Clip the luminance component of the coefficient data with the value obtained by subtracting 1 from the power of 2 with the bit depth as the exponent as the upper limit value. Clip the luminance component of the coefficient data with the value obtained by multiplying the power of 2 with the bit depth as the exponent by -1 as the lower limit value.
  • the color component of the coefficient data is clipped with the value obtained by subtracting 1 from the power of 2 having the value obtained by adding 1 to the bit depth as the power index as the upper limit value.
  • the clip processing unit is Of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, the value obtained by subtracting 1 from the power of 2 having the smaller one as the power index is set as the upper limit value, and the brightness component of the coefficient data is used.
  • Clip and Of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer the value obtained by multiplying the power of 2 with the smaller one as the power index by -1 is set as the lower limit value, and the brightness of the coefficient data.
  • Clip the ingredients Of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, the value obtained by subtracting 1 from the power of 2 having the smaller one as the power index is set as the upper limit value.
  • the image processing apparatus which clips the color component of the coefficient data.
  • the image processing apparatus which clips the color component of the coefficient data.
  • the image processing apparatus wherein the level is a value based on the dynamic range of a buffer that stores the coefficient data.
  • the clip processing unit is The brightness component and color component of the coefficient data are clipped with the value obtained by subtracting 1 from the power of 2 having the value obtained by subtracting 1 from the dynamic range of the buffer as the power index as the upper limit value.
  • the brightness component and the color component of the coefficient data are clipped with the value obtained by multiplying the power of 2 having the value obtained by subtracting 1 from the dynamic range of the buffer as the power index and multiplying by -1 as the lower limit value (12).
  • (14) Further provided with an inverse orthogonal transform unit for inversely orthogonal transforming the orthogonal transform coefficient and generating the coefficient data adaptively color-converted by the reversible method.
  • an inverse quantization unit that inversely quantizes the quantization coefficient and generates the orthogonal transformation coefficient.
  • the image processing apparatus wherein the inverse orthogonal transform unit is configured to perform inverse orthogonal transform on the orthogonal transform coefficient generated by the inverse quantization unit.
  • the dequantization unit is configured to dequantize the quantization coefficient generated by the decoding unit.
  • the inverse adaptive color conversion unit is configured to perform reverse adaptive color conversion of the coefficient data clipped at the level by the clip processing unit by the lossless method to generate predicted residual data of image data.
  • the image processing apparatus further including a calculation unit that adds the prediction data of the image data to the prediction residual data generated by the inverse adaptive color conversion unit and generates the image data.
  • the image data is subjected to adaptive color conversion by a reversible method to generate the coefficient data, the coefficient data is orthogonally converted to generate the orthogonal conversion coefficient, and the orthogonal conversion coefficient is quantized to obtain the quantization coefficient.
  • the conversion quantization unit to be generated and Further provided with a coding unit that encodes the quantization coefficient generated by the conversion quantization unit to generate coded data.
  • the dequantization unit is configured to dequantize the quantization coefficient generated by the conversion quantization unit.
  • a calculation unit for subtracting the prediction data of the image data from the image data and generating the prediction residual data is further provided.
  • the conversion quantization unit generates the coefficient data by performing adaptive color conversion of the predicted residual data generated by the calculation unit by a reversible method, and orthogonally converts the coefficient data to generate the orthogonal conversion coefficient.
  • the coefficient data that has undergone adaptive color conversion by the lossless method is clipped at a level based on the bit depth of the coefficient data. An image processing method in which the coefficient data clipped at the level is subjected to inverse adaptive color conversion by the reversible method.
  • 100 reverse adaptive color conversion device 101 selection unit, 102 clip processing unit, 103 inverse YCgCo-R conversion unit, 200 reverse quantization reverse conversion device, 201 inverse quantization unit, 202 inverse orthogonal conversion unit, 203 reverse adaptive color conversion unit , 400 image decoding device, 401 control unit, 412 decoding unit, 413 inverse quantization inverse conversion unit, 414 arithmetic unit, 415 in-loop filter unit, 416 sorting buffer, 417 frame memory, 418 prediction unit, 500 image encoding device , 501 control unit, 511 sorting buffer, 512 arithmetic unit, 313 conversion quantization unit, 514 coding unit, 515 storage buffer, 516 inverse quantization inverse conversion unit, 517 arithmetic unit, 518 in-loop filter unit, 519 frame memory , 520 Prediction unit, 521 rate control unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
PCT/JP2021/002296 2020-01-24 2021-01-22 画像処理装置および方法 Ceased WO2021149812A1 (ja)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US17/782,173 US12309397B2 (en) 2020-01-24 2021-01-22 Image processing apparatus and method
KR1020227020349A KR20220128338A (ko) 2020-01-24 2021-01-22 화상 처리 장치 및 방법
CN202180009380.3A CN114982235B (zh) 2020-01-24 2021-01-22 图像处理装置和方法
EP21744921.4A EP4087242A4 (en) 2020-01-24 2021-01-22 IMAGE PROCESSING DEVICE AND METHOD
JP2021572823A JP7622652B2 (ja) 2020-01-24 2021-01-22 画像処理装置および方法
JP2025004343A JP7747236B2 (ja) 2020-01-24 2025-01-10 画像処理装置および方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202062965767P 2020-01-24 2020-01-24
US62/965767 2020-01-24

Publications (1)

Publication Number Publication Date
WO2021149812A1 true WO2021149812A1 (ja) 2021-07-29

Family

ID=76992967

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/002296 Ceased WO2021149812A1 (ja) 2020-01-24 2021-01-22 画像処理装置および方法

Country Status (6)

Country Link
US (1) US12309397B2 (https=)
EP (1) EP4087242A4 (https=)
JP (2) JP7622652B2 (https=)
KR (1) KR20220128338A (https=)
CN (1) CN114982235B (https=)
WO (1) WO2021149812A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021229784A1 (ja) * 2020-05-15 2021-11-18 三菱電機株式会社 攻撃検知システム、攻撃検知方法および攻撃検知プログラム

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016123232A1 (en) * 2015-01-30 2016-08-04 Qualcomm Incorporated Clipping for cross-component prediction and adaptive color transform for video coding

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4568242A1 (en) * 2016-05-13 2025-06-11 Sony Group Corporation Image processing apparatus and method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016123232A1 (en) * 2015-01-30 2016-08-04 Qualcomm Incorporated Clipping for cross-component prediction and adaptive color transform for video coding

Non-Patent Citations (10)

* Cited by examiner, † Cited by third party
Title
"Advanced video coding for generic audiovisual services", RECOMMENDATION ITU-T H.264, April 2017 (2017-04-01)
BENJAMIN BROSSJIANLE CHENSHAN LIUYE-KUI WANG: "Versatile Video Coding (Draft 7", JVET-P2001-VE, JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 16TH MEETING:FENEVA, CH
JIANLE CHENYAN YESEUNG HWAN KIM: "Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7", JVET-P2002-VL, JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 16TH MEETING:GENEVA, CH
See also references of EP4087242A4
X. XIU (KWAI), Y.-W. CHEN, T.-C. MA, H.-J. JHU, X. WANG (KWAI), J. ZHAO (LGE), S.-H. KIM (LGE), J. XU (BYTEDANCE), L. ZHANG, W. ZH: "On clipping input residuals to IACT", 130. MPEG MEETING; 20200420 - 20200424; ALPBACH; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), no. m53413 ; JVET-R0355, 18 April 2020 (2020-04-18), pages 1 - 5, XP030286913 *
XIAOYU XIUYI-WEN CHENTSUNG-CHUAN MAHONG-JHENG JHUXIANGLIN WANG: "AHG16:Clipping residual samples for ACT", JVET-Q0513_R2, JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 17TH MEETING:BRUSSELS, BE
XIAOYU XIUYI-WEN CHENTSUNG-CHUAN MAHONG-JHENG JHUXIANGLIN WANGJIE ZHAOHENDRY, SEETHAL PALURISEUNG HWAN KIMWEIJIA ZHUJIZHENG XU: "ACT colour conversion for both lossless and lossy coding", JVET-Q0510_R2, JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 17TH MEETING:BRUSSELS, BE
XIU (KWAI) X; Y-W CHEN; T-C MA; H-J JHU; WANG (KWAI) X; ZHAO (LGE) J; HENDRY; PALURI S; S-H KIM (LGE); ZHU (BYTEDANCE) W; XU J; ZH: "ACT colour conversion for both lossless and lossy coding", 17. JVET MEETING; 20200107 - 20200117; BRUSSELS; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), no. JVET-Q0510 ; m52109, 12 January 2020 (2020-01-12), Brussels, BE, pages 1 - 8, XP030223774 *
XIU, XIAOYU ET AL.: "AHG16: Clipping residual samples for ACT", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JVET-Q0513_R2, 17TH MEETING, 14 January 2020 (2020-01-14), Brussels, BE, pages 1 - 4, XP030223795 *
XIU, XIAOYU ET AL.: "Mismatch on clipping input residuals to IACT", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JVET-R0329, 18TH MEETING: BY TELECONFERENCE, April 2020 (2020-04-01), pages 1 - 2, XP030286521 *

Also Published As

Publication number Publication date
US20230045106A1 (en) 2023-02-09
EP4087242A1 (en) 2022-11-09
CN114982235B (zh) 2025-09-30
US12309397B2 (en) 2025-05-20
JPWO2021149812A1 (https=) 2021-07-29
JP7622652B2 (ja) 2025-01-28
KR20220128338A (ko) 2022-09-20
JP7747236B2 (ja) 2025-10-01
CN114982235A (zh) 2022-08-30
EP4087242A4 (en) 2023-05-03
JP2025061217A (ja) 2025-04-10

Similar Documents

Publication Publication Date Title
US12177435B2 (en) Image processing device and method
WO2021045188A1 (ja) 画像処理装置および方法
US20240380888A1 (en) Image processing apparatus and method
WO2021039650A1 (ja) 画像処理装置および方法
WO2021100588A1 (ja) 画像処理装置および方法
US12244837B2 (en) Image processing apparatus and method
JP7747236B2 (ja) 画像処理装置および方法
JP2026012356A (ja) 非一時的なコンピュータ読み取り可能な媒体、および、伝送方法
US11973948B2 (en) Image processing device and method
US12501057B2 (en) Image processing apparatus and method
WO2022044845A1 (ja) 画像処理装置および方法
WO2021045187A1 (ja) 画像処理装置および方法
US12621452B2 (en) Image processing device and method
US12192462B2 (en) Image processing device and method
US20220021899A1 (en) Image encoding apparatus, image encoding method, image decoding apparatus, and image decoding method
WO2023195330A1 (ja) 画像処理装置および方法
WO2021106612A1 (ja) 画像処理装置および方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21744921

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021572823

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021744921

Country of ref document: EP

Effective date: 20220803

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 17782173

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 202180009380.3

Country of ref document: CN