EP4351143A1 - Dispositif et procédé de traitement d'image - Google Patents

Dispositif et procédé de traitement d'image Download PDF

Info

Publication number
EP4351143A1
EP4351143A1 EP22816141.0A EP22816141A EP4351143A1 EP 4351143 A1 EP4351143 A1 EP 4351143A1 EP 22816141 A EP22816141 A EP 22816141A EP 4351143 A1 EP4351143 A1 EP 4351143A1
Authority
EP
European Patent Office
Prior art keywords
range extension
flag
control information
range
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP22816141.0A
Other languages
German (de)
English (en)
Other versions
EP4351143A4 (fr
Inventor
Takeshi Tsukuba
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Publication of EP4351143A1 publication Critical patent/EP4351143A1/fr
Publication of EP4351143A4 publication Critical patent/EP4351143A4/fr
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/98—Adaptive-dynamic-range coding [ADRC]

Definitions

  • the present disclosure relates to an image processing device and method, and more particularly, to an image processing device and method that enable the use of an encoding tool for bit-depth extension while maintaining compatibility with a specification that does not extend a bit depth of an image.
  • Non-Patent Documents 1 and Non-Patent Document 2 there has been proposed an encoding method of deriving a prediction residual of a moving image and performing coefficient transform and quantization for encoding (e.g., see Non-Patent Documents 1 and Non-Patent Document 2). Furthermore, in such image encoding, it has been proposed to support extending a bit depth of an image to be processed by extending an operation range (also referred to as a dynamic range) (e.g., see Non-Patent Document 3). In addition, an encoding tool (operation-range extension tool) as a function for a high bit depth and a high bit rate has been proposed (e.g., see Non-Patent Documents 4 to 6).
  • Non-Patent Document 3 By applying the method described in Non-Patent Document 3, an operation range (dynamic range) of each of an encoder and a decoder conforming to the encoding/decoding methods described in Non-Patent Document 1 and Non-Patent Document 2 can be extended, and the operation-range extension tools described in Non-Patent Documents 4 to 6 can be used.
  • the operation-range extension tools described in Non-Patent Document 4 to Non-Patent Document 6 are encoding tools for processing an image with an extended bit depth, and are not used in a case where the operation range is not extended.
  • the method described in Non-Patent Document 3 there has been a possibility of generating a bitstream that is not compatible with the image encoding method in a case where the operation range described in Non-Patent Document 1 and Non-Patent Document 2 is not extended. Therefore, there has been a possibility that a decoder conforming to the decoding methods described in Non-Patent Document 1 and Non-Patent Document 2 cannot decode the bitstream.
  • the present disclosure has been made in view of such a situation, and enables the use of an encoding tool for bit-depth extension while maintaining compatibility with a specification that does not extend a bit depth of an image.
  • An image processing device is an image processing device including: an operation-range extension flag encoding unit that encodes an operation-range extension flag in accordance with a bit depth of an image; and an operation-range extension tool control information encoding unit that encodes operation-range extension tool control information in accordance with the operation-range extension flag.
  • the operation-range extension flag is flag information regarding extension of an operation range of the image
  • the operation-range extension tool control information is control information of an operation-range extension tool used in a case where the operation range is extended.
  • An image processing method is an image processing method including: encoding an operation-range extension flag in accordance with a bit depth of an image; and encoding operation-range extension tool control information in accordance with the operation-range extension flag.
  • the operation-range extension flag is flag information regarding extension of an operation range of the image
  • the operation-range extension tool control information is control information of an operation-range extension tool used in a case where the operation range is extended.
  • An image processing device is an image processing device including: an operation-range extension flag encoding unit that encodes an operation-range extension flag; and an operation-range extension tool control information encoding unit that encodes operation-range extension tool control information in accordance with a bit depth of an image.
  • the operation-range extension flag is flag information regarding extension of an operation range of the image
  • the operation-range extension tool control information is control information of an operation-range extension tool used in a case where the operation range is extended.
  • An image processing method is an image processing method including: encoding an operation-range extension flag; and encoding operation-range extension tool control information in accordance with a bit depth of an image.
  • the operation-range extension flag is flag information regarding extension of an operation range of the image
  • the operation-range extension tool control information is control information of an operation-range extension tool used in a case where the operation range is extended.
  • the operation-range extension flag is encoded in accordance with a bit depth of an image
  • the operation-range extension tool control information is encoded in accordance with the operation-range extension flag
  • an operation-range extension flag is encoded, and operation-range extension tool control information is encoded in accordance with a bit depth of an image.
  • the scope disclosed in the present technology includes not only the content described in the embodiments but also the content described in the following non-patent documents and the like that are known at the time of filing, the content of other documents referred to in the following non-patent documents, and the like.
  • the content described in the above-described non-patent documents also serves as a basis for determining a support requirement.
  • the quad-tree block structure and the quad-tree plus binary tree (QTBT) block structure described in the above-described non-patent documents are not directly described in the embodiments, they are within the scope of disclosure of the present technology and are assumed to satisfy the support requirements of the claims.
  • technical terms such as parsing, syntax, and semantics are also within the scope of disclosure of the present technology even in a case where there is no direct description in the embodiments, and are assumed to meet the support requirements of the claims.
  • a "block” (not a block indicating a processing unit) used in the description as a partial area or a unit of processing in an image (picture) indicates an arbitrary partial area in the picture unless otherwise specified, and does not limit its size, shape, characteristics, and the like.
  • examples of the "block” include an arbitrary partial region (unit of processing) such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a smallest coding unit (SCU), a coding unit (CU), a largest coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a sub-block, a macroblock, a tile, or a slice described in an above-described Non-Patent Document.
  • a transform block a transform unit (TU), a prediction block (PB), a prediction unit (PU), a smallest coding unit (SCU), a coding unit (CU), a largest coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a sub-block, a macroblock, a tile, or a slice described in an above-described Non-Patent Document.
  • the block size may be specified not only directly but also indirectly.
  • the block size may be specified using identification information that identifies the size.
  • the block size may be specified by a ratio or difference with the size of the reference block (e.g., LCU, SCU, etc.).
  • the specification of a block size also includes specification of an area of the block size (e.g., specification of an area of an allowable block size, etc.).
  • Non-Patent Document 1 and Non-Patent Document 2 there has been an encoding method of deriving a prediction residual of a moving image, performing coefficient transform, quantizing, and encoding.
  • Non-Patent Document 3 in such image encoding, there has been a method of supporting extension of a bit depth of an image to be processed by extending an operation range (also referred to as a dynamic range).
  • an encoding tool operation-range extension tool
  • Non-Patent Document 3 By applying the method described in Non-Patent Document 3, an operation range (dynamic range) of each of an encoder and a decoder conforming to the encoding/decoding methods described in Non-Patent Document 1 and Non-Patent Document 2 can be extended, and the operation-range extension tools described in Non-Patent Documents 4 to 6 can be used.
  • extending the operation range means extending a bit depth of an image that can be processed (supported) by an encoder or a decoder.
  • This bit depth may be any number of bits.
  • the encoder or the decoder can process an image having a bit depth of 10 bits or less in the basic specification, and it may be assumed that the encoder or the decoder can process an image having a bit depth (e.g., 12 bits, etc.) greater than 10 bits by extending the operation range.
  • the operation-range extension tool is a tool that can be used in a case where the encoder or the decoder processes (encodes or decodes) an image having a bit depth greater than 10 bits (high bit-depth, high bit-rate image).
  • the extended operation range is also referred to as an extended dynamic range.
  • Non-Patent Documents 4 to 6 are encoding tools for processing an image with an extended bit depth, and are not used in a case where the operation range of the encoder or the decoder is not extended.
  • Fig. 1 is a diagram illustrating an example of syntax related to the operation-range extension tool described in Non-Patent Document 3.
  • Fig. 2 is a diagram illustrating an example of semantics of each syntax element illustrated in Fig. 1 .
  • sps_extension_flag In a case where sps_extension_flag is true (e.g., "1"), it indicates that an operation-range extension flag (sps_range_extension_flag) exists in a sequence parameter set (sequence parameter set (SPS)) or the like. Furthermore, in a case where sps_extension_flag is false (e.g., "0"), it indicates that the operation-range extension flag (sps_range_extension_flag) does not exist in the sequence parameter set or the like.
  • SPS sequence parameter set
  • the operation-range extension flag (sps_range_extension_flag) is flag information regarding the extension of the operation range of the image. In a case where sps_range_extension_flag is true (e.g., "1"), it indicates that operation-range extension tool control information (sps_range_extension()) exists in the sequence parameter set or the like. In a case where sps_range_extension_flag is false (e.g., "0"), it indicates that sps_range_extension() does not exist in the sequence parameter set or the like.
  • the operation-range extension tool control information (sps_range_extension()) is control information of the operation-range extension tool used in a case where the operation range is extended.
  • Fig. 3 illustrates an example of syntax of sps_range_extension().
  • Fig. 4 is a diagram illustrating an example of semantics of each syntax element illustrated in Fig. 3 .
  • the operation-range extension tool control information may include, for example, a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • This control flag is flag information for controlling whether or not to apply the encoding tool for extending the internal calculation precision of the orthogonal transform described in Non-Patent Document 6.
  • extended_precision_processing_flag is true (e.g., "1")
  • extended_precision_processing_flag is false (e.g., "0")
  • extended dynamic range cannot be used for transform coefficients and transform.
  • the operation-range extension tool control information may include, for example, a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to extension of a Rice parameter of transform skip residual coding (TSRC).
  • This control flag is flag information for controlling whether or not to apply the encoding tool for extending the Rice parameter of TSRC described in Non-Patent Document 5.
  • sps_ts_residual_coding_rice_present_in_sh_flag true (e.g., "1")
  • sh_ts_residual_coding_rice_idx_minus1 can exist in the slice header that refers to the sequence parameter set.
  • the operation-range extension tool control information may include, for example, a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to extension of a Rice parameter of regular residual coding (RRC).
  • control flags are flag information for controlling the encoding tool for extending the Rice parameter of the RRC described in Non-Patent Document 4.
  • sps_rrc_rice_extension_flag true (e.g., "1"), it indicates that extension of derivation of a Rice parameter for binarization of abs_remaining[] and dec_abs_level[] is valid.
  • sps_rrc_rice_extension_flag false (e.g., "0"), it indicates that the extension of the derivation of the Rice parameter is invalid.
  • sps_persistent_rice_adaptation_enabled_flag true (e.g., "1")
  • sps_persistent_rice_adaptation_enabled_flag false (e.g., "0"), it indicates that a previous TU state is not used in the derivation of the Rice parameter.
  • the operation-range extension tool control information (sps_range_extension()) is encoded and transmitted.
  • the operation-range extension information regarding the operation-range extension is controlled on the basis of the bit depth.
  • the operation-range extension information can include, for example, an operation-range extension flag and operation-range extension tool control information.
  • Fig. 6 is a diagram illustrating an example of semantics in that case. As indicated in a square frame 11 in Fig. 6 , it has been set as the bitstream constraint that in a case where a bit depth (BitDepth) of an image is 10 bits or less, the value of sps_range_extension_flag is equal to false (e.g., "0").
  • the encoded data of) the operation-range extension tool control information is not stored in the bitstream in a case where the operation range is not extended.
  • the bitstream can maintain compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. Therefore, even a decoder that does not support the extension of the operation range can correctly decode the bitstream. That is, it is possible to use an encoding tool for bit-depth extension while maintaining compatibility with a specification that does not extend a bit depth of an image.
  • Fig. 7 is a diagram illustrating an example of semantics in that case.
  • an internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform may be set to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, the value of the internal variable (ExtendedPrecisionFlag) may be equal to the value of the control flag (extended_precision_processing_flag). That is, in this case, control based on the bit depth (BitDepth) of the image is unnecessary for setting the value of the internal variable (ExtendedPrecisionFlag).
  • Fig. 8 is a block diagram illustrating an example of a configuration of an operation-range extension information encoding device that is one aspect of an image processing device to which the present technology is applied.
  • An operation-range extension information encoding device 100 illustrated in Fig. 8 is a device that performs processing related to encoding of operation-range extension information.
  • the operation-range extension information encoding device 100 acquires operation-range extension information such as an operation-range extension flag and operation-range extension tool control information.
  • the operation-range extension information encoding device 100 acquires a parameter BitDepth indicating a bit depth of an image.
  • the operation-range extension information encoding device 100 uses these pieces of information to perform processing related to the encoding of the operation-range extension information.
  • the operation-range extension information encoding device 100 appropriately outputs encoded data of the operation-range extension information and the like as a processing result.
  • the encoded data is supplied to, for example, an encoder that encodes an image.
  • the encoder encodes an image on the basis of the operation-range extension information and generates a bitstream including the encoded data of the image.
  • the encoded data of the operation-range extension information is supplied to the decoding side, for example, by being stored in the bitstream.
  • the operation-range extension information encoding device 100 may include a processing unit not illustrated as a block in Fig. 8 . Furthermore, the operation-range extension information encoding device 100 may have processing or a data flow that is not illustrated as an arrow or the like in Fig. 8 .
  • the operation-range extension information encoding device 100 includes an operation-range extension flag encoding unit 111 and an operation-range extension tool control information encoding unit 112.
  • the operation-range extension flag encoding unit 111 performs processing related to the encoding of the operation-range extension flag (sps_range_extension_flag). For example, the operation-range extension flag encoding unit 111 may acquire the operation-range extension flag (sps_range_extension_flag). Furthermore, the operation-range extension flag encoding unit 111 may acquire a parameter (BitDepth) indicating a bit depth of an image (encoding target).
  • the operation-range extension flag encoding unit 111 encodes the operation-range extension flag (sps_range_extension_flag) in accordance with the bit depth (BitDepth) of the image. For example, the operation-range extension flag encoding unit 111 may set the operation-range extension flag to false and encode the operation-range extension flag in a case where the bit depth of the image is not extended, and may encode the operation-range extension flag in a case where the bit depth of the image is extended.
  • the operation-range extension flag encoding unit 111 may set the operation-range extension flag to false and encode the operation-range extension flag in a case where the bit depth of the image is 10 bits or less, and may encode the operation-range extension flag in a case where the bit depth of the image is greater than 10 bits.
  • the operation-range extension flag encoding unit 111 may output the encoded data of the operation-range extension flag to the outside of the operation-range extension information encoding device 100 as a processing result. Furthermore, the operation-range extension flag encoding unit 111 may supply the operation-range extension flag (sps_range_extension_flag) to the operation-range extension tool control information encoding unit 112.
  • the operation-range extension tool control information encoding unit 112 performs processing related to the encoding of the operation-range extension tool control information (sps_range_extension()). For example, the operation-range extension tool control information encoding unit 112 may acquire the operation-range extension tool control information (sps_range_extension()). Furthermore, the operation-range extension tool control information encoding unit 112 may acquire the operation-range extension flag (sps_range_extension_flag) supplied from the operation-range extension flag encoding unit 111.
  • the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information in accordance with the operation-range extension flag. For example, in a case where the operation-range extension flag is false, the operation-range extension tool control information encoding unit 112 may skip (omit) encoding the operation-range extension tool control information. In addition, the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information, for which encoding has been skipped, to a value indicating the unavailability of the operation-range extension tool. Furthermore, in a case where the operation-range extension flag is true, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information encoding unit 112 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, as described with reference to Fig. 7 , the value of the internal variable (ExtendedPrecisionFlag) may be equal to the value of the control flag (extended_precision_processing_flag).
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC.
  • the operation-range extension tool control information (sps_range_extension()) may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of regular residual coding (RRC).
  • the operation-range extension tool control information encoding unit 112 may output the encoded data of the operation-range extension tool control information to the outside of the operation-range extension information encoding device 100 as a processing result.
  • the operation-range extension information encoding device 100 can encode the operation-range extension information in compliance with the constraint illustrated in Fig. 6 . That is, the operation-range extension information encoding device 100 can generate a bitstream that does not violate the constraint illustrated in Fig. 6 by supplying the encoded data of the generated operation-range extension information to the encoder. In other words, the operation-range extension information encoding device 100 can generate a bitstream that maintains compatibility with the image encoding methods described in Non-Patent Documents 1 and 2.
  • the operation-range extension information encoding device 100 encodes the operation-range extension information in this manner, whereby the encoder and the decoder can use an encoding tool for bit-depth extension while maintaining compatibility with a specification that does not extend the bit depth of the image.
  • the operation-range extension information encoding device 100 encodes an operation-range extension flag in accordance with the bit depth of the image, and encodes the operation-range extension tool control information in accordance with the operation-range extension flag.
  • the operation-range extension flag encoding unit 111 may set the operation-range extension flag to false and encode the operation-range extension flag in a case where the bit depth of the image is not extended, and may encode the operation-range extension flag in a case where the bit depth of the image is extended.
  • the operation-range extension flag encoding unit 111 may set the operation-range extension flag to false and encode the operation-range extension flag in a case where the bit depth of the image is 10 bits or less, and may encode the operation-range extension flag in a case where the bit depth of the image is greater than 10 bits.
  • step S101 of Fig. 9 the operation-range extension flag encoding unit 111 determines whether or not the bit depth (BitDepth) of the image is greater than 10 bits.
  • step S102 the operation-range extension flag encoding unit 111 encodes the operation-range extension flag (sps_range_extension_flag).
  • step S104 the procedure shifts to step S104.
  • step S101 determines whether the bit depth (BitDepth) of the image is 10 bits or less.
  • step S104 the procedure proceeds to step S104.
  • the operation-range extension tool control information encoding unit 112 may skip encoding the operation-range extension tool control information.
  • the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information, for which encoding has been skipped, to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • step S104 the operation-range extension tool control information encoding unit 112 determines whether or not the operation-range extension flag is true (e.g., "1"). In a case where it is determined that the operation-range extension flag is true, the procedure proceeds to step S105.
  • step S105 the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information. When the processing in step S105 ends, the operation-range extension information encoding processing ends.
  • step S106 the operation-range extension tool control information encoding unit 112 skips encoding the operation-range extension tool control information. Furthermore, the operation-range extension tool control information encoding unit 112 sets (each control flag of) the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • step S106 ends, the operation-range extension information encoding processing ends.
  • the operation-range extension information encoding device 100 can encode the operation-range extension information in compliance with the constraint illustrated in Fig. 6 . That is, the operation-range extension information encoding device 100 can generate a bitstream that does not violate the constraint illustrated in Fig. 6 by supplying the encoded data of the generated operation-range extension information to the encoder. Therefore, the encoder and the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image.
  • Fig. 10 is a block diagram illustrating an example of a configuration of an operation-range extension information decoding device that is one aspect of an image processing device to which the present technology is applied.
  • An operation-range extension information decoding device 200 illustrated in Fig. 10 is a device that performs processing related to the decoding of the encoded data of the operation-range extension information.
  • the operation-range extension information decoding device 200 acquires the encoded data of the operation-range extension information (such as the operation-range extension flag and the operation-range extension tool control information) generated by the operation-range extension information encoding device 100.
  • the operation-range extension information maintains compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. That is, the bitstream of the image encoded using the operation-range extension information can be correctly decoded even by a decoder that does not support the extension of the operation range.
  • the operation-range extension information decoding device 200 performs processing related to the decoding of the encoded data of the operation-range extension information.
  • the operation-range extension information decoding device 200 appropriately outputs the operation-range extension information as a processing result.
  • the operation-range extension information is supplied to, for example, a decoder that decodes encoded data of an image.
  • the decoder decodes the encoded data on the basis of the operation-range extension information and generates (restores) an image.
  • the operation-range extension information decoding device 200 may include a processing unit not illustrated as a block in Fig. 10 . Furthermore, the operation-range extension information decoding device 200 may have processing or a data flow that is not illustrated as an arrow or the like in Fig. 10 .
  • the operation-range extension information decoding device 200 includes an operation-range extension flag decoding unit 211 and an operation-range extension tool control information decoding unit 212.
  • the operation-range extension flag decoding unit 211 performs processing related to the decoding of the encoded data of an operation-range extension flag (sps_range_extension_flag). For example, the operation-range extension flag decoding unit 211 may acquire the encoded data of the operation-range extension flag. Furthermore, the operation-range extension flag decoding unit 211 may decode the encoded data and generate (restore) the operation-range extension flag. Also, the operation-range extension flag decoding unit 211 may output the operation-range extension flag to the outside of the operation-range extension information decoding device 200 as a processing result. Furthermore, the operation-range extension flag decoding unit 211 may supply the operation-range extension flag to the operation-range extension tool control information decoding unit 212.
  • the operation-range extension tool control information decoding unit 212 performs processing related to the decoding of the encoded data of the operation-range extension tool control information (sps_range_extension()). For example, the operation-range extension tool control information decoding unit 212 may acquire the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may acquire the operation-range extension flag supplied from the operation-range extension flag decoding unit 211. The operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag. Furthermore, the operation-range extension tool control information decoding unit 212 may output the operation-range extension tool control information to the outside of the operation-range extension information decoding device 200 as a processing result.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the decoder may be set to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of the TSRC.
  • the operation-range extension tool control information may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of the RRC.
  • a control flag sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag
  • the operation-range extension information decoding device 200 can correctly decode the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. That is, the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image.
  • the operation-range extension flag decoding unit 211 decodes the encoded data of the operation-range extension flag (sps_range_extension_flag) and generates (restores) the operation-range extension flag in step S131 of Fig. 11 .
  • step S132 the operation-range extension tool control information decoding unit 212 determines whether or not the operation-range extension flag is true (e.g., "1") .
  • step S133 the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information (sps_range_extension()).
  • step S133 the operation-range extension information decoding processing ends.
  • step S134 in a case where it is determined in step S132 that the operation-range extension flag is false (e.g., "0"), the procedure proceeds to step S134.
  • the operation-range extension tool control information decoding unit 212 skips (omits) decoding the operation-range extension tool control information. Then, the operation-range extension tool control information decoding unit 212 sets (each control flag of) the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension information decoding device 200 can correctly decode the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. That is, the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image.
  • the decoder may decode the bitstream only in a case where there is no constraint violation in the bitstream, and may end or suspend decoding (further perform notification, etc.) in a case where there is a constraint violation. In this manner, the decoder can prevent the decoding of an invalid bitstream and can more safely decode the bitstream (prevent the occurrence of failure, etc.).
  • Fig. 12 is a block diagram illustrating an example of the main configuration of the operation-range extension information decoding device 200 in that case. Also in this case, the operation-range extension information decoding device 200 acquires and decodes the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. The operation-range extension information decoding device 200 appropriately outputs the operation-range extension information as a processing result.
  • the operation-range extension information is supplied to, for example, a decoder that decodes encoded data of an image. The decoder decodes the encoded data on the basis of the operation-range extension information and generates (restores) an image.
  • the operation-range extension information decoding device 200 in this case includes a constraint violation processing unit 213 in addition to the configuration (operation-range extension flag decoding unit 211 and operation-range extension tool control information decoding unit 212) described with reference to Fig. 10 .
  • the operation-range extension flag decoding unit 211 and the operation-range extension tool control information decoding unit 212 in this case each execute processing as in the case of Fig. 10 .
  • the operation-range extension flag decoding unit 211 decodes the encoded data of the operation-range extension flag (sps_range_extension_flag).
  • the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information (sps_range_extension()) in accordance with the operation-range extension flag.
  • the operation-range extension tool control information decoding unit 212 may skip decoding the encoded data of the operation-range extension tool control information and set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information decoding unit 212 may control the constraint violation processing unit 213 so as to determine whether or not there is a constraint violation in the bitstream. Furthermore, the operation-range extension tool control information decoding unit 212 may acquire the determination result. Moreover, the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information on the basis of the determination result.
  • the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information. Furthermore, in a case where it is determined that there is a constraint violation in the bitstream, the operation-range extension tool control information decoding unit 212 may skip (omit) the decoding. Also, in that case, the operation-range extension tool control information decoding unit 212 may end or suspend the decoding of the bitstream by the decoder.
  • the constraint violation processing unit 213 performs processing related to the bitstream constraint. For example, the constraint violation processing unit 213 acquires the parameter (BitDepth) indicating the bit depth of the image. Furthermore, the constraint violation processing unit 213 may check whether there is a constraint violation in the bitstream under the control of the operation-range extension tool control information decoding unit 212. That is, the constraint violation processing unit 213 can also be referred to as a checking unit.
  • the constraint violation processing unit 213 checks whether there is a constraint violation in the bitstream. As the checking as to whether there is a constraint violation in the bitstream, the constraint violation processing unit 213 may check whether the bit depth of the image has been extended in the bitstream. For example, the constraint violation processing unit 213 may check whether the bit depth of the image is greater than 10 bits in the bitstream.
  • the constraint violation processing unit 213 may supply the checking result (determination result as to whether or not there is a constraint violation in the bitstream) obtained as described above to the operation-range extension tool control information decoding unit 212. Furthermore, in a case where it is determined that there is a constraint violation in the bitstream, the constraint violation processing unit 213 may provide notification that the bitstream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify the user, the application, or the like of the fact.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information decoding unit 212 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, as described with reference to Fig. 7 , the value of the internal variable (ExtendedPrecisionFlag) may be made equal to the value of the control flag (extended_precision_processing_flag).
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC.
  • the operation-range extension tool control information (sps_range_extension()) may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of regular residual coding (RRC).
  • the operation-range extension information decoding device 200 can prevent the decoding of an invalid bitstream.
  • the decoder can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • the operation-range extension information decoding device 200 decodes the encoded data of the operation-range extension flag, and decodes the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag.
  • step S161 of Fig. 13 the operation-range extension flag decoding unit 211 decodes the encoded data of the operation-range extension flag (sps_range_extension_flag).
  • step S162 the operation-range extension tool control information decoding unit 212 determines whether or not the operation-range extension flag is true (e.g., "1"). In a case where it is determined that the operation-range extension flag is true, the procedure proceeds to step S163.
  • the operation-range extension flag e.g., "1"
  • the constraint violation processing unit 213 may check whether or not there is a constraint violation in the bitstream. For example, the constraint violation processing unit 213 may check whether or not the bit depth of the image is extended in the bitstream. For example, the constraint violation processing unit 213 may check whether or not the bit depth of the image is greater than 10 bits in the bitstream. For example, in step S163, the constraint violation processing unit 213 determines whether or not the bit depth of the image is greater than 10 bits. In a case where it is determined that the bit depth of the image is greater than 10 bits, the procedure proceeds to step S164.
  • the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information. For example, in step S164, the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information and generates (restores) the operation-range extension tool control information. When the processing in step S164 ends, the operation-range extension information decoding processing ends.
  • step S165 the procedure proceeds to step S165. That the operation-range extension flag is true and the bit depth of the image is 10 bits or less violates the constraint described with reference to Fig. 6 .
  • the constraint violation processing unit 213 may provide notification that the bitstream does not satisfy the constraint.
  • step S165 the constraint violation processing unit 213 notifies the user (or application, etc.) that the bitstream does not satisfy the constraint.
  • step S166 the operation-range extension tool control information decoding unit 212 may skip decoding the encoded data of the operation-range extension tool control information and set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • step S166 the operation-range extension tool control information decoding unit 212 omits the decoding of the encoded data of the operation-range extension tool control information, and sets each control flag of the operation-range extension tool control information to a value indicating the unavailability of the tool.
  • the operation-range extension information decoding device 200 can prevent the decoding of an invalid bitstream. As a result, the decoder can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • the value of the operation-range extension tool control information may be controlled on the basis of the bit depth (method 2).
  • Figs. 14 and 15 are diagrams illustrating examples of semantics in that case. As illustrated in a square frame 251 in Fig. 14 , it has been set as the bitstream constraint that in a case where the bit depth (BitDepth) of the image is 10 bits or less, the value of extended_precision_processing_flag is equal to false (e.g., "0"). Furthermore, as illustrated in a square frame 253 in Fig.
  • bitstream constraint in a case where the bit depth (BitDepth) of the image is 10 bits or less, the value of sps_persistent_rice_adaptation_enabled_flag is equal to false (e.g., "0").
  • the encoded data of) the operation-range extension tool control information is not stored in the bitstream in a case where the operation range is not extended.
  • the bitstream can maintain compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. Therefore, even a decoder that does not support the extension of the operation range can correctly decode the bitstream. That is, it is possible to use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform may be set to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, the value of the internal variable (ExtendedPrecisionFlag) may be equal to the value of the control flag (extended_precision_processing_flag). That is, in this case, control based on the bit depth (BitDepth) of the image is unnecessary for setting the value of the internal variable (ExtendedPrecisionFlag).
  • Fig. 16 is a block diagram illustrating an example of the main configuration of the operation-range extension information encoding device 100 in this case.
  • the operation-range extension information encoding device 100 is a device that performs processing related to the encoding of the operation-range extension information.
  • the operation-range extension information encoding device 100 acquires operation-range extension information such as an operation-range extension flag and operation-range extension tool control information.
  • the operation-range extension information encoding device 100 acquires a parameter BitDepth indicating a bit depth of an image.
  • the operation-range extension information encoding device 100 uses these pieces of information to perform processing related to the encoding of the operation-range extension information.
  • the operation-range extension information encoding device 100 appropriately outputs encoded data of the operation-range extension information and the like as a processing result.
  • the encoded data is supplied to, for example, an encoder that encodes an image.
  • the encoder encodes an image on the basis of the operation-range extension information and generates a bitstream including the encoded data of the image.
  • the encoded data of the operation-range extension information is supplied to the decoding side, for example, by being stored in the bitstream.
  • the operation-range extension information encoding device 100 may include a processing unit not illustrated as a block in Fig. 16 . Furthermore, the operation-range extension information encoding device 100 may have processing or a data flow that is not illustrated as an arrow or the like in Fig. 16 .
  • the operation-range extension information encoding device 100 in this case has a configuration similar to the configuration (operation-range extension flag encoding unit 111 and operation-range extension tool control information encoding unit 112) illustrated in Fig. 8 .
  • the parameter (BitDepth) indicating the bit depth of the image is supplied to the operation-range extension tool control information encoding unit 112.
  • the operation-range extension flag encoding unit 111 performs processing related to the encoding of the operation-range extension flag (sps_range_extension_flag). For example, the operation-range extension flag encoding unit 111 may acquire the operation-range extension flag. However, the operation-range extension flag encoding unit 111 encodes the operation-range extension flag regardless of the bit depth of the image. Furthermore, the operation-range extension flag encoding unit 111 may output the encoded data of the operation-range extension flag to the outside of the operation-range extension information encoding device 100 as a processing result. Also, the operation-range extension flag encoding unit 111 may supply the operation-range extension flag to the operation-range extension tool control information encoding unit 112.
  • the operation-range extension tool control information encoding unit 112 performs processing related to the encoding of the operation-range extension tool control information (sps_range_extension()). For example, the operation-range extension tool control information encoding unit 112 may acquire the operation-range extension tool control information. Furthermore, the operation-range extension tool control information encoding unit 112 may acquire the operation-range extension flag supplied from the operation-range extension flag encoding unit 111. Moreover, the operation-range extension tool control information encoding unit 112 may acquire the parameter (BitDepth) indicating the bit depth of the image.
  • the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information in accordance with the bit depth (and the operation-range extension flag) of the image. For example, in a case where the operation-range extension flag is true and the bit depth is not extended, the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool, and encode the operation-range extension tool control information. Furthermore, in a case where the operation-range extension flag is true and the bit depth is extended, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool, and encode the operation-range extension tool control information. Furthermore, in a case where the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • the operation-range extension tool control information encoding unit 112 may skip encoding the operation-range extension tool control information. Also, in that case, the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information encoding unit 112 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, as described with reference to Fig. 14 , the value of the internal variable (ExtendedPrecisionFlag) may be equal to the value of the control flag (extended_precision_processing_flag).
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC.
  • the operation-range extension tool control information (sps_range_extension()) may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of regular residual coding (RRC).
  • the operation-range extension tool control information encoding unit 112 may output the encoded data of the operation-range extension tool control information to the outside of the operation-range extension information encoding device 100 as a processing result.
  • the operation-range extension information encoding device 100 can encode the operation-range extension information in compliance with the constraints illustrated in Figs. 14 and 15 . That is, the operation-range extension information encoding device 100 can generate a bitstream that does not violate the constraints illustrated in Figs. 14 and 15 by supplying the encoded data of the generated operation-range extension information to the encoder. In other words, the operation-range extension information encoding device 100 can generate a bitstream that maintains compatibility with the image encoding methods described in Non-Patent Documents 1 and 2.
  • the operation-range extension information encoding device 100 encodes the operation-range extension information in this manner, whereby the encoder and the decoder can use an encoding tool for bit-depth extension while maintaining compatibility with a specification that does not extend the bit depth of the image.
  • the operation-range extension information encoding device 100 encodes an operation-range extension flag, and encodes the operation-range extension tool control information in accordance with the bit depth of the image.
  • step S201 of Fig. 17 the operation-range extension flag encoding unit 111 encodes an operation-range extension flag (sps_range_extension_flag).
  • step S202 the operation-range extension tool control information encoding unit 112 determines whether or not the operation-range extension flag is true (e.g., "1"). In a case where it is determined that the operation-range extension flag is true, the procedure proceeds to step S203.
  • the operation-range extension flag e.g., "1"
  • the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool, and encode the operation-range extension tool control information. Furthermore, when the bit depth of the image is to be extended, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information. For example, in a case where the bit depth of the image is 10 bits or less, the operation-range extension tool control information encoding unit 112 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool, and encode the operation-range extension tool control information. Furthermore, in a case where the bit depth of the image is greater than 10 bits, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • step S203 the operation-range extension tool control information encoding unit 112 determines whether or not the bit depth (BitDepth) of the image is greater than 10 bits. In a case where it is determined that the bit depth (BitDepth) of the image is greater than 10 bits, the procedure proceeds to step S204.
  • step S204 the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information. When the processing in step S204 ends, the operation-range extension information encoding processing ends.
  • step S205 the operation-range extension tool control information encoding unit 112 sets (each control flag of) the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool, and encodes the operation-range extension tool control information.
  • the operation-range extension information encoding processing ends.
  • step S206 the operation-range extension tool control information encoding unit 112 skips encoding the operation-range extension tool control information. Furthermore, the operation-range extension tool control information encoding unit 112 sets (each control flag of) the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • step S206 ends, the operation-range extension information encoding processing ends.
  • the operation-range extension information encoding device 100 can encode the operation-range extension information in compliance with the constraints illustrated in Figs. 14 and 15 . That is, the operation-range extension information encoding device 100 can generate a bitstream that does not violate the constraints illustrated in Figs. 14 and 15 by supplying the encoded data of the generated operation-range extension information to the encoder. Therefore, the encoder and the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image.
  • the main configuration of the operation-range extension information decoding device 200 that acquires and decodes the encoded data of the operation-range extension information (the operation-range extension flag, the operation-range extension tool control information, etc.) generated by the operation-range extension information encoding device 100 of Fig. 16 is similar to the example described with reference to Fig. 10 . Therefore, its description is omitted.
  • operation-range extension information decoding processing executed by the operation-range extension information decoding device 200 is executed by a flow similar to the example described with reference to the flowchart of Fig. 11 . Therefore, its description is omitted.
  • the decoder may decode the bitstream only in a case where there is no constraint violation in the bitstream, and may end or suspend decoding (further perform notification, etc.) in a case where there is a constraint violation. In this manner, the decoder can prevent the decoding of an invalid bitstream and can more safely decode the bitstream (prevent the occurrence of failure, etc.).
  • the operation-range extension information decoding device 200 in this case has a configuration similar to that of the example described with reference to Fig. 12 . That is, the operation-range extension information decoding device 200 includes the operation-range extension flag decoding unit 211, the operation-range extension tool control information decoding unit 212, and the constraint violation processing unit 213.
  • the operation-range extension flag decoding unit 211 in this case decodes the encoded data of the operation-range extension flag (sps_range_extension_flag). Furthermore, the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information (sps_range_extension()) in accordance with the operation-range extension flag. For example, in a case where the operation-range extension flag is false, the operation-range extension tool control information decoding unit 212 may skip decoding the encoded data of the operation-range extension tool control information and set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information. Then, the operation-range extension tool control information decoding unit 212 may control the constraint violation processing unit 213 so as to determine whether or not there is a constraint violation in the bitstream. That is, the constraint violation processing unit 213 may check whether or not there is a constraint violation in the bitstream.
  • the constraint violation processing unit 213 may check whether the bit depth of the image has not been extended in the bitstream and whether the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool. For example, the constraint violation processing unit 213 may check whether the bit depth is 10 bits or less in the bitstream and whether the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool. Then, in a case where there is a constraint violation in the bitstream, the constraint violation processing unit 213 may provide notification that the bitstream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify the user, the application, or the like of the fact.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information decoding unit 212 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, as described with reference to Fig. 7 , the value of the internal variable (ExtendedPrecisionFlag) may be made equal to the value of the control flag (extended_precision_processing_flag).
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC.
  • the operation-range extension tool control information (sps_range_extension()) may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of regular residual coding (RRC).
  • the operation-range extension information decoding device 200 can prevent the decoding of an invalid bitstream.
  • the decoder can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • the operation-range extension information decoding device 200 decodes the encoded data of the operation-range extension flag, and decodes the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag.
  • step S261 of Fig. 18 the operation-range extension flag decoding unit 211 decodes the encoded data of the operation-range extension flag (sps_range_extension_flag).
  • step S262 the operation-range extension tool control information decoding unit 212 determines whether or not the operation-range extension flag is true (e.g., "1"). In a case where it is determined that the operation-range extension flag is true, the procedure proceeds to step S263. In step S263, the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information, and generates (restores) the operation-range extension tool control information.
  • the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information, and generates (restores) the operation-range extension tool control information.
  • the constraint violation processing unit 213 may check whether there is a constraint violation in the bitstream by using the operation-range extension tool control information. For example, the constraint violation processing unit 213 may check whether the bit depth of the image has not been extended in the bitstream and whether the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool. Furthermore, the constraint violation processing unit 213 may check whether the bit depth is 10 bits or less in the bitstream and whether the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool. Then, in a case where there is a constraint violation in the bitstream, the constraint violation processing unit 213 may provide notification that the bitstream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify the user, the application, or the like of the fact.
  • step S264 the constraint violation processing unit 213 determines whether the bit depth is 10 bits or less in the bitstream and whether the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool. In a case where it is determined that the bit depth is 10 bits or less in the bitstream and the operation-range extension tool control information is not a value indicating the unavailability of the operation-range extension tool, the procedure proceeds to step S265. In step S265, the constraint violation processing unit 213 notifies the user (or application, etc.) that the bitstream does not satisfy the constraint. When the processing in step S265 ends, the operation-range extension information decoding processing ends.
  • step S264 determines whether the bit depth is greater than 10 bits in the bitstream or the operation-range extension tool control information is a value indicating the unavailability of the operation-range extension tool. Furthermore, in a case where it is determined in step S264 that the bit depth is greater than 10 bits in the bitstream or the operation-range extension tool control information is a value indicating the unavailability of the operation-range extension tool, the processing of step S265 is omitted, and the operation-range extension information decoding processing ends.
  • step S266 the operation-range extension tool control information decoding unit 212 may skip decoding the encoded data of the operation-range extension tool control information and set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, in step S266, the operation-range extension tool control information decoding unit 212 omits the decoding of the encoded data of the operation-range extension tool control information, and sets each control flag of the operation-range extension tool control information to a value indicating the unavailability of the tool.
  • the operation-range extension information decoding processing ends.
  • the operation-range extension information decoding device 200 can prevent the decoding of an invalid bitstream. As a result, the decoder can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • Fig. 19 is a diagram illustrating an example of syntax related to the operation-range extension tool control information in that case.
  • a row indicated by gray is added. That is, an if statement with a bit depth as a condition is added. That is, each control flag of the operation-range extension tool control information (sps_range_extension()) is encoded and transmitted only in a case where the bit depth of the image is greater than 10 bits.
  • the encoded data of) the operation-range extension tool control information is not stored in the bitstream in a case where the operation range is not extended.
  • the bitstream can maintain compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. Therefore, even a decoder that does not support the extension of the operation range can correctly decode the bitstream. That is, it is possible to use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • the transmission of the control flag of the encoding tool for high bit depth and high bit rate in the parameter set can be omitted. This can reduce an increase in the code amount.
  • the operation-range extension information encoding device 100 is a device that performs processing related to the encoding of the operation-range extension information.
  • the operation-range extension information encoding device 100 acquires operation-range extension information such as an operation-range extension flag and operation-range extension tool control information.
  • the operation-range extension information encoding device 100 acquires a parameter BitDepth indicating a bit depth of an image. The operation-range extension information encoding device 100 uses these pieces of information to perform processing related to the encoding of the operation-range extension information.
  • the operation-range extension information encoding device 100 appropriately outputs encoded data of the operation-range extension information and the like as a processing result.
  • the encoded data is supplied to, for example, an encoder that encodes an image.
  • the encoder encodes an image on the basis of the operation-range extension information and generates a bitstream including the encoded data of the image.
  • the encoded data of the operation-range extension information is supplied to the decoding side, for example, by being stored in the bitstream.
  • the main configuration of the operation-range extension information encoding device 100 in this case is similar to the example described with reference to Fig. 16 . That is, the operation-range extension information encoding device 100 in this case includes the operation-range extension flag encoding unit 111 and the operation-range extension tool control information encoding unit 112.
  • the operation-range extension flag encoding unit 111 encodes the operation-range extension flag regardless of the bit depth of the image.
  • the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information in accordance with the operation-range extension flag and the bit depth of the image. For example, in a case where the operation-range extension flag is false or the bit depth is not extended, the operation-range extension tool control information encoding unit 112 may skip (omit) encoding the operation-range extension tool control information. Furthermore, in a case where the operation-range extension flag is true and the bit depth is extended, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • the operation-range extension tool control information encoding unit 112 may skip (omit) encoding the operation-range extension tool control information. Furthermore, in a case where the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information.
  • the operation-range extension tool control information encoding unit 112 may further set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. Furthermore, in a case where the operation-range extension flag is false or the bit depth of the image is 10 bits or less, the operation-range extension tool control information encoding unit 112 may further set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information encoding unit 112 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. That is, as described with reference to Fig. 14 , the value of the internal variable (ExtendedPrecisionFlag) may be equal to the value of the control flag (extended_precision_processing_flag).
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC.
  • the operation-range extension tool control information (sps_range_extension()) may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of regular residual coding (RRC).
  • the operation-range extension tool control information encoding unit 112 may output the encoded data of the operation-range extension tool control information to the outside of the operation-range extension information encoding device 100 as a processing result.
  • the operation-range extension information encoding device 100 can prevent (the encoded data of) the operation-range extension tool control information from being stored in the bitstream in a case where the operation range is not extended.
  • the bitstream can maintain compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. Therefore, even a decoder that does not support the extension of the operation range can correctly decode the bitstream. Therefore, the encoder and the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image.
  • the transmission of the control flag of the encoding tool for high bit depth and high bit rate in the parameter set can be omitted. This can reduce an increase in the code amount.
  • the operation-range extension information encoding device 100 encodes the operation-range extension flag, and encodes the operation-range extension tool control information in accordance with the operation-range extension flag and the bit depth of the image.
  • the operation-range extension flag encoding unit 111 encodes an operation-range extension flag (sps_range_extension_flag) in step S301 of Fig. 20 .
  • the operation-range extension tool control information encoding unit 112 may skip (omit) encoding the operation-range extension tool control information. Furthermore, in a case where the operation-range extension flag is true and the bit depth of the image is extended, the operation-range extension tool control information encoding unit 112 may encode the operation-range extension tool control information. For example, in a case where the operation-range extension flag is false or the bit depth of the image is not extended, the operation-range extension tool control information encoding unit 112 may further set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • the operation-range extension tool control information encoding unit 112 may further set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool.
  • step S302 the operation-range extension tool control information encoding unit 112 determines whether or not the operation-range extension flag is true and the bit depth of the image is greater than 10 bits. In a case where it is determined that the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the procedure proceeds to step S303. In step S303, the operation-range extension tool control information encoding unit 112 encodes the operation-range extension tool control information. When the processing in step S303 ends, the operation-range extension information encoding processing ends.
  • step S304 the operation-range extension tool control information encoding unit 112 skips (omits) encoding the operation-range extension tool control information. Furthermore, the operation-range extension tool control information encoding unit 112 sets the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, when the processing in step S303 ends, the operation-range extension information encoding processing ends.
  • the operation-range extension information encoding device 100 can prevent (the encoded data of) the operation-range extension tool control information from being stored in the bitstream in a case where the operation range is not extended. Therefore, the encoder and the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image. Furthermore, in this manner, in a case where the bit depth of the image is not extended (in a case where the bit depth is 10 bits or less), the transmission of the control flag of the encoding tool for high bit depth and high bit rate in the parameter set can be omitted. This can reduce an increase in the code amount.
  • Fig. 21 is a block diagram illustrating an example of the main configuration of the operation-range extension information decoding device 200 in that case. Also in this case, the operation-range extension information decoding device 200 performs processing related to the decoding of the encoded data of the operation-range extension information.
  • the operation-range extension information decoding device 200 acquires the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. Furthermore, the operation-range extension information decoding device 200 acquires the parameter (BitDepth) indicating the bit depth of the image.
  • the operation-range extension information maintains compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. That is, the bitstream of the image encoded using the operation-range extension information can be correctly decoded even by a decoder that does not support the extension of the operation range.
  • the operation-range extension information decoding device 200 decodes the encoded data of the operation-range extension information by using the parameter (BitDepth) indicating the bit depth of the image.
  • the operation-range extension information decoding device 200 appropriately outputs the operation-range extension information as a processing result.
  • the operation-range extension information is supplied to, for example, a decoder that decodes encoded data of an image.
  • the decoder decodes the encoded data on the basis of the operation-range extension information and generates (restores) an image.
  • the operation-range extension information decoding device 200 in this case has a configuration similar to the configuration described with reference to Fig. 10 . That is, the operation-range extension information decoding device 200 includes the operation-range extension flag decoding unit 211 and the operation-range extension tool control information decoding unit 212.
  • the operation-range extension flag decoding unit 211 performs processing related to the decoding of the encoded data of an operation-range extension flag (sps_range_extension_flag). For example, the operation-range extension flag decoding unit 211 may acquire the encoded data of the operation-range extension flag. Furthermore, the operation-range extension flag decoding unit 211 may decode the encoded data and generate (restore) the operation-range extension flag. Also, the operation-range extension flag decoding unit 211 may output the operation-range extension flag to the outside of the operation-range extension information decoding device 200 as a processing result. Furthermore, the operation-range extension flag decoding unit 211 may supply the operation-range extension flag to the operation-range extension tool control information decoding unit 212.
  • the operation-range extension tool control information decoding unit 212 performs processing related to the decoding of the encoded data of the operation-range extension tool control information (sps_range_extension()). For example, the operation-range extension tool control information decoding unit 212 may acquire the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may acquire the operation-range extension flag supplied from the operation-range extension flag decoding unit 211. Moreover, the operation-range extension tool control information decoding unit 212 may acquire the parameter (BitDepth) indicating the bit depth of the image.
  • the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag and the bit depth (BitDepth) of the image. For example, in a case where the operation-range extension flag is false or the bit depth of the image is not extended, the operation-range extension tool control information decoding unit 212 may skip (omit) decoding the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, in a case where the operation-range extension flag is true and the bit depth of the image is extended, the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information.
  • the operation-range extension tool control information decoding unit 212 may skip (omit) decoding the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, in a case where the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information.
  • the operation-range extension tool control information decoding unit 212 may output the operation-range extension tool control information to the outside of the operation-range extension information decoding device 200 as a processing result.
  • the operation-range extension tool control information may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the decoder may be set to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform.
  • the operation-range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of the TSRC.
  • the operation-range extension tool control information may include a control flag (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of the RRC.
  • a control flag sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag
  • the operation-range extension information decoding device 200 can correctly decode the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. That is, the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image. In addition, an increase in the code amount can be reduced.
  • the operation-range extension information decoding device 200 decodes the encoded data of the operation-range extension flag, and decodes the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag and the bit depth of the image.
  • step S331 of Fig. 22 the operation-range extension flag decoding unit 211 decodes the encoded data of the operation-range extension flag (sps_range_extension_flag) and generates (restores) the operation-range extension flag.
  • the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information in accordance with the operation-range extension flag and the bit depth (BitDepth) of the image. For example, in a case where the operation-range extension flag is false or the bit depth of the image is not extended, the operation-range extension tool control information decoding unit 212 may skip (omit) decoding the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, in a case where the operation-range extension flag is true and the bit depth of the image is extended, the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information.
  • the operation-range extension tool control information decoding unit 212 may skip (omit) decoding the encoded data of the operation-range extension tool control information. Furthermore, the operation-range extension tool control information decoding unit 212 may set the operation-range extension tool control information to a value indicating the unavailability of the operation-range extension tool. For example, in a case where the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the operation-range extension tool control information decoding unit 212 may decode the encoded data of the operation-range extension tool control information.
  • step S332 the operation-range extension tool control information decoding unit 212 determines whether or not the operation-range extension flag is true (e.g., "1") and the bit depth of the image is greater than 10 bits. In a case where it is determined that the operation-range extension flag is true and the bit depth of the image is greater than 10 bits, the procedure proceeds to step S333.
  • the operation-range extension flag e.g., "1”
  • step S333 the operation-range extension tool control information decoding unit 212 decodes the encoded data of the operation-range extension tool control information, and generates (restores) the operation-range extension tool control information.
  • the operation-range extension information decoding processing ends.
  • step S332 determines whether the operation-range extension flag is false or the bit depth of the image is 10 bits or less.
  • the procedure proceeds to step S334.
  • step S334 the operation-range extension tool control information decoding unit 212 omits the decoding of the encoded data of the operation-range extension tool control information. Then, the operation-range extension tool control information decoding unit 212 sets each control flag of the operation-range extension tool control information to a value indicating the unavailability of the tool.
  • the operation-range extension information decoding device 200 can correctly decode the encoded data of the operation-range extension information generated by the operation-range extension information encoding device 100. That is, the decoder can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend the bit depth of the image. In addition, an increase in the code amount can be reduced.
  • Fig. 23 is a block diagram illustrating an example of a configuration of an image encoding device that is one aspect of an image processing device to which the present technology is applied.
  • An image encoding device 300 illustrated in Fig. 23 is a device that encodes image data of a moving image.
  • the image encoding device 300 can encode image data of a moving image by an encoding method described in any one of the above-described Non-Patent Documents.
  • the image encoding device 300 may include a processing unit not illustrated as a block in Fig. 23 .
  • the image encoding device 300 may have processing or a data flow that is not illustrated as an arrow or the like in Fig. 23 .
  • the image encoding device 300 includes a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transform unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transform unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.
  • the control unit 301 divides moving image data held by the rearrangement buffer 311 into blocks (CU, PU, transform block, etc.) serving as units of processing on the basis of a block size in units of processing specified outside or in advance. Furthermore, the control unit 301 determines an encoding parameter (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block on the basis of, for example, rate-distortion optimization (RDO).
  • RDO rate-distortion optimization
  • control unit 301 supplies the encoding parameters to each block. Specifically, details are as follows.
  • the header information Hinfo is supplied to each block.
  • the prediction mode information Pinfo is supplied to the encoding unit 315 and the prediction unit 322.
  • the transform information Tinfo is supplied to the encoding unit 315, the orthogonal transform unit 313, the quantization unit 314, the inverse quantization unit 317, and the inverse orthogonal transform unit 318.
  • the filter information Finfo is supplied to the in-loop filter unit 320.
  • Each of fields (input images) of moving image data is input to the image encoding device 300 in order of regeneration (order of display) thereof.
  • the rearrangement buffer 311 acquires and holds (stores) each of the input images in order of reproduction (order of display) thereof.
  • the rearrangement buffer 311 rearranges the input images in order of encoding (order of decoding) or partitions the input images into blocks serving as units of processing on the basis of control by the control unit 301.
  • the rearrangement buffer 311 supplies each of the processed input images to the calculation unit 312. Furthermore, the rearrangement buffer 311 also supplies each of the input images (original images) to the prediction unit 322 and the in-loop filter unit 320.
  • the calculation unit 312 uses an image I corresponding to the block serving as the unit of processing and a predicted image P supplied from the prediction unit 322 as inputs, subtracts the predicted image P from the image I as shown in the following equation, and derives a prediction residual D, and supplies the prediction residual D to the orthogonal transform unit 313.
  • D I ⁇ P
  • the orthogonal transform unit 313 uses the prediction residual D supplied from the calculation unit 312 and the transform information Tinfo supplied from the control unit 301 as inputs, performs orthogonal transform on the prediction residual D on the basis of the transform information Tinfo, and derives a transform coefficient Coeff.
  • the orthogonal transform unit 313 performs primary transform on the prediction residual D to generate a primary transform coefficient, and performs secondary transform on the primary transform coefficient on the basis of an ST identifier to generate a secondary transform coefficient.
  • the orthogonal transform unit 313 supplies the obtained secondary transform coefficient to the quantization unit 314 as the transform coefficient Coeff.
  • the orthogonal transform unit 313 can perform an arbitrary coefficient transform, not limited to the orthogonal transform. That is, the transform coefficient Coeff may be derived by performing arbitrary coefficient transform on the prediction residual D. Therefore, the orthogonal transform unit 313 can also be referred to as a coefficient transform unit.
  • the quantization unit 314 uses the transform coefficient Coeff supplied from the orthogonal transform unit 313 and the transform information Tinfo supplied from the control unit 301 as inputs, and scales (quantizes) the transform coefficient Coeff on the basis of the transform information Tinfo. Note that the rate of the quantization is controlled by the rate control unit 323.
  • the quantization unit 314 supplies a quantization coefficient level (also referred to as a quantization coefficient qcoeff), which is a level value of the transform coefficient quantized in this manner, to the encoding unit 315 and the inverse quantization unit 317.
  • the encoding unit 315 uses, as inputs, the quantization coefficient level supplied from the quantization unit 314, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) supplied from the control unit 301, information regarding filtering, such as a filter coefficients, supplied from the in-loop filter unit 320, and information regarding the optimum prediction mode supplied from the prediction unit 322.
  • the encoding unit 315 performs variable-length encoding (e.g., arithmetic encoding) on the quantization coefficient level and generates a bit string (encoded data).
  • the encoding unit 315 includes, in the filter information Finfo, information regarding filtering supplied from the in-loop filter unit 320 and includes, in the prediction mode information Pinfo, information regarding the optimum prediction mode supplied from the prediction unit 322. Then, the encoding unit 315 encodes the above-described various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.), and generates a bit string.
  • the encoding unit 315 multiplexes the bit string (encoded data) of various information generated as described above and generates a bitstream of the encoded data.
  • the encoding unit 315 supplies the bitstream to the accumulation buffer 316.
  • the accumulation buffer 316 temporarily holds the bitstream of encoded data obtained in the encoding unit 315.
  • the accumulation buffer 316 outputs the held bitstream of the encoded data to the outside of the image encoding device 300 at a predetermined timing.
  • the bitstream is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, or the like. That is, the accumulation buffer 316 is also a transmission unit that transmits a bitstream (encoded data).
  • the inverse quantization unit 317 performs processing related to inverse quantization. For example, the inverse quantization unit 317 uses the quantization coefficient level supplied from the quantization unit 314 and the transform information Tinfo supplied from the control unit 301 as inputs, and scales (inversely quantizes) a value of the quantization coefficient level on the basis of the transform information Tinfo. Note that this inverse quantization is the inverse processing of the quantization performed in the quantization unit 314.
  • the inverse quantization unit 317 supplies a transform coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.
  • the inverse orthogonal transform unit 318 performs processing related to inverse orthogonal transform.
  • the inverse orthogonal transform unit 318 uses the transform coefficient Coeff_IQ supplied from the inverse quantization unit 317 and the transform information Tinfo supplied from the control unit 301 as inputs, performs inverse orthogonal transform on the transform coefficient Coeff_IQ on the basis of the transform information Tinfo, and derives a prediction residual D'.
  • this inverse orthogonal transform is the inverse processing of the orthogonal transform performed by the orthogonal transform unit 313.
  • the inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 319.
  • the inverse orthogonal transform unit 318 executes the inverse processing of the processing executed by the orthogonal transform unit 313. That is, similarly to the case of the orthogonal transform unit 313, the inverse orthogonal transform unit 318 can perform an arbitrary inverse coefficient transform, not limited to the inverse orthogonal transform.
  • This inverse coefficient transform is the inverse processing of the coefficient transform executed by the orthogonal transform unit 313. That is, the prediction residual D' may be derived by performing arbitrary inverse coefficient transform on the transform coefficient Coeff_IQ. Therefore, the inverse orthogonal transform unit 318 can also be referred to as an inverse coefficient transform unit.
  • the calculation unit 319 uses the prediction residual D' supplied from the inverse orthogonal transform unit 318 and the predicted image P supplied from the prediction unit 322 as inputs.
  • the calculation unit 319 adds the prediction residual D' and the predicted image P corresponding to the prediction residual D', and derives a locally decoded image Rlocal.
  • the calculation unit 319 supplies the derived locally decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.
  • the in-loop filter unit 320 performs processing related to in-loop filter processing.
  • the in-loop filter unit 320 uses the locally decoded image Rlocal supplied from the calculation unit 319, the filter information Finfo supplied from the control unit 301, and the input image (original image) supplied from the rearrangement buffer 311 as inputs.
  • the information to be input to the in-loop filter unit 320 is arbitrary, and information other than these pieces of information may be input. For example, a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, information of a block (CU, CTU, etc.), and the like may be input to the in-loop filter unit 320 as necessary.
  • the in-loop filter unit 320 appropriately performs filter processing on the locally decoded image Rlocal on the basis of the filter information Finfo.
  • the in-loop filter unit 320 also uses an input image (original image) or other input information for the filter processing as necessary.
  • the in-loop filter unit 320 applies four in-loop filters of a bilateral filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF) in this order. Note that which filters are applied in which order is arbitrary, and can be selected as appropriate.
  • DPF deblocking filter
  • SAO sample adaptive offset
  • ALF adaptive loop filter
  • the filter processing performed by the in-loop filter unit 320 is arbitrary and is not limited to the above example.
  • the in-loop filter unit 320 may apply a wiener filter or the like.
  • the in-loop filter unit 320 supplies the locally decoded image Rlocal subjected to the filter processing to the frame memory 321. Note that, for example, in a case where information regarding filtering such as filter coefficients are transmitted to the decoding side, the in-loop filter unit 320 supplies the information regarding filtering to the encoding unit 315.
  • the frame memory 321 performs processing related to storage of data related to an image. For example, the frame memory 321 uses the locally decoded image Rlocal supplied from the calculation unit 319 or the locally decoded image Rlocal subjected to the filter processing and supplied from the in-loop filter unit 320 as an input, and holds (stores) the locally decoded image Rlocal. Furthermore, the frame memory 321 reconstructs a decoded image R in each picture unit using the locally decoded image Rlocal, and holds (stores in the buffer in the frame memory 321) the decoded image R. The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.
  • the prediction unit 322 performs processing related to generation of a predicted image.
  • the prediction unit 322 uses the prediction mode information Pinfo supplied from the control unit 301, the input image (original image) supplied from the rearrangement buffer 311, and the decoded image R (or a part thereof) read from the frame memory 321 as inputs.
  • the prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction with reference to the decoded image R as a reference image, performs motion compensation processing on the basis of the prediction result, and generates the predicted image P.
  • the prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319.
  • the prediction unit 322 supplies information regarding the prediction mode selected by the above processing, that is, the optimum prediction mode, to the encoding unit 315 as necessary.
  • the rate control unit 323 performs processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 so that overflow or underflow does not occur on the basis of the code amount of the encoded data accumulated in the accumulation buffer 316.
  • each processing unit may be configured by a logic circuit that implements the above-described processing.
  • each processing unit may include a central processing unit (CPU), a read-only memory (ROM), a random-access memory (RAM), and the like, for example, and execute a program by using them to realize the above-described processing.
  • CPU central processing unit
  • ROM read-only memory
  • RAM random-access memory
  • each processing unit may have both the configurations, and a part of the above-described processing may be achieved by the logic circuit and another may be achieved by executing the program.
  • the configurations of the processing units may be independent from each other, and, for example, among the processing units, some processing units may achieve a part of the above-described processing with a logic circuit, some other processing units may achieve the above-described processing by executing a program, and still some other processing units may achieve the above-described processing with both a logic circuit and execution of a program.
  • Fig. 24 is a block diagram illustrating an example of the main configuration of the encoding unit 315.
  • the encoding unit 315 includes an operation-range extension information encoding unit 351.
  • the operation-range extension information encoding unit 351 executes processing related to encoding of the operation-range extension information.
  • the operation-range extension information encoding unit 351 may acquire the operation-range extension information supplied as an encoding parameter from the control unit 301.
  • the operation-range extension information encoding unit 351 may acquire the parameter (BitDepth) indicating the bit depth of the image supplied as an encoding parameter from the control unit 301.
  • the operation-range extension information encoding unit 351 may perform processing related to the encoding of the operation-range extension information by using these pieces of information.
  • the operation-range extension information encoding unit 351 may appropriately output the encoded data of the operation-range extension information and the like as a processing result.
  • the encoded data is stored in a bitstream including encoded data of an image in the encoding unit 315.
  • the bitstream is output to the outside of the image encoding device 300 via the accumulation buffer 316.
  • the bitstream is supplied to the decoding side.
  • the operation-range extension information encoded by the operation-range extension information encoding unit 351, the operation-range extension information set by the operation-range extension information encoding unit 351, or internal variables corresponding thereto are supplied as encoding parameters to an arbitrary processing unit of the image encoding device 300 and used.
  • the operation-range extension information may be supplied to the encoding unit 315 and used for encoding and the like. Furthermore, the operation-range extension information (or internal variable) may be supplied to the orthogonal transform unit 313 and used for orthogonal transform or the like. Also, the operation-range extension information (or internal variable) may be supplied to the inverse orthogonal transform unit 318 and used for inverse orthogonal transform or the like.
  • each of the above-described methods (Method 1, Method 2, or Method 3) of the present technology may be applied.
  • the operation-range extension information encoding unit 351 may have a configuration similar to that of the operation-range extension information encoding device 100 illustrated in Fig. 8 and execute the operation-range extension information encoding processing illustrated in Fig. 9 .
  • the image encoding device 300 can obtain the effect described above in ⁇ 3.
  • Control of operation-range extension flag> (the effect in the case of applying Method 1). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • the operation-range extension information encoding unit 351 may have a configuration similar to that of the operation-range extension information encoding device 100 illustrated in Fig. 16 and execute the operation-range extension information encoding processing illustrated in Fig. 17 .
  • the image encoding device 300 can obtain the effect described above in ⁇ 4.
  • Control of value of operation-range extension tool control information> (the effect in the case of applying Method 2). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • the operation-range extension information encoding unit 351 may have a configuration similar to that of the operation-range extension information encoding device 100 illustrated in Fig. 16 and execute the operation-range extension information encoding processing illustrated in Fig. 20 .
  • the image encoding device 300 can obtain the effect described above in ⁇ 5. Transmission control of operation-range extension tool control information> (the effect in the case of applying Method 3). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image. In addition, an increase in the code amount can be reduced.
  • step S401 the rearrangement buffer 311 is controlled by the control unit 301 to rearrange the order of the frames of the input moving image data from the order of display to the order of encoding.
  • step S402 the control unit 301 sets units of processing (performs block division) for the input image held by the rearrangement buffer 311.
  • step S403 the control unit 301 determines (sets) the encoding parameters for the input image held by the rearrangement buffer 311.
  • step S404 the prediction unit 322 performs prediction processing and generates a predicted image or the like in the optimum prediction mode. For example, in the prediction processing, the prediction unit 322 performs intra prediction to generate a predicted image or the like in the optimum intra prediction mode, performs inter prediction to generate a predicted image or the like in the optimum inter prediction mode, and selects the optimum prediction mode from the predicted image on the basis of a cost function value or the like.
  • step S405 the calculation unit 312 calculates a difference between the input image and the predicted image in the optimum mode selected by the prediction processing in step S304. That is, the calculation unit 312 generates the prediction residual D between the input image and the predicted image.
  • the prediction residual D obtained in this manner has a smaller amount of data than the original image data. Therefore, the amount of data can be compressed compared to a case where an image is encoded as it is.
  • step S406 the orthogonal transform unit 313 performs orthogonal transform processing on the prediction residual D generated by the processing in step S405, and derives the transform coefficient Coeff.
  • step S407 the quantization unit 314 quantizes the transform coefficient Coeff obtained by the processing in step S406 by using the quantization parameter calculated by the control unit 301 or the like, and derives the quantization coefficient level.
  • step S408 the inverse quantization unit 317 inversely quantizes the quantization coefficient level generated by the processing in step S407 with a characteristic corresponding to the quantization characteristic in step S407, and derives the transform coefficient Coeff_IQ.
  • step S409 the inverse orthogonal transform unit 318 inversely orthogonally transforms the transform coefficient Coeff_IQ obtained by the processing in step S408 by a method corresponding to the orthogonal transform processing in step S406, and derives the prediction residual D'.
  • step S410 the calculation unit 319 adds the predicted image obtained by the prediction processing in step S404 to the prediction residual D' derived by the processing in step S409, thereby generating a locally decoded image.
  • step S411 the in-loop filter unit 320 performs in-loop filter processing on the locally decoded image derived by the processing in step S410.
  • step S412 the frame memory 321 stores the locally decoded image derived by the processing of step S410 and the locally decoded image subjected to the filter processing in step S411.
  • step S413 the encoding unit 315 encodes the quantization coefficient level, various encoding parameters, and the like obtained by the processing in step S407, and generates a bitstream of the encoded data.
  • step S414 the accumulation buffer 316 accumulates the bitstream obtained in step S413 and outputs the bitstream to the outside of the image encoding device 300.
  • This bitstream is transmitted to the decoding side via a transmission path or a recording medium, for example.
  • the rate control unit 323 performs rate control as necessary.
  • step S414 ends, the image encoding processing ends.
  • step S413 of such image encoding processing each of the above-described methods (Method 1, Method 2, or Method 3) of the present technology may be applied.
  • step S413 the operation-range extension information encoding unit 351 of
  • the encoding unit 315 may execute the operation-range extension information encoding processing illustrated in Fig. 9 to encode the operation-range extension information (encoding parameter).
  • the image encoding device 300 can obtain the effect described above in ⁇ 3.
  • Control of operation-range extension flag> the effect in the case of applying Method 1). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • step S413 the operation-range extension information encoding unit 351 of
  • the encoding unit 315 may execute the operation-range extension information encoding processing illustrated in Fig. 17 to encode the operation-range extension information (encoding parameter).
  • the image encoding device 300 can obtain the effect described above in ⁇ 4.
  • Control of value of operation-range extension tool control information> the effect in the case of applying Method 2). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image.
  • the encoding unit 315 may execute the operation-range extension information encoding processing illustrated in Fig. 20 to encode the operation-range extension information (encoding parameter).
  • the image encoding device 300 can obtain the effect described above in ⁇ 5. Transmission control of operation-range extension tool control information> (the effect in the case of applying Method 3). That is, the image encoding device 300 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image. In addition, an increase in the code amount can be reduced.
  • Fig. 26 is a block diagram illustrating an example of a configuration of an image decoding device that is one aspect of an image processing device to which the present technology is applied.
  • the image decoding device 400 illustrated in Fig. 26 is a device that encodes encoded data of a moving image.
  • the image decoding device 400 can decode the encoded data by the decoding method described in any one of the above Non-Patent Documents.
  • the image decoding device 400 decodes the encoded data (bitstream) generated by the above-described image encoding device 300.
  • the image decoding device 400 may include a processing unit not illustrated as a block in Fig. 26 .
  • the image decoding device 400 may have processing or a data flow that is not illustrated as an arrow or the like in Fig. 26 .
  • the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transform unit 414, a calculation unit 415, an in-loop filter unit 416, a rearrangement buffer 417, a frame memory 418, and a prediction unit 419.
  • the prediction unit 419 includes an intra prediction unit, an inter prediction unit, and the like (not illustrated).
  • the image decoding device 400 is a device that generates moving image data by decoding a bitstream including encoded data of a moving image. For example, the image decoding device 400 can correctly decode a bitstream generated by the image encoding device 300.
  • the accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400.
  • the accumulation buffer 411 supplies the accumulated bitstream to the decoding unit 412 at a predetermined timing or in a case where a predetermined condition is satisfied.
  • the decoding unit 412 performs processing related to image decoding. For example, the decoding unit 412 uses the bitstream supplied from the accumulation buffer 411 as an input, performs variable-length decoding on the syntax value of each syntax element from the bit string according to the definition of the syntax table, and derives a parameter.
  • the syntax element and the parameter derived from the syntax value of the syntax element include, for example, header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, and the like. That is, the decoding unit 412 parses (analyzes and acquires) these pieces of information from the bitstream. These pieces of information will be described below.
  • the header information Hinfo includes, for example, header information such as video parameter set (VPS)/sequence parameter set (SPS)/Picture Parameter Set (PPS)/slice header (SH).
  • the header information Hinfo includes, for example, information defining an image size (width PicWidth, height PicHeight), a bit depth (luminance bitDepthY, color difference bitDepthC), a chroma array type ChromaArrayType, a maximum value MaxCUSize/minimum value MinCUSize of CU size, a maximum depth MaxQTDepth/a minimum depth MinQTDepth of quadtree partitioning (also referred to as Quad-tree partitioning), a maximum depth MaxBTDepth/minimum depth MinBTDepth of binary tree partitioning (Binary-tree partitioning), a maximum value MaxTSSize of a transform skip block (also referred to as a maximum transform skip block size), an on/off flag (also referred to as an enabled flag)
  • the on/off flag of the encoding tool included in the header information Hinfo there is an on/off flag related to the following transform and quantization processing.
  • the on/off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax related to the encoding tool exists in the encoded data.
  • the value of the on/off flag is one (true)
  • it indicates the availability of the encoding tool
  • the value of the on/off flag is zero (false)
  • a cross-component prediction enabled flag is flag information indicating whether or not cross-component prediction (cross-component prediction (CCP), also referred to as CC prediction) is available. For example, in a case where the flag information is "1" (true), it indicates the availability of the flag information, and in a case where the flag information is "0" (false), it indicates the unavailability of the flag information.
  • CCP cross-component linear prediction
  • CCLMP cross-component linear prediction
  • the SPS of) the header information Hinfo may include the above-described operation-range extension information.
  • the prediction mode information Pinfo includes, for example, information such as size information PBSize (predicted block size) of a processing target PB (predicted block), intra prediction mode information IPinfo, and motion prediction information MVinfo.
  • the intra-prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, a luminance intra-prediction mode IntraPredModeY derived from the syntax, and the like.
  • the intra-prediction mode information IPinfo includes, for example, a cross-component prediction flag (ccp_flag(cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chroma sample position type identifier (chroma_sample_loc_type_idx), a chroma MPM identifier (chroma_mpm_idx), a luminance intra-prediction mode (IntraPredModeC) derived from these syntaxes, and the like.
  • a cross-component prediction flag ccp_flag(cclmp_flag)
  • mclm_flag multi-class linear prediction mode flag
  • chroma sample position type identifier chroma_sample_loc_type_idx
  • chroma MPM identifier chroma_mpm_idx
  • luminance intra-prediction mode derived from these syntaxes, and the like.
  • the multi-class linear prediction mode flag is information regarding a mode of linear prediction (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether or not to set the multi-class linear prediction mode. For example, in the case of "0", a one-class mode (single-class mode) (e.g., CCLMP) is indicated, and in the case of "1", a two-class mode (multi-class mode) (e.g., MCLMP) is indicated.
  • a one-class mode single-class mode
  • multi-class mode e.g., MCLMP
  • the chrominance sample position type identifier (chroma_sample_loc_type_idx) is an identifier that identifies a type (also referred to as a chrominance sample position type) of a pixel position of the chrominance component. For example, in a case where the chrominance array type (ChromaArrayType), which is information regarding a color format, indicates 420 format, the chrominance sample position type identifier is assigned as in the following expression.
  • chroma_sample_loc_type_idx is transmitted as (stored in) information regarding a pixel position of a chroma component (chroma_sample_loc_info()).
  • the chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in a chroma intra-prediction mode candidate list (intraPredModeCandListC) is specified as a chroma intra-prediction mode.
  • the information included in the prediction mode information Pinfo is arbitrary, and information other than these pieces of information may be included.
  • the transform information Tinfo includes, for example, the following information. Needless to say, the information included in the transform information Tinfo is arbitrary, and information other than these pieces of information may be included.
  • log2TBWSize and log2TBHSize may be included in the transform information Tinfo instead of TBWSize and TBHSize.
  • log2TBWSize is a logarithmic value of TBWSize with base 2.
  • log2TBHSize is the logarithmic value of TBHSize with base 2.
  • the transform skip flag is a flag indicating whether or not to skip inverse coefficient transform (inverse primary transform and inverse secondary transform).
  • the filter information Finfo includes, for example, control information regarding each filter processing described below.
  • filter information Finfo is arbitrary, and information other than these pieces of information may be included.
  • the description returns to the description of the decoding unit 412.
  • the decoding unit 412 derives the quantization coefficient level with reference to syntax related to the quantization coefficient level obtained by decoding the bitstream.
  • the decoding unit 412 supplies the quantization coefficient level to the inverse quantization unit 413.
  • the decoding unit 412 supplies encoding parameters such as the parsed header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, and the like to each block.
  • the decoding unit 412 supplies the header information Hinfo to the inverse quantization unit 413, the inverse orthogonal transform unit 414, the prediction unit 419, and the in-loop filter unit 416.
  • the decoding unit 412 supplies the prediction mode information Pinfo to the inverse quantization unit 413 and the prediction unit 419.
  • the decoding unit 412 supplies the transform information Tinfo to the inverse quantization unit 413 and the inverse orthogonal transform unit 414.
  • the decoding unit 412 supplies the filter information Finfo to the in-loop filter unit 416.
  • each encoding parameter may be supplied to an arbitrary processing unit.
  • other information may be supplied to an arbitrary processing unit.
  • the inverse quantization unit 413 has at least a configuration necessary for performing processing related to inverse quantization. For example, the inverse quantization unit 413 uses the transform information Tinfo and the quantization coefficient level supplied from the decoding unit 412 as inputs, scales (inversely quantizes) the value of the quantization coefficient level on the basis of the transform information Tinfo, and derives the transform coefficient Coeff_IQ after inverse quantization. The inverse quantization unit 413 supplies the derived transform coefficient Coeff_IQ to the inverse orthogonal transform unit 414.
  • this inverse quantization is performed as the inverse processing of the quantization by the quantization unit 314 of the image encoding device 300. Furthermore, this inverse quantization is processing similar to the inverse quantization by the inverse quantization unit 317 of the image encoding device 300. That is, the inverse quantization unit 317 of the image encoding device 300 performs processing (inverse quantization) similar to the inverse quantization unit 413.
  • the inverse orthogonal transform unit 414 performs processing related to inverse orthogonal transform.
  • the inverse orthogonal transform unit 414 uses the transform coefficient Coeff_IQ supplied from the inverse quantization unit 413 and the transform information Tinfo supplied from the decoding unit 412 as inputs, performs inverse orthogonal transform processing on the transform coefficient Coeff_IQ on the basis of the transform information Tinfo, and derives the prediction residual D'.
  • the inverse orthogonal transform unit 414 performs inverse secondary transform on the transform coefficient Coeff_IQ on the basis of an ST identifier to generate a primary transform coefficient, and performs primary transform on the primary transform coefficient to generate the prediction residual D'.
  • the inverse orthogonal transform unit 414 supplies the derived prediction residual D' to the calculation unit 415.
  • this inverse orthogonal transform is performed as the inverse processing of the orthogonal transform by the orthogonal transform unit 313 of the image encoding device 300. Furthermore, this inverse orthogonal transform is processing similar to the inverse orthogonal transform by the inverse orthogonal transform unit 318 of the image encoding device 300. That is, the inverse orthogonal transform unit 318 of the image encoding device 300 performs processing (inverse orthogonal transform) similar to the inverse orthogonal transform unit 414.
  • the inverse orthogonal transform unit 414 can perform an arbitrary inverse coefficient transform, not limited to the inverse orthogonal transform.
  • This inverse coefficient transform is the inverse processing of the coefficient transform executed by the orthogonal transform unit 313 of the image encoding device 300. That is, the prediction residual D' may be derived by performing arbitrary inverse coefficient transform on the transform coefficient Coeff_IQ. Therefore, the inverse orthogonal transform unit 414 can also be referred to as an inverse coefficient transform unit.
  • the in-loop filter unit 416 performs processing related to in-loop filter processing.
  • the in-loop filter unit 416 uses the locally decoded image Rlocal supplied from the calculation unit 415 and the filter information Finfo supplied from the decoding unit 412 as inputs.
  • the information to be input to the in-loop filter unit 416 is arbitrary, and information other than these pieces of information may be input.
  • the in-loop filter unit 416 appropriately performs filter processing on the locally decoded image Rlocal on the basis of the filter information Finfo.
  • the in-loop filter unit 416 can apply four in-loop filters of a bilateral filter, a deblocking filter (DBF), an adaptive offset filter (sample adaptive offset (SAO)), and an adaptive loop filter (ALF) in this order. Note that which filters are applied in which order is arbitrary, and can be selected as appropriate.
  • DPF deblocking filter
  • SAO sample adaptive offset
  • ALF adaptive loop filter
  • the in-loop filter unit 416 performs filter processing corresponding to the filter processing performed by the encoding side (e.g., the in-loop filter unit 320 of the image encoding device 300). Needless to say, the filter processing performed by the in-loop filter unit 416 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 416 may apply a wiener filter, or the like.
  • the in-loop filter unit 416 supplies the locally decoded image Rlocal subjected to the filter processing to the rearrangement buffer 417 and the frame memory 418.
  • the rearrangement buffer 417 uses the locally decoded image Rlocal supplied from the in-loop filter unit 416 as an input, and holds (stores) the locally decoded image Rlocal.
  • the rearrangement buffer 417 reconstructs the decoded image R in each picture unit using the locally decoded image Rlocal, and holds (stores in the buffer) the decoded image R.
  • the rearrangement buffer 417 rearranges the obtained decoded images R from the order of decoding to the order of reproduction.
  • the rearrangement buffer 417 outputs a group of the rearranged decoded images R to the outside of the image decoding device 400 as moving image data.
  • the frame memory 418 performs processing related to storage of data related to an image. For example, the frame memory 418 uses the locally decoded image Rlocal supplied from the calculation unit 415 as an input, reconstructs the decoded image R in each picture unit, and stores the decoded image R in the buffer in the frame memory 418.
  • the frame memory 418 uses the locally decoded image Rlocal subjected to the in-loop filter processing and supplied from the in-loop filter unit 416 as an input, reconstructs the decoded image R in each picture unit, and stores the decoded image R in the buffer in the frame memory 418.
  • the frame memory 418 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 419 as a reference image.
  • the frame memory 418 may store the header information Hinfo, the prediction mode information Pinfo, the transform information Tinfo, the filter information Finfo, and the like related to the generation of the decoded image.
  • the prediction unit 419 performs processing related to generation of a predicted image. For example, the prediction unit 419 uses the prediction mode information Pinfo supplied from the decoding unit 412 as an input, performs prediction by a prediction method specified by the prediction mode information Pinfo, and derives the predicted image P. During the derivation, the prediction unit 419 uses the decoded image R (or a part thereof) before filtering or after filtering stored in the frame memory 418, specified by the prediction mode information Pinfo, as a reference image. The prediction unit 419 supplies the derived predicted image P to the calculation unit 415.
  • the prediction mode information Pinfo supplied from the decoding unit 412 as an input
  • the prediction unit 419 uses the prediction mode information Pinfo supplied from the decoding unit 412 as an input, performs prediction by a prediction method specified by the prediction mode information Pinfo, and derives the predicted image P.
  • the prediction unit 419 uses the decoded image R (or a part thereof) before filtering or after filtering stored in the frame memory 418, specified by the prediction
  • each processing unit may be configured by a logic circuit that implements the above-described processing.
  • each processing unit may have, for example, a CPU, a ROM, a RAM, and the like, and execute a program by using the CPU, the ROM, the R_AM, and the like to achieve the above-described processing.
  • each processing unit may have both the configurations, and a part of the above-described processing may be achieved by the logic circuit and another may be achieved by executing the program.
  • the configurations of the processing units may be independent from each other, and, for example, among the processing units, some processing units may achieve a part of the above-described processing with a logic circuit, some other processing units may achieve the above-described processing by executing a program, and still some other processing units may achieve the above-described processing with both a logic circuit and execution of a program.
  • Fig. 27 is a block diagram illustrating an example of the main configuration of the decoding unit 412.
  • the decoding unit 412 includes an operation-range extension information decoding unit 451.
  • the operation-range extension information decoding unit 451 executes processing related to the decoding of the encoded data of the operation-range extension information.
  • the operation-range extension information decoding unit 451 may acquire the operation-range extension information extracted from the bitstream by the decoding unit 412.
  • the operation-range extension information decoding unit 451 may acquire the parameter (BitDepth) indicating the bit depth of the image extracted from the bitstream by the decoding unit 412.
  • the operation-range extension information decoding unit 451 may perform processing related to the decoding of the encoded data of the operation-range extension information by using these pieces of information.
  • the operation-range extension information decoding unit 451 may appropriately output the operation-range extension information and the like as a processing result.
  • the operation-range extension information or an internal variable corresponding thereto is supplied to an arbitrary processing unit of the image decoding device 400 as an encoding parameter and used.
  • the operation-range extension information (or internal variable) may be supplied to the decoding unit 412 and used for decoding or the like.
  • the operation-range extension information (or internal variable) may be supplied to the inverse orthogonal transform unit 414 and used for inverse orthogonal transform or the like.
  • each of the above-described methods (Method 1-1, Method 2-1, or Method 3) of the present technology may be applied.
  • the operation-range extension information decoding unit 451 may have a configuration similar to that of the operation-range extension information decoding device 200 illustrated in Fig. 12 and execute the operation-range extension information decoding processing illustrated in Fig. 13 .
  • the image decoding device 400 can obtain the effect described above in ⁇ 3.
  • Control of operation-range extension flag> (the effect in the case of applying Method 1-1). That is, the image decoding device 400 can prevent the decoding of an invalid bitstream. As a result, the image decoding device 400 can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • the operation-range extension information decoding unit 451 may have a configuration similar to that of the operation-range extension information decoding device 200 illustrated in Fig. 12 and execute the operation-range extension information decoding processing illustrated in Fig. 18 .
  • the image decoding device 400 can obtain the effect described above in ⁇ 4.
  • Control of value of operation-range extension tool control information> (the effect in the case of applying Method 2-1). That is, the image decoding device 400 can prevent the decoding of an invalid bitstream. As a result, the image decoding device 400 can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • the operation-range extension information decoding unit 451 may have a configuration similar to that of the operation-range extension information decoding device 200 illustrated in Fig. 21 and execute the operation-range extension information decoding processing illustrated in Fig. 22 .
  • the image decoding device 400 can obtain the effect described above in ⁇ 5. Transmission control of operation-range extension tool control information> (the effect in the case of applying Method 3). That is, the image decoding device 400 can correctly decode the bitstream generated by the image encoding device 300. That is, the image decoding device 400 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image. In addition, an increase in the code amount can be reduced.
  • step S501 the accumulation buffer 411 acquires and holds (accumulates) a bitstream (coded data) supplied from outside of the image decoding device 400.
  • step S502 the decoding unit 412 executes decoding processing to decode the encoded data (bitstream) to obtain the quantization coefficient level. Furthermore, by perform this decoding, the decoding unit 412 parses (analyzes and acquires) various encoding parameters from the encoded data (bitstream).
  • step S503 the inverse quantization unit 413 performs inverse quantization, which is the inverse processing of the quantization performed on the encoding side, on the quantization coefficient level obtained by the processing in step S502 to obtain the transform coefficient Coeff_IQ.
  • step S504 the inverse orthogonal transform unit 414 performs an inverse orthogonal transform processing, which is the inverse processing of the orthogonal transform processing performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S503 to obtain the prediction residual D'.
  • step S505 the prediction unit 419 executes prediction processing by a prediction method designated by the encoding side on the basis of the information parsed in step S502, and generates the predicted image P with reference to the reference image stored in the frame memory 418 or the like.
  • step S506 the calculation unit 415 adds the prediction residual D' obtained in step S504 and the predicted image P obtained in step S505, and derives the locally decoded image Rlocal.
  • step S507 the in-loop filter unit 416 performs in-loop filter processing on the locally decoded image Rlocal obtained by the processing in step S506.
  • step S508 the rearrangement buffer 417 derives the decoded image R using the locally decoded image Rlocal subjected to the filter processing, obtained by the processing in step S507, and rearranges the order of the group of the decoded images R from the order of decoding to the order of reproduction.
  • the group of the decoded images R rearranged in the order of reproduction is output as a moving image to the outside of the image decoding device 400.
  • the frame memory 418 stores at least one of the locally decoded image Rlocal obtained by the processing in step S506 or the locally decoded image Rlocal after the filter processing obtained by the processing in step S507.
  • step S509 When the processing of step S509 ends, the image decoding processing ends.
  • step S502 of such an image decoding processing each of the above-described methods (Method 1-1, Method 2-1, or Method 3) of the present technology may be applied.
  • step S502 the operation-range extension information decoding unit 451 of the decoding unit 412 may execute the operation-range extension information decoding processing illustrated in Fig. 13 to decode the encoded data of the operation-range extension information (encoding parameter).
  • the image decoding device 400 can obtain the effect described above in ⁇ 3.
  • Control of operation-range extension flag> the effect in the case of applying Method 1-1). That is, the image decoding device 400 can prevent the decoding of an invalid bitstream. As a result, the image decoding device 400 can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • step S502 the operation-range extension information decoding unit 451 of
  • the decoding unit 412 may execute the operation-range extension information decoding processing illustrated in Fig. 18 .
  • the image decoding device 400 can obtain the effect described above in ⁇ 4.
  • Control of value of operation-range extension tool control information> the effect in the case of applying Method 2-1). That is, the image decoding device 400 can prevent the decoding of an invalid bitstream. As a result, the image decoding device 400 can decode the bitstream more safely (prevent the occurrence of failure, etc.).
  • step S502 the operation-range extension information decoding unit 451 of
  • the decoding unit 412 may execute the operation-range extension information decoding processing illustrated in Fig. 22 .
  • the image decoding device 400 can obtain the effect described above in ⁇ 5. Transmission control of operation-range extension tool control information> (the effect in the case of applying Method 3). That is, the image decoding device 400 can correctly decode the bitstream generated by the image encoding device 300. That is, the image decoding device 400 can use the encoding tool for bit-depth extension while maintaining compatibility with the specification that does not extend a bit depth of an image. In addition, an increase in the code amount can be reduced.
  • BitDepth is used as the information indicating the bit depth of the image, but the information for obtaining the bit depth of the image is arbitrary and is not limited to this example.
  • sps_bitdepth_minus8 which is a parameter indicating "Bit depth -8"
  • This parameter is stored in the sequence parameter set.
  • the operation-range extension information (operation-range extension flag, operation-range extension tool control information, etc.) is stored in the sequence parameter set.
  • the storage location of the operation-range extension information is arbitrary and is not limited to this example.
  • the operation-range extension information may be stored in, for example, a picture parameter set (PPS) or a slice header (slice_header).
  • the above-described series of processes can be executed by hardware or software.
  • a program constituting the software is installed in a computer.
  • the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like, for example.
  • Fig. 29 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processes by a program.
  • a central processing unit (CPU) 901, a read-only memory (ROM) 902, and a random-access memory (RAM) 903 are interconnected via a bus 904.
  • an input/output interface 910 is also connected to the bus 904.
  • An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input/output interface 910.
  • the input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like.
  • the output unit 912 includes, for example, a display, a speaker, an output terminal, and the like.
  • the storage unit 913 includes, for example, a hard disk, a R_AM disk, a nonvolatile memory, and the like.
  • the communication unit 914 includes a network interface, for example.
  • the drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the CPU 901 loads a program stored in the storage unit 913 into the RAM 903 via the input/output interface 910 and the bus 904 and executes the program, whereby the above-described series of processes is performed. Furthermore, the RAM 903 also appropriately stores data and the like necessary for the CPU 901 to execute various processes.
  • the program executed by the computer can be applied by being recorded on, for example, the removable medium 921 as a package medium or the like.
  • the program can be installed in the storage unit 913 via the input/output interface 910 by attaching the removable medium 921 to the drive 915.
  • this program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be received by the communication unit 914 and installed in the storage unit 913.
  • this program can be installed in the ROM 902 or the storage unit 913 in advance.
  • the present technology can be applied to any image encoding system or decoding system. That is, the specifications of various processes related to the image encoding and decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction are arbitrary as long as they do not contradict the present technology described above, and are not limited to the above-described examples. Furthermore, some of these processes may be omitted unless there is a contradiction with the present technology described above.
  • the present technology can be applied to a multi-view image encoding system that encodes a multi-view image including images of a plurality of viewpoints (views). Also, the present technology can be applied to a multi-view image decoding system that decodes encoded data of a multi-view image including images of a plurality of viewpoints (views). In that case, the present technology is only required to be applied to encoding and decoding of each viewpoint (view).is only required to
  • the present technology can be applied to a hierarchical image encoding (scalable encoding) system that encodes a hierarchical image layered (hierarchized) so as to have a scalability function for a predetermined parameter.
  • the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes encoded data of a hierarchical image layered (hierarchized) so as to have a scalability function for a predetermined parameter.
  • the present technology is only required to be applied to encoding and decoding of each layer (layer).
  • the present technology can be applied to any configuration.
  • the present technology can be applied to various electronic devices such as a transmitter and a receiver (e.g., a television receiver and a mobile phone) in satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to a terminal by cellular communication, or a device (e.g., a hard disk recorder and a camera) that records an image on a medium such as an optical disk, a magnetic disk, and a flash memory, or reproduces an image from the storage medium.
  • a transmitter and a receiver e.g., a television receiver and a mobile phone
  • cable broadcasting such as cable TV
  • distribution on the Internet distribution to a terminal by cellular communication
  • a device e.g., a hard disk recorder and a camera
  • records an image on a medium such as an optical disk, a magnetic disk, and a flash memory, or reproduces an image from the storage medium.
  • the present technology can also be implemented as a partial configuration of a device, such as a processor (e.g., a video processor) as a system large-scale integration (LSI) or the like, a module (e.g., a video module) using a plurality of processors or the like, a unit (e.g., a video unit) using a plurality of modules or the like, or a set (e.g., a video set) obtained by further adding other functions to a unit.
  • a processor e.g., a video processor
  • LSI system large-scale integration
  • a module e.g., a video module
  • a unit e.g., a video unit
  • a set e.g., a video set
  • the present technology can also be applied to a network system including a plurality of devices.
  • the present technology may be implemented as cloud computing shared and processed in cooperation by a plurality of devices via a network.
  • the present technology may be implemented in a cloud service that provides a service related to an image (moving image) to any terminal such as a computer, an audiovisual (AV) device, a portable information processing terminal, or an Internet of Things (IoT) device.
  • AV audiovisual
  • IoT Internet of Things
  • a system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device in which a plurality of modules is housed in one housing are all systems.
  • the system, device, processing unit and the like to which the present technology is applied may be used in arbitrary fields such as traffic, medical care, crime prevention, agriculture, livestock industry, mining, beauty care, factory, household appliance, weather, and natural surveillance, for example. Furthermore, any application thereof may be used.
  • the present technology can be applied to systems and devices used for providing content for appreciation and the like.
  • the present technology can also be applied to systems and devices used for traffic, such as traffic condition management and automated driving control.
  • the present technology can also be applied to systems and devices used for security.
  • the present technology can be applied to systems and devices used for automatic control of a machine or the like.
  • the present technology can also be applied to systems and devices provided for use in agriculture and livestock industry.
  • the present technology can also be applied to systems and devices that monitor, for example, the status of nature such as a volcano, a forest, and the ocean, wildlife, and the like.
  • the present technology can also be applied to systems and devices used for sports.
  • the "flag” is information for identifying a plurality of states, and includes not only information used for identifying two states of true (1) and false (0) but also information capable of identifying three or more states. Therefore, the value that may be taken by the "flag” may be, for example, a binary of 1/0 or a ternary or more. That is, the number of bits forming this "flag” is arbitrary, and may be one bit or a plurality of bits.
  • identification information (including the flag) is assumed to include not only identification information thereof in a bitstream but also difference information of the identification information with respect to a certain reference information in the bitstream, and thus, in the present description, the "flag” and “identification information” include not only the information thereof but also the difference information with respect to the reference information.
  • various information may be transmitted or recorded in any form as long as it is associated with the encoded data.
  • the term "associate" means, for example, that one piece of data can be used (linked) when the other piece of data is processed. That is, the data associated with each other may be collected as one piece of data or may be made individual data.
  • information associated with the encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image).
  • the information associated with the encoded data (image) may be recorded in a recording medium different from that of the encoded data (image) (or another recording area of the same recording medium).
  • the "association" may be for a part of information instead of the entire information.
  • an image and information corresponding to the image may be associated with each other in any unit such as a plurality of frames, one frame, or a part within a frame.
  • a configuration described as one device may be divided and configured as a plurality of devices (or processing units).
  • configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit).
  • a configuration other than the above-described configurations may be added to the configuration of each device (or each processing unit).
  • a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit).
  • the above-described program may be executed in any device.
  • the device is only required to have a necessary function (functional block, etc.) and obtain necessary information.
  • each step of one flowchart may be executed by one device, or may be shared and executed by a plurality of devices.
  • the plurality of processes may be executed by one device, or may be shared and executed by a plurality of devices.
  • a plurality of processes included in one step can also be executed as processes of a plurality of steps.
  • the processes described as a plurality of steps can be collectively executed as one step.
  • the program executed by the computer may have the following features.
  • the processes of the steps describing the program may be executed in time series in the order described in the present specification.
  • the processes of the steps describing the program may be executed in parallel.
  • the processes of the steps describing the program may be individually executed at the necessary timing, such as when the program is called. That is, the respective processes of the steps may be executed in an order different from the above-described order as long as there is no contradiction.
  • processes of steps describing this program may be executed in parallel with processes of another program.
  • the processes of the steps describing this program may be executed in combination with the processes of another program.
  • a plurality of techniques related to the present technology can be implemented independently as a single body as long as there is no contradiction.
  • any two or more of the present technologies can also be used and implemented in combination.
  • a part or all of the present technologies described in any of the embodiments can be implemented in combination with a part or all of the present technologies described in other embodiments.
  • a part or all of any of the present technologies described above can be implemented using together with another technology that is not described above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
EP22816141.0A 2021-06-03 2022-06-01 Dispositif et procédé de traitement d'image Pending EP4351143A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163196283P 2021-06-03 2021-06-03
PCT/JP2022/022248 WO2022255395A1 (fr) 2021-06-03 2022-06-01 Dispositif et procédé de traitement d'image

Publications (2)

Publication Number Publication Date
EP4351143A1 true EP4351143A1 (fr) 2024-04-10
EP4351143A4 EP4351143A4 (fr) 2024-05-22

Family

ID=84323423

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22816141.0A Pending EP4351143A4 (fr) 2021-06-03 2022-06-01 Dispositif et procédé de traitement d'image

Country Status (6)

Country Link
EP (1) EP4351143A4 (fr)
JP (1) JPWO2022255395A1 (fr)
KR (1) KR20240018432A (fr)
CN (1) CN117413522A (fr)
MX (1) MX2023014425A (fr)
WO (1) WO2022255395A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240298011A1 (en) * 2021-06-24 2024-09-05 Interdigital Ce Patent Holdings, Sas Method and apparatus for video encoding and decoding

Also Published As

Publication number Publication date
JPWO2022255395A1 (fr) 2022-12-08
WO2022255395A1 (fr) 2022-12-08
CN117413522A (zh) 2024-01-16
MX2023014425A (es) 2023-12-15
KR20240018432A (ko) 2024-02-13
EP4351143A4 (fr) 2024-05-22

Similar Documents

Publication Publication Date Title
EP4068778A1 (fr) Dispositif et procédé de traitement d'image
JP7235030B2 (ja) 画像処理装置および方法
JP2024105675A (ja) 画像処理装置および方法
JP7235031B2 (ja) 画像処理装置および方法
US20210006836A1 (en) Image encoding apparatus, image encoding method, image decoding apparatus, and image decoding method
EP4351143A1 (fr) Dispositif et procédé de traitement d'image
US20210392354A1 (en) Image processing apparatus and method
US20210385456A1 (en) Image processing apparatus and method
EP4060995A1 (fr) Dispositif de traitement d'images, procédé de génération de flux de bits, procédé de génération de données de coefficient et procédé de génération de coefficient de quantification
EP4050896B1 (fr) Dispositif et procédé de traitement d'image
EP4412215A1 (fr) Dispositif et procédé de traitement d'image
EP4087242A1 (fr) Dispositif et procédé de traitement d'image
EP4362460A1 (fr) Dispositif et procédé de traitement d'image
US20240357129A1 (en) Image processing apparatus and method
EP4013055A1 (fr) Dispositif et procédé de traitement d'image
US20240163437A1 (en) Image processing device and method
US20220021899A1 (en) Image encoding apparatus, image encoding method, image decoding apparatus, and image decoding method

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20231221

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

A4 Supplementary search report drawn up and despatched

Effective date: 20240423

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/91 20140101ALI20240417BHEP

Ipc: H04N 19/60 20140101ALI20240417BHEP

Ipc: H04N 19/70 20140101AFI20240417BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)