WO2023274044A1 - 用于视频数据的编码方法、解码方法、计算设备和介质 - Google Patents

用于视频数据的编码方法、解码方法、计算设备和介质 Download PDF

Info

Publication number
WO2023274044A1
WO2023274044A1 PCT/CN2022/100950 CN2022100950W WO2023274044A1 WO 2023274044 A1 WO2023274044 A1 WO 2023274044A1 CN 2022100950 W CN2022100950 W CN 2022100950W WO 2023274044 A1 WO2023274044 A1 WO 2023274044A1
Authority
WO
WIPO (PCT)
Prior art keywords
bit
depth
information
picture
inverse transform
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/100950
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
张乾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Beijing BOE Technology Development Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Beijing BOE Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd, Beijing BOE Technology Development Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to EP22831858.0A priority Critical patent/EP4366305A4/en
Priority to KR1020247002827A priority patent/KR20240026202A/ko
Priority to JP2023580579A priority patent/JP2024524397A/ja
Priority to US18/574,357 priority patent/US20240340452A1/en
Publication of WO2023274044A1 publication Critical patent/WO2023274044A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/67—Circuits for processing colour signals for matrixing

Definitions

  • Embodiments of the present disclosure relate to an encoding method, a decoding method, a computing device, and a medium for video data.
  • Digital video capabilities can be incorporated into a wide variety of devices including digital television, digital live broadcast systems, over-the-air broadcast systems, laptop or desktop computers, tablet computers, e-readers, digital cameras, digital recording devices, digital media players , video game equipment, video game consoles, smartphones, video teleconferencing equipment, and video streaming equipment, among others.
  • Digital video equipment can implement video codec technologies such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC), those video codec technologies described in the standard defined by ITU-T H.265/High Efficiency Video Codec, and extensions to such standards.
  • video devices can more efficiently send, receive, encode, decode and/or store digital video information.
  • Some embodiments of the present disclosure provide an encoding method for video data, a decoding method, a computing device, and a medium for flexibly processing a bit depth of a picture in a codec.
  • a coding method for video data including: performing bit-depth transformation processing on a picture, for generating a transformed picture and generating bit-depth transformation information, wherein the bit-depth transformation information indicates the same Information associated with the bit-depth transformation processing of the picture, the picture being one frame of pictures in the video data; and encoding the transformed picture to generate coding information corresponding to the transformed picture, wherein the bit-depth transformation information and The encoded information is used to form a bitstream.
  • the bit-depth transformation process is bit-depth compression processing
  • performing bit-depth transformation processing on the picture includes: performing bit-depth compression on the picture using a bit-depth compression method to reduce the bit-depth of the picture, wherein , the bit depth transformation information includes bit depth compression control information, wherein the bit depth compression control information indicates information associated with bit depth compression processing.
  • the bit-depth compression control information includes at least one of the following: information indicating the bit-depth compression method, information indicating the initial bit-depth before the bit-depth compression process, indicating the compression after the bit-depth compression process bit depth information.
  • the encoding method further includes: performing decoding processing on the encoded information to generate a decoded picture; and performing bit-depth inverse transform processing on the decoded picture to generate an inverse-transformed picture and generate a bit-depth inverse transform information, wherein the bit-depth inverse transform process and the bit-depth transform process are inverse operations, and the bit-depth inverse transform information indicates information associated with the bit-depth inverse transform process performed on the decoded picture.
  • the encoding method further includes: comparing the inverse transformed picture with the picture to perform effect verification for generating bit depth verification effect information, wherein the bit depth verification effect information includes the following At least one: verification control information indicating whether to perform effect verification, information indicating an effect verification method for effect verification, and information indicating the verification result of the effect verification method.
  • the bit depth inverse transform information includes bit depth inverse transform control information, wherein the bit depth inverse transform control information indicates information associated with bit depth inverse transform processing.
  • the bit-depth inverse transform control information includes bit-depth inverse transform switch information, and indicates whether to perform bit-depth inverse transform processing on the decoded picture.
  • performing bit-depth inverse transform processing on the decoded picture includes: performing bit-depth inverse transform processing in response to the bit-depth inverse transform switch information indication, performing bit-depth inverse transform on the decoded picture using a bit-depth inverse transform method, Used to change the bit depth of a decoded picture in the opposite direction of the bit depth transform.
  • the bit depth inverse transform control information further includes at least one of the following: information indicating the bit depth inverse transform method, information indicating the input bit depth before the bit depth inverse transform process, indicating the bit depth inverse Output bit depth information after transform processing.
  • the bit depth transform process is bit depth compression process
  • the bit depth inverse transform process is bit depth expansion process
  • the bitstream includes bit-depth transformation extension bits for conveying bit-depth transformation information in the bitstream.
  • a decoding method for video data including: parsing bit-depth transformation information and encoding information corresponding to a picture from a received bitstream, wherein the bit-depth transformation information indicates the same Information associated with bit-depth transformation processing performed in the process of encoding a picture, the picture being one frame of pictures in the video data; decoding and conversion processing is performed according to the bit-depth transformation information and encoding information, and used to generate a display picture.
  • the bit-depth transformation process is a bit-depth compression process
  • the bit-depth transformation information includes bit-depth compression control information indicating that it is associated with the bit-depth compression process, wherein decoding is performed according to the bit-depth transformation information and the encoding information
  • the conversion processing includes: performing decoding conversion processing on the coded information with reference to the bit-depth compression control information.
  • the decoding method further includes: parsing bit-depth inverse transform information corresponding to the picture from the bit stream, wherein the bit-depth inverse transform information indicates the The inverse transform processes the associated information.
  • performing decoding conversion processing according to the bit-depth transformation information and the coding information includes: performing decoding processing on the coding information to generate a decoded picture corresponding to the picture; referring to the bit-depth inverse transformation information to bit-code the decoded picture Depth inverse transform processing for generating display images.
  • the bit-depth transformation processing is bit-depth compression processing
  • the bit-depth inverse transformation processing is bit-depth extension processing
  • the bit-depth inverse transformation information includes bit-depth extension control information
  • the bit-depth extension control information includes indication Information about the bit depth extension method
  • the decoding method further includes: determining whether the bit depth extension method is available; if it is determined that the bit depth extension method is available, extending the bit depth according to the bit depth extension method; and determining the bit depth extension method If not available, receive a bit-depth extension method for performing bit-depth extension on the decoded picture according to the bit-depth extension method.
  • the decoding method before performing bit-depth inverse transform processing on the decoded picture, the decoding method further includes: receiving control information indicating whether to perform bit-depth inverse transform processing, and referring to the control information to determine whether to perform bit-depth inverse transform processing on the decoded picture Depth inverse transform processing, wherein the control information is generated based on at least one of the following: the computing power of the decoder, the power information of the decoder, the bit depth display requirements of the display; or, parsing the corresponding image from the bit stream bit depth verification effect information; comparing the decoded picture with the bit depth extension effect information; and judging whether to perform bit depth inverse transformation processing on the decoded picture with reference to the comparison result.
  • parsing the bit-depth transformation information from the received bitstream includes: parsing the bit-depth transformation information from bit-depth transformation extension bits in the bitstream.
  • a computing device comprising: a processor, and a non-transitory memory having computer readable code thereon, wherein the computer readable code, when executed by the processor, executes the above The encoding method for video data described above, or perform the decoding method for video data as described above.
  • a computer-readable storage medium on which instructions are stored, wherein, when the instructions are executed by a processor, the processor executes the above-mentioned encoding method for video data , or perform the decoding method for video data as described above.
  • bit-depth transformation processing it is possible to perform bit-depth transformation processing on a picture before the encoding process, and then encode the generated transformed picture to form the picture
  • bit-depth transformation information is generated, and bit-depth transformation information and coding information together form a bit stream, so that the decoder can process correspondingly according to the bit-depth transformation information parsed in the bit stream
  • the bit depth transformation information transmitted in the bit stream is conducive to a more flexible implementation of the bit depth transformation process, and at the same time can realize the information communication about the bit depth transformation between the decoding end and the encoding end.
  • FIG. 1 is a block diagram illustrating an example video codec system capable of performing a codec method according to some embodiments of the present disclosure
  • Figure 2 is a block diagram illustrating an example video encoder according to some embodiments of the present disclosure
  • Figure 3 is a block diagram illustrating an example video decoder according to some embodiments of the present disclosure
  • FIG. 4A is a flowchart illustrating an encoding method according to some embodiments of the present disclosure
  • Figure 4B is a flowchart illustrating an example method for encoding a current block using an encoding method according to some embodiments of the present disclosure
  • FIG. 5A is a flowchart illustrating a decoding method according to some embodiments of the present disclosure
  • 5B is a flowchart illustrating an example method for decoding a current block using a decoding method according to some embodiments of the present disclosure
  • FIG. 6A is a schematic diagram illustrating bit depth transformation according to some embodiments of the present disclosure.
  • FIG. 6B is another schematic diagram illustrating bit depth transformation according to some embodiments of the present disclosure.
  • FIG. 7 is a schematic diagram illustrating an example application according to some embodiments of the present disclosure.
  • FIG. 8 is a schematic diagram showing another example application according to some embodiments of the present disclosure.
  • Figure 9 is a schematic block diagram illustrating a computing device according to some embodiments of the present disclosure.
  • Figure 10 is a schematic diagram illustrating the architecture of a computing device according to some embodiments of the present disclosure.
  • Figure 11 is a schematic diagram illustrating a non-transitory computer readable storage medium according to some embodiments of the present disclosure.
  • Video codecs generally include electronic circuits or software that compress or decompress digital video, and are continually improved to provide greater coding efficiency.
  • a video codec converts uncompressed video into a compressed format and vice versa.
  • Video quality, amount of data used to represent video (determined by bit rate), complexity of encoding and decoding algorithms, susceptibility to data loss and errors, ease of editing, random access, end-to-end latency (delay time ) have complex relationships.
  • Compression formats typically conform to standard video compression specifications, such as the High Efficiency Video Codec (HEVC) standard (also known as H.265), the pending Versatile Video Codec (VVC) standard (also known as H.266), or others Current and/or future video codec standards.
  • HEVC High Efficiency Video Codec
  • VVC Versatile Video Codec
  • bit depth bit depth
  • 12bit/14bit bit depth
  • bit depth bit depth
  • display there are currently display devices on the market that support various bit depths. In order to adapt to the display requirements of various display devices, take into account the viewing needs of consumers and the amount of data transmission, etc., it is necessary to increase the bit depth processing process in the codec to flexibly process the bit of the picture in the codec depth.
  • some embodiments of the present disclosure provide a codec framework, in which a processing module for transforming the bit depth of pictures in a video is added to a general codec, so that the encoding and decoding can be performed according to actual needs and other factors to perform operations such as bit-depth compression and bit-depth expansion on the video, reduce the bit rate under the condition of maximally retaining color diversity, and at the same time meet the requirements of the display device for the bit-depth of the video.
  • information about bit-depth transformation can be generated during the encoding process, and the generated information can be transmitted to the decoding end in the bit stream.
  • corresponding processing can be performed according to the information about bit depth transformation parsed from the bit stream, such as extending the bit depth of the decoded picture, so as to provide more flexibility in codec communication. It can be understood that, in the present disclosure, codecs with the same structure can be used for the encoding end and the decoding end.
  • FIG. 1 is a block diagram illustrating an example video codec system 1000 that can perform a codec method according to some embodiments of the present disclosure.
  • the techniques of this disclosure generally relate to coding (encoding and/or decoding) video data.
  • video data includes any data used to process video, thus, video data may include unencoded raw video, encoded video, decoded (eg, reconstructed) video, and video metadata such as syntax data.
  • a video can include one or more pictures, or called a picture sequence.
  • a system 1000 includes a source device 102 for providing encoded video data to be decoded by a destination device 116 for display, the encoded video data being used to form a bitstream (bitstream ) to be transmitted to the decoding end, where the bit stream can also be referred to as the bit stream.
  • source device 102 provides encoded video data to destination device 116 via computer-readable medium 110 .
  • Source device 102 and destination device 116 may be implemented as a variety of devices, such as desktop computers, notebook (i.e., portable) computers, tablet computers, mobile devices, set-top boxes, smartphones, handheld phones, televisions, cameras, display devices, digital media players , video game consoles, video streaming devices, etc.
  • source device 102 and destination device 116 may be equipped for wireless communication, and thus may also be referred to as wireless communication devices.
  • source device 102 includes video source 104 , memory 106 , video encoder 200 , and output interface 108 .
  • Destination device 116 includes input interface 122 , video decoder 300 , memory 120 and display device 118 .
  • the video encoder 200 of the source device 102 and the video decoder 300 of the destination device 116 may be configured to implement the encoding method and the decoding method according to some embodiments of the present disclosure.
  • source device 102 represents an example of a video encoding device
  • destination device 116 represents an example of a video decoding device.
  • source device 102 and destination device 116 may include other components or configurations.
  • source device 102 may receive video data from an external video source such as an external camera.
  • the destination device 116 may interface with an external display device without the integrated display device 118 built in.
  • System 1000 shown in FIG. 1 is only one example.
  • any digital video encoding and/or decoding device can execute the encoding method and decoding method according to some embodiments of the present disclosure.
  • Source device 102 and destination device 116 are merely examples of such codec devices, where source device 102 generates a bitstream for transmission to destination device 116 .
  • This disclosure refers to a "codec" device as a device that performs codec (encoding and/or decoding) of data. Accordingly, the video encoder 200 and the video decoder 300 represent examples of codec devices, respectively.
  • devices 102, 116 operate in a substantially symmetrical manner such that both devices 102, 116 include video encoding and decoding components, ie, devices 102, 116 can both implement video encoding and decoding processes.
  • system 1000 may support one-way or two-way video transmission between video devices 102 and 116, such as may be used for video streaming, video playback, video broadcasting, or video telephony communications.
  • video source 104 represents a source of video data (i.e., unencoded raw video data) and provides a continuous series of pictures (also referred to as "frames") of the video data to video encoder 200, which encodes the pictures data is encoded.
  • Video source 104 of source device 102 may include a video capture device, such as a video camera, a video archive containing previously captured raw video, and/or a video feed interface for receiving video from a video content provider.
  • video source 104 may generate computer graphics-based data as source video or a combination of live, archived, and computer-generated video.
  • video encoder 200 encodes captured, pre-captured, or computer-generated video data.
  • Video encoder 200 may rearrange the pictures from the order in which they were received (sometimes referred to as "display order") into an encoding order for encoding. Video encoder 200 may generate a bitstream including encoded video data. Source device 102 may then output the generated bitstream via output interface 108 to computer-readable medium 110 for receipt and/or retrieval, eg, by input interface 122 of destination device 116 .
  • Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory.
  • memory 106 and memory 120 may store raw video data, such as raw video data from video source 104 and decoded video data from video decoder 300 .
  • the memory 106 and the memory 120 may respectively store software instructions executable by the video encoder 200 and the video decoder 300 , respectively.
  • video encoder 200 and video decoder 300 may also include internal memory for functionally similar or equivalent purposes.
  • the memory 106 and the memory 120 may store encoded video data output from the video encoder 200 and input to the video decoder 300 or the like.
  • portions of memory 106 and memory 120 may be allocated as one or more video buffers, such as to store decoded raw video data and/or encoded raw video data.
  • Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116 .
  • computer-readable medium 110 represents a communication medium to enable source device 102 to transmit a bitstream directly to destination device 116 in real-time via a radio frequency network, a computer network, or the like.
  • the output interface 108 can modulate transmission signals including encoded video data
  • the input interface 122 can modulate received transmission signals.
  • the communications medium may comprise wireless or wired communications media, or both, such as a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the communications medium may form part of a packet-based network such as a Local Area Network, a Wide Area Network or a global network such as the Internet.
  • Communication media may include routers, switches, base stations, or any other device operable to facilitate communication from source device 102 to destination device 116.
  • source device 102 may output encoded data from output interface 108 to storage device 112 .
  • destination device 116 may access encoded data from storage device 112 via input interface 122 .
  • Storage devices 112 may include various distributed data storage media or locally accessed data storage media, such as hard drives, Blu-ray discs, digital video discs (DVDs), compact discs-read-only drives (CD-ROMs), flash memory, volatile or Non-volatile memory or any other suitable digital storage medium for storing encoded video data.
  • source device 102 may output the encoded data to file server 114 or another intermediate storage device that may store encoded video generated by source device 102 .
  • Destination device 116 may access the stored video data from file server 114 via online or download means.
  • File server 114 may be any type of server device capable of storing encoded data and transmitting the encoded data to destination device 116 .
  • File server 114 may represent a web server (eg, for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device.
  • Destination device 116 may access the encoded data from file server 114 over any standard data connection, including an Internet connection.
  • This may include wireless channels such as Wi-Fi connections suitable for accessing encoded video data stored on file server 114, wired connections such as Digital Subscriber Line (DSL) and cable modems, or a combination of wireless channels and wired connections.
  • the file server 114 and the input interface 122 may be configured to operate according to a streaming transport protocol, a download transport protocol, or a combination thereof.
  • Output interface 108 and input interface 122 may represent wired networking components such as wireless transmitters/receivers, modems, Ethernet cards, wireless communication components operating according to any of various IEEE 802.11 standards, or other physical components.
  • the output interface 108 and the input interface 122 include wireless components
  • the output interface 108 and the input interface 122 may be configured to comply with fourth-generation mobile communication technology (4G), 4G long-term evolution (4G-LTE), LTE-Advanced (LTE Advanced), fifth-generation mobile communication technology (5G) or other cellular communication standards to transmit data such as encoded data.
  • output interface 108 includes a wireless transmitter
  • output interface 108 and input interface 122 may be configured to transmit encoded data and other data.
  • source device 102 and/or destination device 116 may include corresponding system-on-chip (SoC) devices.
  • SoC system-on-chip
  • source device 102 may include a SoC device to perform the functions of video encoder 200 and/or output interface 108
  • destination device 116 may include a SoC device to perform functions such as video decoder 300 and/or input interface 122 .
  • the disclosed technology can be applied to video coding supporting multiple multimedia applications, such as Internet streaming video transmission such as wireless TV broadcasting, cable TV transmission, satellite TV transmission, dynamic adaptive streaming based on HTTP, digital data encoded on data storage media, etc. Video, decoding of digital video stored on data storage media, or other applications.
  • multimedia applications such as Internet streaming video transmission such as wireless TV broadcasting, cable TV transmission, satellite TV transmission, dynamic adaptive streaming based on HTTP, digital data encoded on data storage media, etc.
  • Video decoding of digital video stored on data storage media, or other applications.
  • Input interface 122 of destination device 116 receives a bitstream from computer readable medium 110 (eg, storage device 112, file server 114, etc.).
  • the bitstream may include signaling information defined by the video encoder 200 that is also used by the video decoder 300, such as a code describing video blocks or other coding units (such as slices, pictures, groups of pictures, sequences, etc.) A syntax element for the value of a property and/or process.
  • Display device 118 displays decoded pictures of the decoded video data to a user.
  • Display device 118 may be various types of display devices, such as cathode ray tube (CRT) based devices, liquid crystal displays (LCD), plasma displays, organic light emitting diode (OLED) displays, or other types of display devices, among others.
  • CTR cathode ray tube
  • LCD liquid crystal displays
  • plasma displays organic light emitting diode (OLED) displays
  • other types of display devices among others.
  • video encoder 200 and video decoder 300 may each be integrated with an audio encoder and/or audio decoder, and may include appropriate multiplexing-demultiplexing (MUX -DEMUX) unit or other hardware and/or software to process multiplexed streams including both audio and video in a common data stream.
  • MUX -DEMUX multiplexing-demultiplexing
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols such as User Datagram Protocol (UDP).
  • UDP User Datagram Protocol
  • Both video encoder 200 and video decoder 300 may be implemented as any suitable codec circuit, such as a microprocessor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) , discrete logic elements, software, hardware, firmware, or any combination thereof.
  • a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more of the above processors to perform the techniques of this disclosure .
  • Both video encoder 200 and video decoder 300 may be included in one or more encoders or decoders, either of which may be integrated as a combined encoder/decoder (CODEC) in a corresponding device a part of.
  • a device including video encoder 200 and/or video decoder 300 may be an integrated circuit, a microprocessor, and/or a wireless communication device such as a cellular telephone.
  • Video encoder 200 and video decoder 300 may operate according to a video codec standard, for example, according to a video codec standard such as ITU-T H.265 (also known as High Efficiency Video Codec (HEVC)), or according to Multiview and/or Scalable Video Codec extensions such as HEVC extensions to operate.
  • video encoder 200 and video decoder 300 may operate according to other proprietary or industry standards, such as the Joint Exploratory Test Model (JEM) or the Common Video Codec (VVC) standard currently under development.
  • JEM Joint Exploratory Test Model
  • VVC Common Video Codec
  • the video encoder 200 and the video decoder 300 can encode and decode video data expressed in YUV (eg, Y, Cb, Cr) format. That is, instead of red-green-blue (RGB) data for picture samples, video encoder 200 and video decoder 300 may encode luma and chrominance components, which may include red hues and blue The chroma component of the hue hue.
  • video encoder 200 converts the received RGB formatted data to YUV format before encoding
  • video decoder 300 converts the YUV format to RGB format.
  • pre-processing units and post-processing units may perform these transformations.
  • video encoder 200 and video decoder 300 may perform a block-based encoding and decoding process of pictures.
  • the term "block” generally refers to a structure comprising data to be processed (eg, encoded, decoded, or otherwise used in the encoding and/or decoding process).
  • a block may include a two-dimensional matrix of luma and/or chroma data samples.
  • a picture may first be divided into multiple blocks for encoding processing, and a block in the picture that is undergoing encoding and decoding processing may be referred to as a "current block".
  • embodiments of the present disclosure may also relate to encoding and decoding a picture to include a process of encoding or decoding picture data.
  • the present disclosure may relate to encoding blocks of a picture to include encoding or decoding data for the blocks, such as predictive and/or residual encoding.
  • the bitstream resulting from the encoding process typically includes a series of values for syntax elements that represent encoding decisions (such as encoding modes) and information for partitioning the picture into blocks. Therefore, encoding a picture or a block can generally be understood as encoding the values of the syntax elements forming the picture or block.
  • HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs).
  • a video encoder such as video encoder 200 partitions a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video encoder partitions the CTU and CU into four equal non-overlapping blocks, and each node of the quadtree has zero or four child nodes. Nodes with no child nodes may be referred to as "leaf nodes," and CUs of such leaf nodes may include one or more PUs and/or one or more TUs.
  • a video encoder can further partition PUs and TUs.
  • a residual quadtree represents the partitioning of a TU.
  • PU means inter prediction data
  • TU means residual data.
  • An intra-frame predicted CU includes intra-frame prediction information such as an intra-frame mode indication.
  • Video encoder 200 and video decoder 300 may be configured to use quadtree partitioning in accordance with HEVC, quadtree binary tree (QTBT) partitioning in accordance with JEM, or use other partitioning structures. It should be appreciated that the techniques of this disclosure are also applicable to video encoders configured to use quadtree partitioning or other partitioning types.
  • the video encoder 200 encodes video data of a CU representing prediction information and/or residual information and other information.
  • the prediction information indicates how to predict a CU to form a prediction block of the CU.
  • the residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the samples of the prediction block.
  • Video encoder 200 may further generate syntax data for video decoder 300, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for example, in a picture header, block header, slice header, etc. , or generate other syntax data such as a sequence parameter set (SPS), picture parameter set (PPS) or video parameter set (VPS).
  • Video decoder 300 may likewise decode such syntax data to determine how to decode the corresponding video data.
  • video encoder 200 may generate a bitstream including encoded video data, such as syntax elements describing partitioning of a picture into blocks (eg, CUs) and prediction information and/or residual information for the blocks.
  • video decoder 300 may receive the bitstream and decode the encoded video data.
  • video decoder 300 performs a process reciprocal to that performed by video encoder 200 to decode encoded video data in a bitstream.
  • video decoder 300 may decode values of syntax elements of a bitstream in a manner substantially similar to video encoder 200 .
  • the syntax element can define the partition information of the picture as CTU, and partition each CTU according to the corresponding partition structure such as QTBT structure, so as to define the CU of the CTU.
  • the syntax elements may further define prediction information and residual information for a block of video data, such as a CU.
  • the residual information may be represented by, for example, quantized transform coefficients.
  • the video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of the block to reproduce a residual block of the block.
  • the video decoder 300 uses the prediction mode (intra or inter prediction) signaled in the bitstream and associated prediction information (such as motion information for inter prediction) to form a predictive block for a block. Video decoder 300 may then combine (on a sample-by-sample basis) the predictive block and the residual block to reproduce the original block. Additionally, video decoder 300 may also perform additional processing, such as performing a deblocking process to reduce visual artifacts along block boundaries.
  • Some embodiments of the present disclosure provide a codec framework, adding a processing module for converting the bit depth of pictures in a video to a general codec, so that the pictures to be processed can be processed according to actual needs at the encoding end and decoding end Perform operations such as bit depth compression and bit depth expansion, and generate information about bit depth transformation during the encoding process, and transmit the generated information to the decoding end in the bit stream.
  • corresponding processing can be performed according to the information about bit depth transformation parsed from the bit stream, such as extending the bit depth of the decoded picture, so as to provide more flexibility in codec communication.
  • FIG. 2 is a block diagram illustrating an example video encoder according to some embodiments of the present disclosure
  • FIG. 3 is a block diagram illustrating an example video decoder according to some embodiments of the present disclosure, for example, shown in FIG. 2
  • the encoder shown in FIG. 1 can be implemented as the video encoder 200 in FIG. 1
  • the decoder shown in FIG. 3 can be implemented as the video decoder 300 in FIG. 1 .
  • the codec according to some embodiments of the present disclosure will be described in detail below with reference to FIG. 2 and FIG. 3 .
  • FIGS. 2 and 3 are provided for purposes of explanation and should not be viewed as limitations on the techniques broadly illustrated and described in this disclosure.
  • this disclosure describes video encoder 200 and video decoder 300 in the context of a developing video codec standard, such as the HEVC video codec standard or the H.266 video codec standard, but the techniques of this disclosure are not limited to These video codec standards.
  • the various units (or referred to as modules) in FIG. 2 are shown to aid in understanding the operations performed by video encoder 200 . These units may be implemented as fixed function circuits, programmable circuits or a combination of both.
  • Fixed-function circuits are circuits that provide a specific function and are preprogrammed on executable operations.
  • Programmable circuits refer to circuits that can be programmed to perform a variety of tasks and provide flexible functionality in executable operations.
  • a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware.
  • Fixed-function circuits execute software instructions (to receive parameters, output parameters, etc.), but the types of operations performed by fixed-function circuits are usually fixed.
  • one or more units may be different circuit blocks (fixed function circuit blocks or programmable circuit blocks), and in some examples, one or more units may be integrated circuits.
  • the video encoder 200 shown in FIG. 2 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), digital circuits, analog circuits, and/or a programmable core formed of programmable circuits.
  • ALU arithmetic logic unit
  • EFU elementary function unit
  • digital circuits analog circuits
  • programmable core formed of programmable circuits.
  • memory 106 FIG. 1 may store object code for software received and executed by video encoder 200, or other A memory (not shown) is used to store such instructions.
  • the video encoder 200 may receive an input video, for example, may receive the input video from such as a video data storage, or may directly receive the input video from a video capture device.
  • the video data storage may store video data to be encoded by the video encoder 200 component.
  • Video encoder 200 may receive video data stored in a video data store, such as video source 104 (shown in FIG. 1 ).
  • the decoding buffer can be used as a reference picture memory to store reference video data for use by the video encoder 200 when predicting subsequent video data.
  • the video data memory and decode buffer can be formed from various memory devices such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices.
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • MRAM magnetoresistive RAM
  • RRAM resistive RAM
  • the video data storage and the decode buffer can be provided by the same storage device or different storage devices.
  • the video data memory may be located on the same chip as other components of the video encoder 200 as shown in FIG. 2 , or may not be located on the same chip as other components.
  • references to video data memory should not be construed as limited to memory internal to video encoder 200 (unless specifically described as such) or to memory external to video encoder 200 (unless specifically described as such). More precisely, the reference to the video data storage should be understood as a reference storage for storing the video data received by the video encoder 200 for encoding (such as the video data of the current block to be encoded).
  • the memory 106 in FIG. 1 may also provide temporary storage for the output of each unit in the video encoder 200 .
  • the video encoder 200 includes a bit-depth transformation unit for performing bit-depth transformation on a video, such as received from a video data storage, to change its bit depth.
  • the bit-depth transformation unit is configured to perform bit-depth transformation processing on pictures in the video, for generating transformed pictures and generating bit-depth transformation information, wherein the bit-depth transformation information indicates the bit depth transformation performed on the picture The depth transform processes the associated information.
  • the bit-depth transformation information associated with the bit-depth transformation is entropy-encoded together with the encoding information of the video data to form a bit stream to be transmitted to the decoding end.
  • the bit depth transformation unit may be implemented as a bit depth compression unit for compressing the bit depth of the video, for example, the bit depth of the original video may be 10 bits, after The processing of the bit depth compression unit can compress the bit depth of the original video to 8bit.
  • the bit depth of a video means the bit depth of pictures included in the video.
  • bit depth values for example, 10bit and 8bit
  • 10bit means that the number of color depth bits is 10 bits, which means that the picture has more color levels, more natural color transitions, less prone to color separation, and 10bit has a larger dynamic range, and the smallest signal that can be recognized is finer .
  • 10bit means that the amount of data that needs to be stored is larger, and the storage space required for color adjustment in the later stage is larger.
  • the bit-depth transformation process performed by the bit-depth transformation unit on the picture includes performing bit-depth compression on the picture by using a bit-depth compression method, so as to reduce the bit-depth of the picture.
  • the bit-depth transformation information includes bit-depth compression control information, wherein the bit-depth compression control information indicates information associated with bit-depth compression processing.
  • the encoder 200 may record bit-depth compression control information associated with the compression process.
  • the bit depth compression control information includes one or more of the following: information indicating the above bit depth compression method, information indicating the initial bit depth before bit depth compression processing (such as 10bit), indicating Information such as compressed bit depth information (such as 8bit) after bit depth compression processing.
  • the above data about the bit depth compression control information can be entropy encoded to be written into the bit stream, so as to be transmitted to such as the video decoder 300 together with the encoding information of the video.
  • the bit-depth transformation unit may also be implemented as a bit-depth extension unit for extending the bit-depth of the video, for example, the bit-depth of the original video may be 10 bits, After processing by the bit depth extension unit, the bit depth of the original video can be extended to 12 bits. For example, this situation may be applicable to situations that have high requirements on the color of the video or do not consider the amount of transmitted data.
  • the encoder 200 may record the method used for the extension processing, and perform entropy encoding on the data representing the extension processing method to write into the bitstream , to be transmitted to such as the video decoder 300 together with the encoding information of the video.
  • the encoder 200 may also record the bit depth before the bit depth extension and the bit depth after the bit depth extension, and the above information may be used as bit depth transformation information.
  • bit depth compression or expansion is not limited, and existing or future bit depth processing methods may be used to implement depth transformation.
  • bit-depth transformation is performed on the input video
  • encoding processing may be performed on the depth-transformed video.
  • a general encoding process will be described below with reference to FIG. 2 .
  • a mode selection unit typically coordinates multiple encoding passes to test combinations of encoding parameters and the rate-distortion values resulting from these combinations.
  • the encoding parameters may include partitioning of CTUs into CUs, prediction modes of CUs, transformation types of residual data of CUs, quantization parameters of residual data of CUs, and the like.
  • the mode selection unit may finally select the encoding parameter combination with better rate-distortion value than other tested combinations.
  • Video encoder 200 may partition a picture retrieved from video memory into a series of CTUs and pack one or more CTUs into slices.
  • the mode selection unit may divide the CTUs of the picture according to the tree structure (such as the above-mentioned QTBT structure or the HEVC quadtree structure).
  • video encoder 200 may form one or more CUs by partitioning a CTU according to a tree structure. Such CUs may also be commonly referred to as "blocks" or "video blocks.”
  • the mode selection unit also controls its components (such as motion estimation unit, motion compensation unit, and intra prediction unit) to generate the prediction block of the current block (such as the current CU or the overlapping part of PU and TU in HEVC).
  • the motion estimation unit may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (eg, one or more decoded pictures stored in a decoding cache).
  • the motion estimation unit can calculate the difference between the potential reference block and the current A value for the degree of block similarity, the motion estimation unit can typically perform these calculations using the sample-by-sample difference between the current block and the reference block under consideration.
  • the motion estimation unit may identify the reference block with the lowest value resulting from these calculations, indicating the reference block that most closely matches the current block.
  • a motion estimation unit may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of the current block in the current picture.
  • the motion estimation unit may then provide the motion vectors to the motion compensation unit.
  • MVs motion vectors
  • the motion estimation unit may provide a single motion vector
  • bi-directional inter prediction the motion estimation unit may provide two motion vectors.
  • the motion compensation unit may then use the motion vectors to generate a predictive block.
  • a motion compensation unit may use a motion vector to retrieve data for a reference block.
  • the motion compensation unit may interpolate the prediction block according to one or more interpolation filters.
  • the motion compensation unit may retrieve data of two reference blocks identified by corresponding motion vectors, and combine the retrieved data by sample-by-sample averaging or weighted averaging, etc.
  • an intra prediction unit may generate a prediction block from samples adjacent to a current block.
  • an intra prediction unit may typically mathematically combine the values of neighboring samples and fill in these computed values along a defined direction on the current block to produce a predicted block.
  • the intra prediction unit may calculate an average of samples neighboring the current block, and generate a prediction block to include the resulting average for each sample of the prediction block.
  • the mode selection unit may provide the prediction block to the residual unit.
  • the residual unit receives the bit-depth transformed video from the bit-depth transform unit, and receives the prediction block from the mode selection unit.
  • the residual unit computes the sample-by-sample difference between the current block and the predicted block.
  • the resulting sample-by-sample difference defines the residual block for the current block.
  • the residual unit may also determine differences between sample values in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM).
  • RPCM residual differential pulse code modulation
  • the residual unit may be formed by one or more subtractor circuits that perform binary subtraction.
  • each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit.
  • Video encoder 200 and video decoder 300 may support various PUs of different sizes. As described above, the size of a CU may refer to the size of a luma coding block of the CU, and the size of a PU may refer to the size of a luma prediction unit of the PU.
  • the video encoder 200 may support 2N ⁇ 2N or N ⁇ N sized PUs for intra prediction, and 2N ⁇ 2N, 2N ⁇ N, N ⁇ N for inter prediction Symmetric PUs of ⁇ 2N, N ⁇ N, or similar size.
  • Video encoder 200 and video decoder 300 may also support asymmetric partitioning of 2NxnU, 2NxnD, nLx2N, and nRx2N sized PUs for inter prediction.
  • each CU may be associated with a luma-encoded block and a corresponding chroma-encoded block.
  • the size of a CU may refer to the size of a luma-encoded block of the CU.
  • Video encoder 200 and video decoder 300 may support 2Nx2N, 2NxN, or Nx2N sized CUs.
  • the mode selection unit can generate the The predicted block for the current block.
  • the mode selection unit may not generate a prediction block, but instead generate syntax elements indicating the manner in which the block is reconstructed according to the selected palette. In such modes, the mode selection unit may provide these syntax elements to the entropy coding unit for encoding.
  • the residual unit receives the current block and the corresponding prediction block.
  • the residual unit then generates a residual block for the current block.
  • the residual unit computes the sample-by-sample difference between the predicted block and the current block.
  • a transform unit (“Transform & Sampling & Quantization” shown in Figure 2) applies one or more transforms to a residual block to produce a block of transform coefficients (eg, referred to as a "transform coefficient block").
  • a transform unit may apply various transforms to a residual block to form a block of transform coefficients.
  • the transform unit may apply discrete cosine transform (DCT), directional transform, Karlow transform (KLT), or conceptually similar transforms to the residual block.
  • the transform unit may perform multiple transforms on the residual block, eg, a primary transform and a secondary transform, such as a rotation transform.
  • the transform unit may not apply transforms to the residual block.
  • the transform unit may quantize the transform coefficients in the transform coefficient block to produce a block of quantized transform coefficients.
  • a transform unit may quantize transform coefficients of a transform coefficient block according to a quantization parameter (QP) value associated with a current block.
  • Video encoder 200 eg, via the mode select unit
  • the encoder 200 may further include an encoding control unit for generating control information for operations in the encoding process.
  • an inverse quantization and inverse transform unit (“inverse quantization & inverse transform” shown in FIG. 2 ) may respectively apply inverse quantization and inverse transform to the quantized transform coefficient block to obtain a reconstructed residual block from the transform coefficient block.
  • the reconstruction unit may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block produced by the mode selection unit. For example, the reconstruction unit may add the samples of the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit to generate the reconstructed block.
  • the reconstructed block may be subjected to a filtering process, such as an in-loop filtering unit shown in FIG. 2, to perform one or more filtering operations.
  • a filtering process such as an in-loop filtering unit shown in FIG. 2, to perform one or more filtering operations.
  • the filtering process may include deblocking operations to reduce blocking artifacts along CU edges.
  • the filtering process may be skipped.
  • the video encoder 200 can store the reconstructed block in the decoding cache.
  • the reconstruction unit may store the reconstructed block in a decoding cache.
  • the filtered reconstructed block may be stored in a decoding cache.
  • the motion estimation unit and the motion compensation unit may retrieve reference pictures formed of reconstructed (and possibly filtered) blocks from the decoding cache for inter prediction of blocks of subsequently encoded pictures.
  • the intra prediction unit may use the reconstructed block in the decoding buffer of the current picture to perform intra prediction on other blocks in the current picture.
  • the operations described above are with respect to blocks. This description should be understood as operating for luma-encoded blocks and/or chroma-encoded blocks.
  • the luma-encoded block and the chroma-encoded block are the luma and chroma components of the CU.
  • the luma-encoded block and the chroma-encoded block are the luma and chroma components of the PU.
  • the video encoder 200 may further include a bit-depth inverse transform unit.
  • the bit-depth inverse transform unit may be configured to perform bit-depth inverse transform processing on the decoded picture obtained through loop filtering, for example, to generate an inverse-transformed picture and generate bit-depth inverse transform information. It can be understood that the bit-depth inverse transform process and the bit-depth transform process are inverse operations, and the bit-depth inverse transform information indicates information associated with the bit-depth inverse transform process performed on the decoded picture.
  • Reciprocal operations can be understood as bit-depth inverse transform processing is compressed bit-depth processing when bit-depth transform processing is extended bit-depth processing, or, in the case of bit-depth transform processing is compressed bit-depth processing, bit-depth inverse transform Processing is extended bit depth processing.
  • bit-depth inverse transform process and the bit-depth transform process may not be inverse operations of each other.
  • the bit-depth transformation unit before encoding a picture in the input video, the bit-depth transformation unit first reduces the bit-depth of the picture, and then performs the bit-depth reduction on the picture with the reduced bit-depth Encoding processing.
  • This is because although a higher bit depth image can provide better color transition effects, it will also occupy more storage space and transmit data.
  • the original image file with a higher bit depth can compress the bit depth of the picture in the input video during the encoding process to reduce the bit depth of the picture in the input video, for example, from 10bit to 8bit, so that the transmission bit rate can be reduced .
  • bit depth inverse transformation unit can be implemented as a bit depth extension unit to increase the bit depth of the decoded picture, for example, restore the bit depth of the decoded picture from 8bit to 10bit for the subsequent effect verification unit, slightly The operation of the effect verification unit will be described later.
  • the bit depth inverse transform information includes bit depth inverse transform control information, wherein the bit depth inverse transform control information indicates information associated with bit depth inverse transform processing.
  • the bit-depth inverse transform control information includes bit-depth inverse transform switch information, and indicates whether to perform bit-depth inverse transform processing on the decoded picture.
  • the control information about the bit-depth inverse transformation switch information can be generated by the encoding control unit.
  • the encoding control unit can judge whether to open the bit-depth inverse transformation unit according to the current computing power of the encoder, real-time display requirements, etc. , that is, it is judged whether to perform bit-depth inverse transform processing on the decoded picture.
  • the bit depth inverse transform unit can determine whether to perform inverse transform processing on the decoded picture according to the control information, and record the bit depth inverse transform control information associated with the bit depth inverse transform process, and the bit depth inverse transform control information can be entropy coded to transmitted to the decoder.
  • the bit depth inverse transform control information further includes at least one of the following: information indicating a bit depth inverse transform method, information indicating an input bit depth before bit depth inverse transform processing (for example, 8 bits), Information (for example, 10 bits) indicating the output bit depth after the bit depth inverse transform process.
  • the effect verification unit may be configured to compare the inverse-transformed picture (for example, 10bit) generated by the bit-depth inverse-transform unit with the original original picture without depth transformation (for example, the same 10bit) for comparison to perform effect check on bit depth transformation, and generate bit depth check effect information.
  • the inverse-transformed picture for example, 10bit
  • the original original picture without depth transformation for example, the same 10bit
  • the bit depth verification effect information may include at least one of the following: verification control information indicating whether to perform effect verification, information indicating an effect verification method for effect verification, and information indicating effect verification. Information about the verification result of the verification method.
  • the bit depth check effect information can also be used for entropy coding to be transmitted to the decoding end.
  • the effect verification method can use Peak Signal to Noise Ratio (PSNR) to calculate the distortion between the generated inverse transformed image and the original image, and the calculated distortion value represents the effect verification Information about the validation results of the method.
  • PSNR Peak Signal to Noise Ratio
  • bit_convertion_extension() may be included in the bitstream, and the bit-depth conversion extension bit is used to transmit information such as bit-depth conversion information in the bitstream .
  • bit_convertion_extension() may be included in the bitstream, and the bit-depth conversion extension bit is used to transmit information such as bit-depth conversion information in the bitstream.
  • an entropy encoding unit may entropy encode syntax elements received from other functional components of video encoder 200 .
  • the entropy encoding unit may entropy encode quantized transform coefficient blocks from the transform unit, and may also entropy encode information such as bit depth transform information.
  • the entropy coding unit may entropy code the prediction syntax elements from the mode selection unit, such as motion information for inter prediction or intra mode information for intra prediction, to generate entropy coded data.
  • the entropy coding unit may perform context-adaptive variable-length coding (CAVLC) operations, context-adaptive binary arithmetic coding (CABAC) operations, variable-length coding operations, syntax-based context-adaptive binary arithmetic coding (SBAC) operations on the data operation, Probabilistic Interval Partitioning Entropy (PIPE) encoding operation, Exponential Golomb encoding operation, or other type of entropy encoding operation.
  • the entropy coding unit may operate in a bypass mode in which syntax elements are not entropy coded.
  • the video encoder 200 may output a bitstream including entropy encoding syntax elements required to reconstruct blocks of slices or pictures.
  • bit-depth processing (for example, including bit-depth transformation and bit-depth inverse transformation) can be performed on pictures in the input video, and information associated with the bit-depth processing performed on the pictures can be recorded (for example, bit-depth transformation information, bit-depth inverse transformation information, bit-depth verification effect information, etc.), the encoder 200 can selectively entropy-encode it to be transmitted to the decoding end via the bit stream, so that the decoder 300
  • the above information parsed from the bit stream can be used as reference information for bit-depth processing at the decoding end, so that based on the generated and transmitted information such as bit-depth transformation information, the decoder can more flexibly and effectively decode pictures Operations such as bit-depth transformation are performed to realize information communication between the decoder and the encoder about bit-depth processing.
  • the operation of the decoder 300 will be described below in conjunction with FIG. 3 .
  • Using the encoder provided by the embodiments of the present disclosure can perform bit-depth transformation on the picture before encoding the picture, and then encode the generated transformed picture to form the encoding information of the picture.
  • bit-depth transformation information for transmission in the bitstream together with the encoding information, so that the decoder can process the bit-depth of the decoded picture correspondingly according to the bit-depth transformation information parsed in the bitstream to satisfy
  • the bit-depth transformation information used to form the bitstream is conducive to more flexible implementation of the bit-depth transformation process, and at the same time can realize the information communication about the bit-depth transformation between the decoding end and the encoding end.
  • FIG. 3 is a block diagram illustrating an example video decoder according to some embodiments of the present disclosure, for example, the decoder shown in FIG. 3 may be the video decoder 300 in FIG. 1 . It will be appreciated that FIG. 3 is provided by way of explanation and not limitation of the techniques broadly illustrated and described in this disclosure. For purposes of explanation, the video decoder 300 is described according to HEVC technology. However, the disclosed techniques may be performed by video decoding devices configured to other video codec standards.
  • the basic structure of the video decoder 300 can be similar to that of the video encoder shown in FIG. Both the decoder 300 can implement video encoding and decoding processes.
  • the encoder 200 and the decoder 300 may be collectively referred to as a codec. Therefore, the system composed of the encoder 200 and the decoder 300 can support one-way or two-way video transmission between devices, such as can be used for video streaming, video playback, video broadcasting or video telephony communication.
  • video decoder 300 may include more, fewer or different functional components than those shown in FIG. 3 . For purposes of explanation, components related to the decoding conversion process according to some embodiments of the present disclosure are shown in FIG. 3 .
  • the video decoder 300 includes a memory, an entropy decoding unit, a prediction processing unit, an inverse quantization and inverse transformation unit (“inverse quantization & inverse transformation unit” shown in FIG. 3 ), a reconstruction unit, a filter processor unit, decode buffer, and bit depth inverse transform unit.
  • the prediction processing unit may include a motion compensation unit and an intra prediction unit.
  • the prediction processing unit may, for example, also include an addition unit to perform prediction according to other prediction modes.
  • a prediction processing unit may include a palette unit, an intra block copy unit (which may form part of a motion compensation unit), an affine unit, a linear model (LM) unit, and the like.
  • video decoder 300 may include more, fewer or different functional components.
  • the decoder 300 may receive a bitstream including encoded video data.
  • the memory in FIG. 3 may be referred to as a Codec Picture Buffer (CPB) to store a bitstream comprising the encoded video data, awaiting decoding by the components of video decoder 300 .
  • Video data stored in the CPB may be obtained, for example, from computer readable medium 110 (FIG. 1) or the like.
  • the CPB may also store temporary data such as output of each unit of the video decoder 300 .
  • the decode buffer typically stores decoded pictures, which may be output by video decoder 300 and/or used as reference video data when decoding subsequent data or pictures of the bitstream.
  • the CPB memory and decode buffer can be formed from various memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices.
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • MRAM magnetoresistive RAM
  • RRAM resistive RAM
  • the CPB memory and decoding cache can be provided by the same storage device or different storage devices.
  • the CPB memory may be located on the same chip as other components of the video decoder 300 , as shown in the figure, or may not be located on the same chip as other components.
  • a fixed-function circuit refers to a circuit that provides a specific function and is preset on an executable operation.
  • Programmable circuits refer to circuits that can be programmed to perform a variety of tasks and provide flexible functionality in executable operations.
  • a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware.
  • Fixed-function circuits execute software instructions (to receive parameters, output parameters, etc.), but the types of operations performed by fixed-function circuits are usually fixed.
  • one or more units may be different circuit blocks (fixed function circuit blocks or programmable circuit blocks), and in some examples, one or more units may be integrated circuits.
  • Video decoder 300 may include an ALU, an EFU, digital circuits, analog circuits, and/or a programmable core formed of programmable circuits.
  • on-chip or off-chip memory may store instructions (eg, object code) for the software that video decoder 300 receives and executes.
  • the entropy decoding unit may perform entropy decoding on the received bit stream to parse out the bit-depth transformation information and coding information corresponding to the picture, wherein the bit-depth transformation information indicates the bit-depth transformation information that is related to the encoding process of the picture.
  • the depth transform processes the associated information.
  • the decoder 300 may parse the bit-depth conversion information from the above-mentioned bit-depth conversion extension bit (bit_convertion_extension()) in the bitstream.
  • the decoder 300 may perform decoding and conversion processing according to the bit-depth transformation information and the analyzed coding information to generate display video data.
  • the bit-depth transformation process is a bit-depth compression process
  • the bit-depth transformation information includes bit-depth compression control information indicating that it is associated with the bit-depth compression process, wherein decoding is performed according to the bit-depth transformation information and the encoding information
  • the conversion processing includes: performing decoding conversion processing on the coded information with reference to the bit-depth compression control information.
  • the bit depth compression control information may include at least one of the following: information indicating a bit depth compression method, information indicating an initial bit depth (for example, 10 bits) before bit depth compression processing, and indicating a compressed bit depth after bit depth compression processing ( For example, 8bit) information.
  • the decoder 300 can know that the initial input video is encoded and transmitted after being converted from the initial 10 bits to 8 bits according to the indicated bit depth compression method based on the bit depth compression control information obtained through decoding.
  • the decoder 300 may perform corresponding decoding conversion processing by referring to the transmitted bit depth compression control information.
  • the operations that can be performed by the decoder 300 at the decoding end can refer to the decoding conversion process as shown in FIG. , to generate a display picture for display by the display device.
  • the entropy decoding unit may receive a bitstream including encoded video from, for example, the memory 120, and entropy decode it to reproduce syntax elements.
  • the inverse quantization and inverse transform unit (“inverse quantization & inverse transform" shown in FIG. 3), the reconstruction unit and the filter unit may generate decoded video based on syntax elements extracted from the bitstream, for example, generate a decoded picture.
  • video decoder 300 reconstructs a picture block-by-block.
  • the video decoder 300 may individually perform a reconstruction operation on each block, where a block currently being reconstructed (ie, decoded) may be referred to as a "current block".
  • the entropy decoding unit may perform entropy decoding on the syntax elements defining the quantized transform coefficients of the quantized transform coefficient block, as well as transform information such as quantization parameters (QP) and/or transform mode indications.
  • QP quantization parameters
  • the inverse quantization and inverse transform unit may use the QP associated with the block of quantized transform coefficients to determine the degree of quantization, and may likewise determine the degree of inverse quantization to apply.
  • the inverse quantization and inverse transform unit may perform a bitwise left shift operation to inverse quantize the quantized transform coefficients.
  • the inverse quantization and inverse transform unit may thus form a transform coefficient block comprising transform coefficients.
  • the inverse quantization and inverse transform unit may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block.
  • the inverse quantization and inverse transform unit may apply an inverse DCT, an inverse integer transform, an inverse Karo transform (KLT), an inverse rotation transform, an inverse direction transform, or other inverse transforms to the coefficient blocks.
  • the prediction processing unit generates a prediction block from the prediction information syntax elements entropy-decoded by the entropy decoding unit.
  • a motion compensation unit may generate a predictive block if the prediction information syntax element indicates that the current block is inter-predicted.
  • the prediction information syntax element may indicate a reference picture in the decoding cache from which to retrieve the reference block, and a motion vector identifying the position of the reference block in the reference picture relative to the current block in the current picture.
  • the motion compensation unit may generally perform the inter prediction process in a manner substantially similar to that described with respect to the motion compensation unit in FIG. 2 .
  • the intra prediction unit may generate the prediction block according to the intra prediction mode indicated by the prediction information syntax element.
  • an intra prediction unit may generally perform an intra prediction process in a manner substantially similar to that described with respect to the intra prediction unit in FIG. 2 .
  • the intra prediction unit may retrieve data of neighboring samples of the current block from the decoding buffer.
  • the reconstruction unit may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.
  • the filter unit may then perform one or more filter operations on the reconstructed block.
  • the filter unit may perform a deblocking operation to reduce blocking artifacts along reconstructed block edges. It can be appreciated that filtering operations need not be performed in all examples, i.e., filtering operations may be skipped in some cases.
  • Video decoder 300 may store the reconstructed block in a decoding cache.
  • the decoding buffer can provide reference information to units such as motion compensation and motion estimation, such as samples of the current picture for intra prediction and samples of the previously decoded picture for subsequent motion compensation. Additionally, video decoder 300 may output decoded pictures from the decode buffer for subsequent presentation on a display device (eg, display device 118 of FIG. 1 ).
  • the video decoder 300 may further include a bit-depth inverse transform unit for performing bit-depth inverse transform processing on decoded pictures, such as loop-filtered pictures, so as to adapt, for example, to the display device for the displayed video.
  • bit depth requirements As an example, in the case that the display device used to display the output video requires displaying display data with 10 bits, and the bit depth of the decoded picture obtained by decoding is 8 bits, the bit depth of the decoded picture can be performed by using the bit depth inverse transform unit.
  • An inverse transform to increase the bit-depth of the decoded picture and provide bit-depth processed video data as display data. The bit depth inverse transform operation during the decoding transform process will be described in detail below.
  • the entropy decoding unit of the decoder 300 can also parse the bit-depth inverse transform information corresponding to the picture from the bit stream, where the bit-depth inverse transform information indicates the The bit-depth inverse transform processes the associated information.
  • the decoder 300 may parse the bit-depth inverse transform information from the aforementioned bit-depth transform extension bits in the bitstream.
  • performing decoding and conversion processing according to the bit-depth transformation information and the coding information includes: performing decoding processing on the coding information to generate a decoded picture corresponding to the picture; performing bit-depth inverse transformation on the decoded picture with reference to the bit-depth inverse transformation information Processing, used to generate display pictures.
  • bit-depth transformation processing at the encoding end is bit-depth compression processing
  • bit-depth inverse transformation processing is bit-depth expansion processing
  • the bit depth inverse transformation information includes bit depth extension control information
  • the bit depth extension control information includes information indicating the bit depth extension method
  • the decoder 300 can also determine whether the local bit depth extension method of the decoder is available, that is, The decoder 300 may first determine whether it can perform the bit-depth extension method indicated in the bit-depth extension control information by itself. When it is determined that the bit depth extension method is available, the decoder 300 may extend the bit depth of the decoded picture according to the bit depth extension method, for example, increase the bit depth of the decoded picture from 8 bits to 10 bits.
  • the decoder 300 may also receive the bit-depth extension method from, for example, an application layer, so as to perform bit-depth extension on the decoded picture according to the bit-depth extension method. It can be understood that the above process can be applied to the situation where the bit depth extension method configured in the decoder is inconsistent with the indicated bit depth extension method.
  • the cloud receives data about the indicated bit-depth extension method, so that the decoder 300 can execute the indicated bit-depth extension method on the decoded picture.
  • the decoder 300 may also receive control information indicating whether to perform bit-depth inverse transform processing from the application layer, wherein the control information is generated based on at least one of the following: the computing capability of the decoder, The power information of the decoder and the bit depth display requirements of the display. According to the received control information, the decoder 300 can determine whether to enable the bit-depth inverse transform unit configured therein, and if the control information indicates not to enable, the decoder 300 does not perform bit-depth inverse transform processing on the loop-filtered decoded picture.
  • the decoder 300 can also analyze the bit-depth verification effect information corresponding to the picture from the above-mentioned bit-depth transformation extension bit in the bit stream, and compare the decoded picture with the bit-depth extension effect information; refer to the comparison result Determine whether to perform bit-depth inverse transform processing on the decoded picture.
  • the decoder 300 may determine whether to enable the bit-depth inverse transform unit according to the control information received from the application layer about whether to perform bit-depth inverse transform processing and/or according to the bit-depth verification effect information obtained from bitstream parsing.
  • the decoder 300 may also determine whether to perform inverse transformation according to the indicated bit depth inverse transformation method according to the bit depth extension effect information.
  • the bit depth extension effect information may include at least one of the following: verification control information indicating whether to perform effect verification, information indicating an effect verification method for effect verification, and verification information indicating an effect verification method result information.
  • the decoder 300 can know the conversion effect achieved by using the bit-depth inverse conversion method to perform bit-depth inverse transformation, so as to determine whether to At the decoding end, continue to use this method to perform bit-depth inverse transformation. For example, assuming that the conversion effect shows that the converted decoded picture is more distorted than the original picture, the decoder side may not use the bit-depth inverse transformation method for inverse transformation, but use other methods (for example, indicated by the application layer) method) to perform the inverse transformation.
  • the decoder 300 can learn more useful information, so as to perform bit-depth conversion more flexibly.
  • FIG. 4A is a flowchart illustrating an encoding method according to some embodiments of the present disclosure
  • FIG. 4B is a flowchart illustrating an example method of encoding a current block using the encoding method according to some embodiments of the present disclosure.
  • FIGS. 4A and 4B Although described with respect to the video encoder 200 in conjunction with FIGS. 1 and 2 , it is understood that other devices may also be configured to perform encoding methods similar to those shown in FIGS. 4A and 4B .
  • the encoding method includes steps S101 and S102.
  • step S101 bit-depth transformation processing is performed on the picture to generate a transformed picture and generate bit-depth transformation information, wherein the bit-depth Transformation information indicates information associated with bit-depth transformation processing performed on a picture.
  • the picture may be one frame of pictures in the video data.
  • this step S101 may be performed by the bit-depth transformation unit shown in FIG. 2 to process the bit-depth of the picture sequence in the input video, so as to change the bit-depth of the picture.
  • the bit depth transformation unit in order to reduce the bit rate and reduce the amount of data to be transmitted, can be implemented as a bit depth compression unit for compressing the bit depth of the video.
  • the bit depth of the original video can be 10 bits.
  • the processing of the deep compression unit can compress the bit depth of the original video to 8bit.
  • the bit-depth transformation information includes bit-depth compression control information, wherein the bit-depth compression control information indicates information associated with bit-depth compression processing.
  • the bit-depth compression control information includes at least one of the following: information indicating the bit-depth compression method, information indicating the initial bit-depth before the bit-depth compression process, indicating the compression after the bit-depth compression process bit depth information.
  • the bit-depth transformation unit may also be implemented as a bit-depth extension unit for extending the bit-depth of the video.
  • the bit-depth of the original video may be 10 bits.
  • the processing of the bit depth extension unit can extend the bit depth of the original video to 12bit.
  • the bit depth transformation information includes bit depth extension control information, wherein the bit depth extension control information indicates information associated with bit depth extension processing.
  • the bit depth extension control information includes at least one of the following: information indicating the bit depth extension method, information indicating the initial bit depth before bit depth extension processing, indicating compression after bit depth extension processing bit depth information.
  • step S102 encoding processing is performed on the transformed picture to generate encoding information corresponding to the transformed picture.
  • the bit-depth transformation information and the encoding information are used to form a bitstream, that is, the generated bit-depth transformation information and the encoding information can be transmitted to the decoding end, so as to communicate about the bit-depth processing in the codec Provide more flexibility.
  • step S102 it may specifically include steps S1021-S1026 as shown in FIG. 4B.
  • the video encoder 200 may first predict the current block (S1021). For example, the video encoder 200 may form a prediction block of the current block. The video encoder 200 may then calculate a residual block of the current block (S1022). To calculate the residual block, video encoder 200 may calculate the difference between the original uncoded block and the predicted block of the current block. The video encoder 200 may then transform and quantize the coefficients of the residual block (S1023). Next, the video encoder 200 may scan quantized transform coefficients of the residual block (S1024). During or after scanning, the video encoder 200 may entropy-encode the coefficients (S1025). For example, video encoder 200 may entropy encode coefficients using CAVLC or CABAC. Finally, the video encoder 200 may output the entropy-encoded bitstream (S1026).
  • the process of encoding can be understood as including a decoding step, so as to generate decoded pictures for motion estimation, motion compensation, and the like.
  • the process of obtaining the decoded picture may include: performing decoding processing on the coded information to generate the decoded picture.
  • the encoding method may further include performing a bit-depth inverse transform process on the decoded picture to generate an inverse-transformed picture and generate bit-depth inverse transform information, wherein the bit-depth inverse transform process is the same as the step
  • the bit-depth transformation processing in S101 is an opposite operation, and the bit-depth inverse transformation information indicates information associated with the bit-depth inverse transformation processing performed on the decoded picture.
  • the bit-depth transformation processing may be bit-depth compression processing
  • the bit-depth inverse transformation processing may be bit-depth expansion processing. It can be understood that, as another example, bit-depth transformation processing may also be bit-depth expansion processing, and bit-depth inverse transformation processing may be bit-depth compression processing.
  • the encoding method may further include: comparing the inverse transformed picture with the picture to perform effect verification for generating bit depth verification effect information, wherein the bit depth verification effect information includes the following At least one: verification control information indicating whether to perform effect verification, information indicating an effect verification method for effect verification, and information indicating the verification result of the effect verification method.
  • the bit depth inverse transform information includes bit depth inverse transform control information, wherein the bit depth inverse transform control information indicates information associated with bit depth inverse transform processing.
  • the bit-depth inverse transform control information may include bit-depth inverse transform switch information, and indicate whether to perform bit-depth inverse transform processing on the decoded picture.
  • the indication about the bit depth inverse transform switch information may be generated by the encoding control unit in FIG. 2 .
  • performing bit-depth inverse transform processing on the decoded picture includes: performing bit-depth inverse transform processing in response to the bit-depth inverse transform switch information indication, performing bit-depth inverse transform on the decoded picture using a bit-depth inverse transform method, Used to change the bit depth of a decoded picture in the opposite direction of the bit depth transform.
  • bit depth inverse transform control information may further include at least one of the following: information indicating the bit depth inverse transform method, information indicating the input bit depth before the bit depth inverse transform process, indicating the input bit depth after the bit depth inverse transform process Output bit depth information.
  • bit-depth conversion extension bit is included in the bit-stream to transmit such information in the bit-stream, so that after receiving the bit-stream, the decoder obtains the above-mentioned information from the bit-depth conversion extension bit.
  • FIG. 5A is a flowchart illustrating a decoding method according to some embodiments of the present disclosure
  • FIG. 5B is a flowchart illustrating an example method of decoding a current block by using the decoding method according to some embodiments of the present disclosure.
  • the video decoder 300 is described in conjunction with FIGS. 1 and 3 , it is understood that other devices may also be configured to perform decoding methods similar to those shown in FIGS. 5A and 5B .
  • the decoding method includes steps S201 and S202.
  • step S201 the bit-depth transformation information and coding information corresponding to the picture are parsed from the received bitstream, wherein the bit-depth transformation The information indicates information associated with bit-depth transform processing performed in encoding a picture.
  • this step S101 can be performed by the entropy decoding unit shown in FIG. 3, wherein the obtained bit-depth transformation information can be used as reference information for processing such as the bit-depth of a decoded picture to change the bit-depth of the picture. , resulting in a display picture with the desired bit depth.
  • the entropy decoding unit in FIG. 3 may parse the above bit-depth transformation information from the bit-depth transformation extension bits in the bitstream.
  • step S202 a decoding conversion process is performed according to the bit depth conversion information and the encoding information, for generating a display picture.
  • the bit-depth transformation process may be bit-depth compression processing
  • the bit-depth transformation information includes bit-depth compression control information indicating
  • the bit depth transformation unit performs bit depth compression processing on the picture sequence in the input video, so as to reduce the bit depth of the picture.
  • performing decoding conversion processing according to the bit depth transformation information and the encoding information includes: performing decoding conversion processing on the encoding information with reference to the bit depth compression control information.
  • the bit depth compression control information may include at least one of the following: information indicating a bit depth compression method, information indicating an initial bit depth (for example, 10 bits) before bit depth compression processing, and indicating a compressed bit depth after bit depth compression processing ( For example, 8bit) information.
  • the decoder 300 can know that the initial input video is encoded and transmitted after being converted from the initial 10 bits to 8 bits according to the indicated bit depth compression method based on the bit depth compression control information obtained through decoding.
  • the decoding method may further include parsing the bit-depth inverse transform information corresponding to the picture from the bitstream, wherein the bit-depth inverse transform information indicates the bit-depth inverse transformation performed in the process of encoding the picture. Transformation process associated information.
  • performing decoding conversion processing according to the bit-depth transformation information and the coding information includes: performing decoding processing on the coding information to generate a decoded picture corresponding to the picture; referring to the bit-depth inverse transformation information to bit-code the decoded picture Depth inverse transform processing to generate display images.
  • the bit-depth transformation processing is bit-depth compression processing
  • the bit-depth inverse transformation processing is bit-depth extension processing
  • the bit-depth inverse transformation information includes bit-depth extension control information
  • the bit-depth extension control information includes indication Information about the bit depth extension method
  • the decoding method may also include: determining whether the bit depth extension method is available; if it is determined that the bit depth extension method is available, extending the bit depth according to the bit depth extension method; and determining the bit depth extension method If not available, receive a bit-depth extension method for performing bit-depth extension on the decoded picture according to the bit-depth extension method.
  • the decoding method may further include: receiving control information indicating whether to perform bit-depth inverse transform processing, wherein the control information is based on at least one of the following Generated by: the computing power of the decoder, the power information of the decoder, and the bit-depth display requirements of the display; and refer to the control information to determine whether to perform bit-depth inverse transformation processing on the decoded picture.
  • the decoding method before performing bit-depth inverse transform processing on the decoded picture, may further include: parsing the bit-depth verification effect information corresponding to the picture from the bit stream; combining the decoded picture with the bit-depth extension effect information; and refer to the comparison result to determine whether to perform bit-depth inverse transform processing on the decoded picture.
  • the video decoder 300 may perform entropy decoding on received encoding information to determine prediction information of a current block and reproduce coefficients of a residual block (S2021).
  • the video decoder 300 may predict the current block by using an intra or inter prediction mode, etc. indicated by the current block prediction information (S2022), to calculate a predicted block of the current block.
  • the video decoder 300 may then inverse scan the reproduced coefficients (S2023) to create a block of quantized transform coefficients.
  • Video decoder 300 may then inverse quantize and inverse transform the coefficients to generate a residual block (S2024).
  • the video decoder 300 may combine the prediction block and the residual block (S2025), and finally form a decoded picture (S2026).
  • the video decoder 300 may also optionally perform a bit-depth inverse transform on the decoded picture to generate a display picture (S2027).
  • the decoder 300 may receive control information indicating whether to perform bit-depth inverse transform processing from the application layer, so as to determine whether to enable the bit-depth inverse transform unit therein.
  • the decoder 300 may parse the bit-depth verification effect information corresponding to the picture from the bit stream, so as to determine whether to enable the bit-depth inverse transform unit therein according to the verification effect information. It can be understood that the decoder 300 may also comprehensively refer to both the above received control information and the bit depth verification effect information parsed from the bit stream to determine whether to enable the bit depth inverse transform unit.
  • bit-depth inverse transform unit may be implemented as a bit depth compression unit to reduce the bit depth of the decoded picture.
  • bit depth inverse transform unit may be implemented as a bit depth extension unit to increase the bit depth of the decoded picture.
  • FIG. 6A is a schematic diagram illustrating bit-depth transformation according to some embodiments of the present disclosure.
  • video such as captured by a camera device may have a bit depth of 10 bits.
  • the video with a bit depth of 10 bits may first be compressed by the bit depth transformation unit of the encoder 200 as shown in FIG. 2 as an input video, so as to reduce the bit depth to 8 bits for encoding and transmission.
  • this method can reduce the amount of data transmission.
  • information about bit depth processing is also transmitted in the bit depth extension bit in the bit stream to realize the communication of bit depth information between codecs.
  • the decoder 300 may perform decoding processing on the encoded information to obtain a decoded video, which has a bit depth of 8 bits, and according to the above-mentioned control information indicating whether to perform bit-depth inverse transformation processing and/or Or determine whether to perform bit-depth inverse transformation on the decoded video from the bit-depth verification effect information parsed from the bit stream.
  • FIG. 6A shows a situation in which bit-depth inverse transformation is not performed on the decoded video, thus, the decoded video can be directly used as display data.
  • Fig. 6B is a schematic diagram of another bit-depth transformation, in which it shows the situation of performing bit-depth inverse transformation on the decoded video, through inverse transformation, the bit-depth of the decoded video can be processed, for example, up to 10bit for display . It can be understood that FIG. 6A and FIG. 6B only show an example application situation of performing bit-depth transformation by using the coding method and the decoding method according to the embodiments of the present disclosure.
  • bit_convertion_extension() bit_convertion_extension()
  • bit_convertion_extension() syntax elements related to some embodiments of the present disclosure are shown in bold. It can be understood that the following tables are only illustrative, and other grammars can also be defined.
  • bit depth transform extension bits are defined in the extension data.
  • bitDepthExpansionModuleSwitchOnOff specifies the bit depth inverse transform switch information
  • bitDepthExpansionModuleSwitchOnOff 1 specifies to enable the bit depth inverse transform unit in the encoder
  • bitDepthExpansionModuleSwitchOnOff 0 specifies to disable the bit depth inverse transform unit.
  • bitDepthDownScaleMethod specifies the method used to compress the bit depth of the picture during the encoding conversion process.
  • the bit depth compression method can be represented by a 3-bit unsigned integer.
  • Table 3 shows the meaning of the index on bitDepthDownScaleMethod, which shows several compression methods. It can be understood that the above compression methods are only examples, and other compression methods can also be defined. For example, if the index of bitDepthDownScaleMethod is 001, it means that the LTM compression method is used in the encoder. For another example, if the index of bitDepthDownScaleMethod is 000, it means that no bit depth compression method is used in the encoder.
  • bit_convertion_origin and bit_convertion_taiget may also be defined to represent the initial bit-depth before bit-depth compression processing and the bit-depth after compression processing, respectively.
  • bit_depth_origin 001 indicates that the initial bit depth is 10 bits
  • bit_convertion_taiget 000 indicates that the bit depth after compression processing is 8 bits.
  • bitDepthExpansionMethod in Table 2 specifies the method used for the bit depth expansion processing on the picture during the encoding conversion process.
  • the bit depth expansion method may be represented by a 3-bit unsigned integer.
  • Table 5 shows index meanings about bitDepthExpansionMethod, which shows several expansion methods. It can be understood that the above expansion methods are only examples, and other expansion methods can also be defined. For example, if the index of bitDepthExpansionMethod is 001, it indicates that the expansion method of zero padding (ZP) is used in the encoder. For another example, if the index of bitDepthDownScaleMethod is 000, it means that no bit depth scaling method is used in the encoder. In addition, "XXX” is also shown in Table 5 to define other possible extension methods.
  • bit_convertion_input and bit_convertion_output may also be defined to represent the input bit depth before the bit depth extension processing and the output bit depth after the extension processing, respectively.
  • indexes of bit_convertion_input and bit_convertion_output please refer to Table 4 above.
  • bitDepthExpansionEffect in Table 2 specifies the verification control information whether to perform effect verification. For example, bitDepthExpansionEffect equal to 1 can specify that the effect verification process is performed, and bitDepthExpansionEffect equal to 0 specifies that the effect verification process is not performed.
  • expansion_eval_method in Table 2 designates a method of performing an effect, and as an example, the following Table 6 shows index meanings about expansion_eval_method. It can be understood that the verification methods shown in Table 6 are only examples, and other methods can also be defined. For example, if the index of expansion_eval_method is 000, it means that the PSNR verification method is used in the encoder. For another example, if the index of expansion_eval_method is 001, it means that the Structural Similarity algorithm (SSIM) is used in the encoder. In addition, "XXX” is also shown in Table 6 to define other possible verification methods. Next, expansion_eval_effect in Table 2 specifies the result information of the effect verification.
  • SSIM Structural Similarity algorithm
  • expansion_eval_method meaning 000 PSNR 001 SSIM 010 XXX 011 XXX 100-111 reserve
  • Fig. 7 is a schematic diagram showing an example application according to some embodiments of the present disclosure, which shows an example application process of video processing by using a codec method according to some embodiments of the present disclosure, and schematically shows Use of the syntax elements defined in the table.
  • an input video is received by an encoder such as the encoder 200 at the encoding end, and the image sequence of the input video has an initial bit depth of 10 bits.
  • the encoder 200 compresses the bit depth of the input video using a bit depth compression method (LTM) to compress the bit depth to 8 bits, and encodes the compressed video to form a bit stream.
  • LTM bit depth compression method
  • bit depth extension processing is also enabled, so as to restore the compressed video to the original 10bit according to the ZP extension method.
  • PSNR is used to check the effect to determine the degree of image distortion.
  • bit-depth transformation extension bits are defined in the generated bitstream for transmitting the above information about bit transformation processing in the encoding conversion process in the bitstream, and the syntax elements and their indexes included therein are as follows Figure 7 shows.
  • the decoder 300 at the decoding end can receive the above bit stream including bit-depth transformation extension bits, and parse relevant information from it, so that the decoder 300 can refer to the indicated information to perform corresponding bit-depth inverse transformation on the decoded video, for example, according to The indicated extension method ZP extends the bit depth of the decoded video to obtain an output video with a bit depth of 10 bits, and displays the extended video.
  • the decoder 300 may receive control information indicating whether to perform bit-depth inverse transform processing from the application layer, for example, the control information is generated based on the computing capability of the decoder, power information of the decoder, bit-depth display requirements of the display, etc. of. Thus, the decoder 300 can determine whether to perform bit-depth conversion processing on the decoded video according to the current decoding requirement.
  • FIG. 7 shows a situation where the decoder 300 determines that the indicated bit-depth extension method ZP is available, and the decoder 300 can directly extend the bit-depth of the decoded video according to the indicated extension method.
  • Fig. 8 is a schematic diagram showing another example application according to some embodiments of the present disclosure, which shows a situation where the decoder 300 determines that the indicated bit depth extension method ZP is not available. It can be understood that the above process may be applicable to a situation where the bit-depth extension method configured in the decoder 300 (for example, the BR extension method) is inconsistent with the indicated bit-depth extension method (for example, the ZP extension method). In this case, the decoder 300 may acquire relevant data about the indicated ZP extension method from the cloud, such as via a network, so that the decoder 300 can execute the indicated ZP extension method on the decoded picture.
  • the decoder 300 may acquire relevant data about the indicated ZP extension method from the cloud, such as via a network, so that the decoder 300 can execute the indicated ZP extension method on the decoded picture.
  • bit-depth transformation processing on the picture before encoding the picture, and then encode the generated transformed picture to form the encoding information of the picture.
  • Generate bit-depth transformation information for transmission in the bitstream together with encoding information so that the decoder can process the bit-depth of the decoded picture correspondingly according to the bit-depth transformation information parsed in the bitstream to meet requirements such as display
  • using the bit-depth transformation information transmitted in the bit stream is beneficial to realize the bit-depth transformation process more flexibly, and at the same time, it can realize the information communication about the bit-depth transformation between the decoding end and the encoding end.
  • a computing device for executing the encoding method or the decoding method according to some embodiments of the present disclosure.
  • Figure 9 is a schematic block diagram illustrating a computing device according to some embodiments of the present disclosure.
  • the computing device 2000 may include a processor 2010 and a memory 2020 .
  • computer-readable codes are stored in the memory 2020 , and when the computer-readable codes are executed by the processor 2010 , the above-mentioned encoding method or decoding method can be executed.
  • the processor 2010 can perform various actions and processes according to programs stored in the memory 2020 .
  • the processor 2010 may be an integrated circuit with signal processing capability.
  • the above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • Various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed.
  • the general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc., and may be an X86 architecture or an ARM architecture, or the like.
  • the memory 2020 stores computer-executable instruction codes, which are used to implement the encoding method or the decoding method according to the embodiment of the present disclosure when executed by the processor 2010 .
  • Memory 2020 can be volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
  • the nonvolatile memory can be read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or flash memory.
  • Volatile memory can be random access memory (RAM), which acts as external cache memory.
  • RAM Static Random Access Memory
  • DRAM Dynamic Random Access Memory
  • SDRAM Synchronous Dynamic Random Access Memory
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • ESDRAM Enhanced Synchronous Dynamic Random Access Memory
  • SLDRAM Synchronous Linked Dynamic Random Access Memory
  • DRRAM Direct Memory Bus Random Access Memory
  • a computing device for performing an encoding method or a decoding method may be implemented in an architectural form as shown in FIG. 10 .
  • a computing device 3000 may include a bus 3010, one or more CPUs 3020, read only memory (ROM) 3030, random access memory (RAM) 3040, communication ports 3050 to a network, input/output components 3060, hard disk 3070, etc.
  • the storage device in the computing device 3000 such as the ROM 3030 or the hard disk 3070, can store various data or files used in the processing and/or communication of the encoding method or decoding method provided in the present disclosure and program instructions executed by the CPU.
  • Computing device 3000 may also include user interface 3080 .
  • the architecture shown in FIG. 10 is only exemplary, and one or more components in the computing device shown in FIG. 10 may be omitted according to actual needs when implementing different devices.
  • the above-mentioned computing device 3000 may be implemented as a computer installed with a design simulation application program for integrated circuits, which is not limited here.
  • the computing device 3000 may be implemented as a codec as shown in FIG. 2 or FIG. 3 to implement the encoding method or decoding method according to the present disclosure.
  • FIG. 11 shows a schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the disclosure.
  • Computer readable storage media include, but are not limited to, for example, volatile memory and/or nonvolatile memory.
  • the volatile memory may include random access memory (RAM) and/or cache memory (cache), etc., for example.
  • Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, and the like.
  • the computer-readable storage medium 4020 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions 4010 stored on the computer-readable storage medium 4020, the encoding method as described above can be performed or decoding method.
  • a computer program product or computer program comprising computer readable instructions stored in a computer readable storage medium.
  • the processor of the computer device can read the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computer device executes the encoding method or the decoding method described in the foregoing embodiments.
  • bit-depth transformation processing By using the encoding method, decoding method, computing device, and medium for video data provided by the embodiments of the present disclosure, it is possible to perform bit-depth transformation processing on a picture before encoding the picture, and then encode the generated transformed picture to form a coded picture of the picture.
  • bit-depth transformation information is also generated for transmission in the bitstream together with the encoding information, so that the decoder can process the bit-depth of the decoded picture accordingly according to the bit-depth transformation information parsed in the bitstream.
  • bit depth transformation information used to form the bit stream is beneficial to realize the bit depth transformation process more flexibly, and at the same time, it can realize the information communication about the bit depth transformation between the decoding end and the encoding end.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/CN2022/100950 2021-06-29 2022-06-24 用于视频数据的编码方法、解码方法、计算设备和介质 Ceased WO2023274044A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP22831858.0A EP4366305A4 (en) 2021-06-29 2022-06-24 VIDEO DATA CODING METHOD, VIDEO DATA DECODING METHOD, COMPUTER DEVICE AND MEDIUM
KR1020247002827A KR20240026202A (ko) 2021-06-29 2022-06-24 비디오 데이터를 위한 인코딩 방법, 비디오 데이터를 위한 디코딩 방법, 컴퓨팅 디바이스, 및 매체
JP2023580579A JP2024524397A (ja) 2021-06-29 2022-06-24 ビデオデータのための符号化方法、復号方法、計算装置及び媒体
US18/574,357 US20240340452A1 (en) 2021-06-29 2022-06-24 Encoding mehtod for video data, decoding method for video data, computing device, and medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110723894.7 2021-06-29
CN202110723894.7A CN115550666A (zh) 2021-06-29 2021-06-29 用于视频数据的编码方法、解码方法、计算设备和介质

Publications (1)

Publication Number Publication Date
WO2023274044A1 true WO2023274044A1 (zh) 2023-01-05

Family

ID=84690054

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/100950 Ceased WO2023274044A1 (zh) 2021-06-29 2022-06-24 用于视频数据的编码方法、解码方法、计算设备和介质

Country Status (6)

Country Link
US (1) US20240340452A1 (https=)
EP (1) EP4366305A4 (https=)
JP (1) JP2024524397A (https=)
KR (1) KR20240026202A (https=)
CN (1) CN115550666A (https=)
WO (1) WO2023274044A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116320431B (zh) * 2023-03-20 2024-03-01 深圳市与众文化传播有限公司 视频压缩编码数据动态无线网络传输系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5027171B2 (ja) * 2009-02-25 2012-09-19 日本電信電話株式会社 画像符号化方法，画像符号化装置および画像符号化プログラム
JP2017028584A (ja) * 2015-07-24 2017-02-02 日本電信電話株式会社 適応量子化方法、適応量子化装置及び適応量子化プログラム
CN110662061A (zh) * 2018-06-29 2020-01-07 想象技术有限公司 有保证的数据压缩
US20210084334A1 (en) * 2018-02-14 2021-03-18 Sony Corporation Image processing apparatus and image processing method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3111416A1 (en) * 2014-02-26 2017-01-04 Thomson Licensing Method and apparatus for encoding and decoding hdr images
GB201408618D0 (en) * 2014-05-15 2014-06-25 Univ Warwick Compressing high dynamic range images
US20160286226A1 (en) * 2015-03-24 2016-09-29 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
EP3588964A1 (en) * 2018-06-26 2020-01-01 InterDigital VC Holdings, Inc. Metadata translation in hdr distribution
JP7615036B2 (ja) * 2019-03-19 2025-01-16 インテル コーポレイション イマーシブビデオ符号化のための高レベルシンタックス
US11509897B2 (en) * 2020-08-07 2022-11-22 Samsung Display Co., Ltd. Compression with positive reconstruction error

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5027171B2 (ja) * 2009-02-25 2012-09-19 日本電信電話株式会社 画像符号化方法，画像符号化装置および画像符号化プログラム
JP2017028584A (ja) * 2015-07-24 2017-02-02 日本電信電話株式会社 適応量子化方法、適応量子化装置及び適応量子化プログラム
US20210084334A1 (en) * 2018-02-14 2021-03-18 Sony Corporation Image processing apparatus and image processing method
CN110662061A (zh) * 2018-06-29 2020-01-07 想象技术有限公司 有保证的数据压缩

Also Published As

Publication number Publication date
US20240340452A1 (en) 2024-10-10
CN115550666A (zh) 2022-12-30
KR20240026202A (ko) 2024-02-27
JP2024524397A (ja) 2024-07-05
EP4366305A4 (en) 2024-11-27
EP4366305A1 (en) 2024-05-08

Similar Documents

Publication Publication Date Title
TWI862578B (zh) 適應性迴路濾波器組之索引發信
CN113508589B (zh) 视频译码中的组合残差译码
US12075034B2 (en) Multiple adaptive loop filter sets
JP7633174B2 (ja) 変換スキップモードのための係数コーディング
CN113728629B (zh) 视频译码中的运动向量推导
CN113940069A (zh) 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令
CN116508321A (zh) 视频译码期间基于联合分量神经网络的滤波
TW202143717A (zh) 視頻譯碼中的亮度映射與色度縮放（lmcs）
TWI877192B (zh) 用於視訊寫碼之色度內預測單元
TWI826487B (zh) 用於視訊寫碼中之適應性運動向量差解析度及增加的運動向量儲存精確度的運動向量捨位
TW202135531A (zh) 用於視訊編解碼的經解碼圖片緩衝器（dpb）參數訊號傳遞通知
TW202110182A (zh) 視訊寫碼中用於變換略過模式及調色板模式之最小允許量化參數
JP2022521809A (ja) ビデオコーディングにおける係数領域ブロック差分パルスコード変調
TWI879773B (zh) 用於視訊寫碼中之自適應迴路濾波器之剪切索引寫碼
CN110754085B (zh) 用于非4:4:4格式视频内容的颜色重映射
TW202029754A (zh) 用於置零轉換之掃描及最後係數位置寫碼
CN114503590A (zh) 用信号发送针对变换跳过中的残差值的译码方案以进行视频译码
CN114402602B (zh) 用于视频编解码的算术编解码器字节填料信令
TW202119820A (zh) 視訊寫碼中之色度差量量化參數
TW202029774A (zh) 藉由轉換域濾波之量化假影抑制及信號恢復
CN111758255A (zh) 用于视频编解码的位置相关空间变化变换
TWI898055B (zh) 用於視頻譯碼中的跨分量線性模型（ｃｃｌｍ）模式的固定位元深度處理
TW202041018A (zh) 預測係數寫碼
EP4366305A1 (en) Encoding method for video data, decoding method for video data, computing device, and medium
KR20220146469A (ko) 결합된 픽셀/변환 기반 양자화에 의한 비디오 코딩을 위한 양자화 파라미터 제어 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22831858

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023580579

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20247002827

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 1020247002827

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 2022831858

Country of ref document: EP

Ref document number: 2024101684

Country of ref document: RU

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022831858

Country of ref document: EP

Effective date: 20240129

WWP Wipo information: published in national office

Ref document number: 2024101684

Country of ref document: RU

WWG Wipo information: grant in national office

Ref document number: 2024101684

Country of ref document: RU