TW201424393A - Hypothetical reference decoder parameters in video coding - Google Patents

Abstract

A computing device selects, from among a set of hypothetical reference decoder (HRD) parameters in a video parameter set and a set of HRD parameters in a sequence parameter set, a set of HRD parameters applicable to a particular operation point of a bitstream. The computing device performs, based at least in part on the set of HRD parameters applicable to the particular operation point, an HRD operation on a bitstream subset associated with the particular operation point.

Description

Hypothetical reference decoder parameters in video writing code

The present application claims the benefit of U.S. Provisional Patent Application Serial No. 61/705, filed on Sep. 24, 2012, the entire disclosure of which is hereby incorporated by reference.

The present invention relates to video encoding and decoding (i.e., encoding and/or decoding of video data).

Digital video capabilities can be incorporated into a wide range of devices, including digital TVs, digital live systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-books. Readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones, so-called "smart phones", video teleconferencing devices, video streaming devices, and Similar. Digital video devices implement video compression techniques, such as those defined by MPEG-2, MPEG-4, ITU-T H.263, and ITU-T H.264/MPEG-4 Part 10 (Advanced Video Recording (AVC)). Standards, Efficient Video Recording (HEVC) standards currently under development, and their techniques described in the extension of these standards. Video devices can transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video compression techniques.

Video compression technology performs spatial (intra-image) prediction and/or temporal (inter-image) prediction to Reduce or remove the redundancy inherent in video sequences. For block-based video writing, a video block (ie, a portion of a video frame or video frame) can be segmented into video blocks. The video blocks in the in-frame write code (I) tile of the image are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same image. The video blocks in the inter-frame code (P or B) block of the image may use spatial prediction with respect to reference samples in adjacent blocks in the same image or relative reference samples in other reference images. Time prediction. An image may be referred to as a frame, and a reference image may be referred to as a reference frame.

Spatial prediction or temporal prediction results in a predictive block for the block of code to be written. The residual data represents the pixel difference between the original block and the predictive block of the code to be written. The inter-frame write code block is encoded according to a motion vector of a block directed to a reference sample forming a predictive block and a residual data indicating a difference between the coded block and the predictive block. The code block in the frame is coded according to the code writing mode and the residual data in the frame. For further compression, the residual data can be transformed from the pixel domain to the transform domain, causing residual coefficients, which can then be quantized. The quantized coefficients originally configured into a two-dimensional array can be scanned to produce a one-dimensional vector of coefficients, and an entropy write code can be applied to achieve even more compression.

The multiview write code bitstream can be generated by encoding, for example, views from multiple viewpoints. Some three-dimensional (3D) video standards have been developed that utilize multi-view coding patterns. For example, different views can transmit left and right eye views to support 3D video. Alternatively, some 3D video code writing programs may apply so-called multi-view plus depth writing. In multi-view plus depth writing, the 3D video bit stream may contain not only texture view components but also depth view components. For example, each view can include one texture view component and one depth view component.

In general, the present invention describes signaling and selecting hypothetical reference decoder (HRD) parameters in a video code. More specifically, a computing device can be derived from a set of hypothetical reference decoder (HRD) parameters in a video parameter set (VPS) and a sequence of parameter sets (SPS) A set of HRD parameters selected for a particular operating point of one of the meta-streams is selected from a set of HRD parameters. The computing device performs an HRD operation on a subset of the bitstreams associated with the particular operating point based at least in part on the set of HRD parameters applicable to the particular operating point.

In one example, the invention describes a method of processing video material. The method includes selecting a set of HRD parameters suitable for a particular operating point of one of the one-bit streams from a set of HRD parameters in a VPS and a set of HRD parameters in an SPS. The method also includes performing an HRD operation on the subset of bitstreams associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point.

In another example, the invention features a device comprising one or more processors configured to assemble from one of a set of HRD parameters in a VPS and one of an HRD parameter in an SPS A set of HRD parameters in a set that applies to a particular operating point of one of the meta-streams is selected. The one or more processors are also configured to perform an HRD operation on a subset of the bitstreams associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point.

In another example, the present invention describes a device that includes a set of HRD parameters in a VPS and a set of HRD parameters in an SPS selected for a particular operating point of a bit stream A component of a collection of one of the HRD parameters. The device also includes means for performing an HRD operation on a subset of the bitstreams associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point.

In another example, the invention features a computer readable data storage medium having instructions stored thereon that, when executed by one or more processors of a device, configure the device from an HRD parameter in a VPS A set of HRD parameters in one of the sets and one of the SPSs in the SPS selects one of a set of HRD parameters that apply to a particular operating point of the one-bit stream. Moreover, the instructions configure the device at execution time to associate a subset of bitstreams associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point Perform an HRD operation.

The details of one or more embodiments of the invention are set forth in the description Other features, objectives, and advantages will be apparent from the description, drawings, and claims.

10‧‧‧Instance Video Recording System

12‧‧‧ source device

14‧‧‧ Destination device

16‧‧‧ channel

18‧‧‧Video source

20‧‧‧Video Encoder

21‧‧‧Additional devices

22‧‧‧Output interface

28‧‧‧Input interface

30‧‧‧Instance Video Decoder

32‧‧‧Display devices

100‧‧‧Predictive Processing Unit

102‧‧‧Residual generating unit

104‧‧‧Transformation Processing Unit

106‧‧‧Quantification unit

108‧‧‧Anti-quantization unit

110‧‧‧Inverse Transform Processing Unit

112‧‧‧Reconstruction unit

114‧‧‧Filter unit

116‧‧‧Decoded Image Buffer

118‧‧‧Entropy coding unit

120‧‧‧Inter-frame prediction processing unit

122‧‧‧Sports Estimation Unit

124‧‧‧Motion compensation unit

126‧‧‧ In-frame predictive processing unit

150‧‧‧ Entropy decoding unit

151‧‧‧Coded Image Buffer (CPB)

152‧‧‧Predictive Processing Unit

154‧‧‧Anti-quantization unit

156‧‧‧ inverse transform processing unit

158‧‧‧Reconstruction unit

160‧‧‧Filter unit

162‧‧‧Decoded Image Buffer

164‧‧ sports compensation unit

166‧‧‧In-frame prediction processing unit

200‧‧‧Instance operations

250‧‧‧Instance operations

300‧‧‧Instance HRD operations

1 is a block diagram illustrating an example video write code system that may utilize the techniques described in this disclosure.

2 is a block diagram illustrating an example video encoder that can implement the techniques described in this disclosure.

3 is a block diagram illustrating an example video decoder that can implement the techniques described in this disclosure.

4 is a flow chart illustrating an example operation of a device in accordance with one or more techniques of the present invention.

Figure 5 is a flow diagram illustrating an example operation of a device in accordance with one or more techniques of the present invention.

6 is a flow diagram illustrating an example hypothetical reference decoder (HRD) operation of a device in accordance with one or more techniques of the present invention.

The video encoder can generate a stream of bitstreams including encoded video material. A bitstream may contain a series of Network Abstraction Layer (NAL) units. The NAL unit of the bitstream may include a Video Write Code Layer (VCL) NAL unit and a non-VCL NAL unit. The VCL NAL unit may include a coded tile of the image. Non-VCL NAL units may include Video Parameter Sets (VPS), Sequence Parameter Sets (SPS), Picture Parameter Sets (PPS), Supplemental Enhancement Information (SEI), or other types of data. A VPS system may contain syntax structures for syntax elements that are applied to zero or more complete coded video sequences. The SPS system may contain syntax structures for syntax elements that are applied to zero or more complete coded video sequences. Single VPS Can be applied to multiple SPS. The PPS system may contain syntax structures for syntax elements that are applied to zero or more complete coded images. A single SPS can be applied to multiple PPSs. In general, various aspects of VPS, SPS, and PPS can be formed as defined by the HEVC standard.

Devices such as content delivery network (CDN) devices, media-aware network elements (MANEs), or video decoders can extract sub-bitstreams from a bitstream. The device may perform a sub-bitstream extraction procedure by removing certain NAL units from the bitstream. The resulting sub-bitstream includes the remaining unremoved NAL units of the bitstream. As an example, video data decoded from a sub-bitstream may have a lower frame rate and/or may represent fewer views than the original bitstream.

The video writing standard can include various features to support the sub-bit stream extraction procedure. For example, the video data of the bit stream can be divided into a collection of layers. For each of the layers, the data in the lower layer can be decoded without reference to the data in any of the higher layers. Individual NAL units only encapsulate data for a single layer. Thus, the NAL unit of the data of the highest remaining layer of the encapsulated bit stream can be removed from the bit stream without affecting the decodability of the data in the remaining lower layers of the bit stream. In Adjustable Video Recording (SVC), the higher layer may include enhanced data that improves the quality of the image in the lower layer (quality tunability) and magnifies the spatial format of the image in the lower layer (spatial adjustability), or increase the time rate (time adjustability) of images in lower layers. In multi-view code (MVC) and three-dimensional video (3DV) code writing, higher layers may include additional views.

The NAL unit can include a header and a payload. The header of the NAL unit includes the nuh_reserved_zero_6bits syntax element. If the NAL unit is related to a multi-view write code, a 3DV write code, or a base layer in SVC, the nuh_reserved_zero_6bits syntax element of the NAL unit is equal to zero. The data in the base layer of the bitstream can be decoded without reference to data in any other layer of the bitstream. The nuh_reserved_zero_6bits syntax element may have a non-zero value if the NAL unit is not a base layer in multiview code, 3DV or SVC. Specifically, if the NAL unit is not related to multiview code, 3DV or SVC The base layer, the nuh_reserved_zero_6bits syntax element of the NAL unit specifies the layer identifier of the NAL unit.

In addition, some of the images within the layer may be decoded without reference to other images within the same layer. Thus, the NAL units of the material of certain images of the encapsulation layer can be removed from the bit stream without affecting the decodability of other images in the layer. For example, an image with even POC values can be decoded without reference to an image with odd image order count (POC) values. Removing the NAL unit that encapsulates the data of such images reduces the frame rate of the bit stream. Herein, a subset of images within a layer that may be decoded without reference to other images within the layer may be referred to as sub-layers.

A NAL unit may include a temporal_id syntax element. The temporal_id syntax element of the NAL unit specifies the time identifier of the NAL unit. If the time identifier of the first NAL unit is less than the time identifier of the second NAL unit, the data encapsulated by the first NAL unit may be decoded without reference to the material encapsulated by the second NAL unit.

Each operating point of the bitstream is associated with a set of layer identifiers (i.e., a set of nuh_reserved_zero_6bits values) and a time identifier. The set of layer identifiers can be represented as OpLayerIdSet, and the time identifier can be represented as TemporalID. If the layer identifier of the NAL unit is in the layer identifier set of the operating point and the time identifier of the NAL unit is less than or equal to the time identifier of the operating point, the NAL unit is associated with the operating point. The operating point represents a subset of bitstreams associated with the operating point. An operational point representation can include each NAL unit associated with an operating point. The operating point representation does not include VCL NAL units that are not associated with the operating point.

An external source can specify a collection of target layer identifiers for the operating point. For example, a device such as a CDN device or a MANE can specify a set of target layer identifiers. In this example, the device can use a collection of target layer identifiers to identify the operating point. The device can then extract the operating point representation of the operating point and forward the operating point representation instead of the original bit stream to the client device. Extracting the operand representation and forwarding the operand representation to the client device reduces the bitstream Bit rate.

In addition, the video coding standard specifies a video buffer model. The video buffer model can also be referred to as a "hypothetical reference decoder" or "HRD". The HRD describes how the data is buffered for decoding and how the decoded data is buffered for output. For example, the HRD describes the operation of a coded image buffer ("CPB") and a decoded image buffer ("DPB") in a video decoder. The CPB is a FIFO buffer containing access units in the decoding order specified by the HRD. The DPB is a buffer that holds the decoded image for reference, output reordering, or output delay specified by the HRD.

The video encoder can send a collection of HRD parameters. The HRD parameters control various aspects of the HRD. The HRD parameters include initial CPB removal delay, CPB size, bit rate, initial DPB output delay, and DPB size. These HRD parameters are coded in the hrd_paramaters( ) syntax structure specified in the VPS and/or SPS. The HRD parameter can also be specified in a Buffer Period Supplemental Enhancement Information (SEI) message or an image timing SEI message.

As explained above, the operating point representation may have a different frame rate and/or bit rate than the original bit stream. This is because some of the images and/or materials in the original bitstream may not be included because of the operating point representation. Therefore, if the video decoder is to process the original bit stream, it will remove the data from the CPB and/or DPB at a specific rate, and if the video decoder indicates the processing operation point, it will be removed from the CPB and/or DPB at the same rate. Data, the video decoder may remove too much or too little data from the CPB and / or DPB. Thus, the video encoder can signal different sets of HRD parameters for different operating points. In the emerging High Efficiency Video Recording (HEVC) standard, a video encoder can transmit a set of HRD parameters in a VPS, or a video encoder can send a set of HRD parameters in an SPS. The draft of the forthcoming HEVC standard called "HEVC Working Draft 8" is described in Bross et al. "High Efficiency Video Coding (HEVC) Text Specification Draft 8" (ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11) The Joint Collaboration Team on Video Writing (JCT-VC), the 10th meeting in Stockholm, Sweden, July 2012, since May 8, 2013 This working draft is available at http://phenix.int-evry.fr/jct/doc_end_user/documents/10_Stockholm/wg11/JCTVC-J1003-v8.zip.

In some versions of HEVC, only a few sets of HRD parameters in the VPS are selected for HRD operations. That is, although the HRD parameters can be provided in the SPS, the set of HRD parameters in the SPS is not selected by the HEVC video decoder for HRD operations. The video decoder always parses and decodes the VPS of the bitstream. Therefore, the video decoder always parses and decodes the set of HRD parameters of the VPS. This is true whether the bit stream includes non-base layer NAL units. Thus, if the bitstream includes non-base layer NAL units, then parsing and handling the set of HRD parameters in the SPS can be a waste of computing resources. Furthermore, if a set of HRD parameters is present in the VPS, the set of HRD parameters in the SPS can be a wasted bit.

In accordance with the teachings of the present invention, a video encoder can generate a bit stream that includes an SPS suitable for use in a sequence of images. The SPS includes a collection of HRD parameters. The set of HRD parameters applies to each of the operation points of the set of layer identifiers of the bitstream that match the set of target layer identifiers. Therefore, the set of HRD parameters in the SPS is not wasted, but can be used for HRD operations. For example, the device may select a set of HRD parameters suitable for a particular operating point from a set of HRD parameters in the VPS and a set of HRD parameters in the SPS. The device may perform a bitstream conformance test based at least in part on a set of HRD parameters applicable to a particular operating point, the test testing whether a subset of bitstreams associated with a particular operating point is consistent with a video writing standard.

A device such as a video encoder, video decoder, or another type of device, such as a CDN device or MANE, can perform bitstream conformance testing on the operating point representation of the operating point. The bitstream conformance test verifies that the operating point representation is consistent with a video writing standard such as HEVC. As mentioned above, the set of target layer identifiers and the time identifier can be used to identify the operating point. The set of target layer identifiers can be represented as "TargetDecLayerIdSet". The time identifier can be expressed as "TargetDecHighestTid". Problemly, HEVC Working Draft 8 does not specify how to set the TargetDecLayerIdSet or TargetDecHighestTid when performing a bitstream conformance test.

In accordance with one or more techniques of the present invention, a device can perform a decoding process as part of performing a bitstream conformance test. Executing the decoding program includes executing a bitstream stream extractor to decode the operand representation of the operating point defined by the target set of the layer identifier and the target highest time identifier from the bitstream. The target set of layer identifiers (ie, TargetDecLayerIdSet) contains the value of the layer identifier syntax element (eg, nuh_reserved_zero_6bits syntax element) present in the operation point representation. The target set of layer identifiers is a subset of the values of the layer identifier syntax elements of the bit stream. The target highest time identifier (ie, TargetDecHighestTid) is equal to the maximum time identifier present in the operating point representation. The target highest time identifier is less than or equal to the maximum time identifier present in the bit stream. The execution of the decoding program may also include the NAL unit represented by the decoding operation point.

In HEVC, the SPS may include an array of syntax elements denoted sps_max_dec_pic_buffering[i], where i is in the range of the maximum number of temporal layers in the 0-bit stream. When the highest time identifier (HighestTid) is equal to i , sps_max_dec_pic_buffering[i] indicates the maximum required size of the DPB. Sps_max_dec_pic_buffering[i] indicates the required size of the unit according to the image storage buffer.

Furthermore, in HEVC, the SPS may include an array of syntax elements denoted as sps_max_num_reorder_pics[i], where i is in the range of the maximum number of temporal layers in the 0-bit stream. Sps_max_num_reorder_pics[i] indicates the maximum allowable number of images preceding the image in the decoding order and the image following the image in the output order when the highest time identifier (HighestTid) is equal to i .

In HEVC, the set of HRD parameters may include an array of syntax elements denoted cpb_cnt_minus1[i], where i is in the range of the maximum number of time layers in the 0-bit stream. Cpb_cnt_minus1[i] specifies the number of alternative CPB specifications in the bitstream of the coded video sequence when the highest time identifier (HighestTid) is equal to i , where an alternative CPB specification refers to a particular set having CPB parameters Specific CPB operation.

In HEVC Working Draft 8, sps_max_dec_pic_buffering[i], sps_max_num_reorder_pics[i], and cpb_cnt_minus1[i] are not properly selected in HRD operations, bitstream consistency operations, and level restrictions. This is due at least in part to the fact that HEVC Working Draft 8 does not specify the meaning of the highest time identifier (HighestTid).

In accordance with one or more techniques of the present invention, a device such as a video encoder, video decoder or another device can determine the highest temporal identifier of the subset of bitstreams associated with the selected operating point of the bitstream. In addition, the device may determine a particular syntax element based on the highest time identifier from an array of syntax elements (eg, sps_max_dec_pic_buffering[ ], sps_max_num_reorder_pics[ ], or cpb_cnt_minus1[ ]). The device may perform an operation that uses a particular syntax element to determine the consistency of the bitstream with the video writing standard or to determine the consistency of the video decoder with the video writing standard.

1 is a block diagram illustrating an example video write code system 10 that may utilize the techniques of the present invention. As used herein, the term "video codec" refers broadly to both video encoders and video decoders. In the present invention, the term "video writing code" or "writing code" may generally refer to video encoding or video decoding.

As shown in FIG. 1, video write code system 10 includes source device 12 and destination device 14. Source device 12 produces encoded video material. Thus, source device 12 can be referred to as a video encoding device or a video encoding device. Destination device 14 can decode the encoded video material generated by source device 12. Thus, destination device 14 may be referred to as a video decoding device or a video decoding device. Source device 12 and destination device 14 may be examples of video code writing devices or video code writing devices.

Source device 12 and destination device 14 can include a wide range of devices, including desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablets, set-top boxes, such as the so-called "wisdom Phone, mobile phone, TV, camera, Display device, digital media player, video game console, car computer or the like.

Destination device 14 may receive encoded video material from source device 12 via channel 16. Channel 16 may include one or more media or devices capable of moving encoded video material from source device 12 to destination device 14. In one example, channel 16 can include one or more communication media that enable source device 12 to transmit encoded video material directly to destination device 14 in an instant. In this example, source device 12 can modulate the encoded video material in accordance with a communication standard, such as a wireless communication protocol, and can transmit the modulated video data to destination device 14. The one or more communication media may include wireless and/or wired communication media such as a radio frequency (RF) spectrum or one or more physical transmission lines. One or more communication media may form part of a packet-based network such as a regional network, a wide area network, or a global network (eg, the Internet). The one or more communication media may include a router, a switch, a base station, or other device that facilitates communication from the source device 12 to the destination device 14.

In another example, channel 16 can include a storage medium that stores encoded video material generated by source device 12. In this example, destination device 14 can access the storage medium, for example, via disk access or card access. The storage medium may include a variety of locally accessed data storage media such as Blu-ray Disc, DVD, CD-ROM, flash memory or other suitable digital storage medium for storing encoded video material.

In other examples, channel 16 may include a file server or another intermediate storage device that stores encoded video material generated by source device 12. In this example, destination device 14 may access encoded video material stored at a file server or other intermediate storage device via streaming or downloading. The file server can be a type of server capable of storing encoded video material and transmitting the encoded video data to destination device 14. The example file server includes a web server (for example, for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive. In the example of FIG. 1, channel 16 includes additional devices 21. In some examples, the additional device 21 is a CDN device, a MANE, or another type of device.

Destination device 14 can access the encoded video material via a standard data connection, such as an internet connection. Example types of data connections may include wireless channels (eg, Wi-Fi connections), wired connections (eg, digital subscriber line (DSL), cable modems, etc.), or both suitable for accessing stored on a file server A combination of encoded video material. The transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a combination of the two.

The techniques of the present invention are not limited to wireless applications or settings. The technology can be applied to video writing supports for a variety of multimedia applications, such as aerial television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (eg, via the Internet), encoding video data for storage on a data storage medium, Decode video data stored on the data storage medium, or other applications. In some examples, video code writing system 10 can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

1 is merely an example, and the techniques of the present invention are applicable to video code setting (eg, video encoding or video decoding) that does not necessarily include any data communication between the encoding device and the decoding device. In other examples, the data is retrieved from the local memory, streamed over the network, or the like. The video encoding device can encode the data and store the data to the memory, and/or the video decoding device can retrieve the data from the memory and decode the data. In many instances, encoding and decoding are performed by devices that do not communicate with each other, but only encode data to the memory and/or retrieve data from the memory and decode the data.

In the example of FIG. 1, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some examples, output interface 22 can include a modulator/demodulation transformer (data machine) and/or a transmitter. The video source 18 may include, for example, a video capture device of a video camera, a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and/or a computer graphics system for generating video data, or A combination of sources of such video material.

Video encoder 20 can encode video material from video source 18. In some examples, source device 12 transmits encoded video material directly to destination device 14 via output interface 22. In other examples, the encoded video material may also be stored on a storage medium or on a file server for later access by the destination device 14 for decoding and/or playback.

In the example of FIG. 1, destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some examples, input interface 28 includes a receiver and/or a data machine. Input interface 28 can receive encoded video material via channel 16. Display device 32 may be integrated with destination device 14, or may be external to destination device 14. In general, display device 32 displays the decoded video material. Display device 32 can include a variety of display devices such as liquid crystal displays (LCDs), plasma displays, organic light emitting diode (OLED) displays, or another type of display device.

Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), special application integrated circuits (ASICs), field-receivable Stylized gate array (FPGA), discrete logic, hardware, or any combination thereof. If the techniques are implemented partially in software, a device can store instructions for the software in a suitable stable computer readable storage medium, and one or more processors can be used in the hardware to perform the The instructions are executed to carry out the techniques of the present invention. Any of the foregoing (including hardware, software, a combination of software and hardware, etc.) can be considered one or more processors. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated into a combined encoder/decoder (codec) in a respective device Part of (CODEC)).

The present invention may generally relate to video encoder 20 "sending" certain information to another device (such as video decoder 30 or additional device 21). The term "sending" may generally refer to communications used to decode syntax elements and/or other materials of compressed video material. This communication can happen instantly or almost instantaneously. Alternatively, this communication may occur over a period of time, such as possibly storing the syntax elements in computer-readable storage in the encoded bitstream when encoding When the media occurs, the syntax elements can then be retrieved by the decoding device at any time after being stored to the media.

In some examples, video encoder 20 and video decoder 30 are based on, for example, ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC) (including its adjustable video writing code). Video compression standard operation of (SVC) extension, multiview video code (MVC) extension and/or MVC based 3DV extension. In some cases, any bitstream consistent with MVC-based 3DV always contains a sub-bitstream that conforms to the MVC profile (eg, stereo high profile). In addition, efforts are being made to generate a three-dimensional video (3DV) write code extension of H.264/AVC (i.e., AVC-based 3DV). In other examples, video encoder 20 and video decoder 30 may be in accordance with ITU-T H.261, ISO/IEC MPEG-1 Visual, ITU-T H.262, or ISO/IEC MPEG-2 Visual, ITU-T H. .263, ISO/IEC MPEG-4 Visual and ITU-T H.264, ISO/IEC Visual Operations.

In other examples, the video encoder 20 and the video decoder 30 may be based on a joint collaboration between the ITU-T Video Recording Professionals Group (VCEG) and the ISO/IEC Animation Professional Group (MPEG) on video writing (JCT). -VC) Developed by the High Efficiency Video Recording (HEVC) standard. The draft of the forthcoming HEVC standard called "HEVC Working Draft 9" is described in Bross et al. "High Efficiency Video Coding (HEVC) Text Specification Draft 9" (October 2012, Shanghai, China, ITU-T SG16 WP3 and ISO) /IEC JTC1/SC29/WG11 at the 11th meeting of the Joint Collaboration Team on Video Code Writing (JCT-VC), the working draft is available at http://phenix.int-evry as of May 8, 2013 Available at .fr/jct/doc_end_user/documents/11_Shanghai/wg11/JCTVC-K1003-v13.zip. In addition, efforts are being made to generate SVC, multiview code, and 3DV extensions for HEVC. The 3DV extension of HEVC can be referred to as HEVC based 3DV or 3D-HEVC.

In HEVC and other video writing standards, video sequences typically include a series of images. An image can also be called a "frame." The image may include three sample arrays denoted as S _L , S _{Cb ,} and S _Cr . The S _L is a two-dimensional array of brightness samples (ie, blocks). A two-dimensional array of S _Cb Cb chromaticity samples. A two-dimensional array of S _Cr Cr color samples. The chroma samples in this article can also be referred to as "chroma" samples. In other examples, the image may be monochromatic and may include only a luma sample array.

To generate an encoded representation of the image, video encoder 20 may generate a set of write code tree units (CTUs). Each of the CTUs may be a code tree block of the luma sample, two corresponding code tree blocks of the chroma samples, and a syntax structure for writing samples of the code tree block. The code tree block can be an N x N block of the sample. A CTU may also be referred to as a "tree block" or a "maximum code unit" (LCU). The CTU of HEVC can be broadly similar to macroblocks of other standards such as H.264/AVC. However, the CTU is not necessarily limited to a particular size and may include one or more code writing units (CUs). A tile may include an integer number of CTUs that are consecutively ordered in a raster scan.

To generate a coded CTU, video encoder 20 may recursively perform quadtree partitioning on the CTU's code tree block to divide the code tree block into code blocks (hence the name "code tree" Type unit"). The code block is an N x N block of the sample. The CU may be a code writing block having a luma sample of the brightness sample array, the Cb sample array, and the Cr sample array, and two corresponding code writing blocks of the chroma sample, and the sample for writing the code block. The syntax structure of the code. Video encoder 20 may partition the code block of the CU into one or more prediction blocks. The prediction block may be a rectangular (i.e., square or non-square) block to which the sample is applied. The prediction unit (PU) of the CU may be a prediction block of the luma sample, two corresponding prediction blocks of the chroma samples of the image, and a syntax structure for predicting the prediction block samples. Video encoder 20 may generate predictive luma, Cb, and Cr blocks for the luma, Cb, and Cr prediction blocks for each PU of the CU.

Video encoder 20 may use intra-frame prediction or inter-frame prediction to generate predictive blocks for the PU. If video encoder 20 uses in-frame prediction to generate a predictive block for the PU, video encoder 20 may generate a predictive block for the PU based on the decoded samples of the image associated with the PU.

If video encoder 20 uses inter-frame prediction to generate a predictive block for a PU, video encoder 20 may generate PU predictability based on decoded samples of one or more images that are different from the image associated with the PU. Block. Video encoder 20 may use one-way prediction or bi-directional prediction to generate predictive blocks for the PU. When video encoder 20 uses unidirectional prediction to generate predictive blocks for a PU, the PU may have a single motion vector. When video encoder 20 uses bi-directional prediction to generate predictive blocks for a PU, the PU may have two motion vectors.

After the video encoder 20 generates the predictive luma, Cb, and Cr blocks of one or more PUs of the CU, the video encoder 20 may generate a luma residual block of the CU. Each sample of the luma residual block of the CU indicates the difference between the luma sample in one of the predictive luma blocks of the CU and the corresponding sample in the original luma write block of the CU. In addition, video encoder 20 may generate a Cb residual block of the CU. Each sample of the Cb residual block of the CU may indicate a difference between a Cb sample in one of the CU's predictive Cb blocks and a corresponding sample in the original Cb code block of the CU. Video encoder 20 may also generate a Cr residual block of the CU. Each sample of the Cr residual block of the CU may indicate a difference between a Cr sample in one of the CU's predictive Cr blocks and a corresponding sample in the original Cr code block of the CU.

In addition, video encoder 20 may use quadtree partitioning to decompose the CU's luma, Cb, and Cr residual blocks into one or more luma, Cb, and Cr transform blocks. The transform block can be a rectangular block to which the same transform is applied. The transform unit (TU) of the CU may be a transform block of the luma sample, two corresponding transform blocks of the chroma sample, and a syntax structure for transforming the transform block sample. Thus, each TU of a CU can be associated with a luma transform block, a Cb transform block, and a Cr transform block. The luma transform block associated with the TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU.

Video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be pure quantities. Video encoder 20 may apply one or more transforms to the Cb transform region of the TU The block is to generate a Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

After generating a coefficient block (eg, a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to the process of quantizing transform coefficients to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode the syntax elements to indicate the quantized transform coefficients. For example, video encoder 20 may perform a context adaptive binary arithmetic write code (CABAC) on the syntax elements to indicate the quantized transform coefficients. Video encoder 20 may output the entropy encoded syntax elements in the bitstream.

Video encoder 20 may output a stream of bits comprising a sequence of bits that form a representation of the coded image and associated material. The bitstream may contain a sequence of Network Abstraction Layer (NAL) units. The NAL unit may contain an indication of the subsequent data type and, if necessary, a syntax structure of the byte containing the material in the form of the original byte sequence payload (RBSP) interspersed in the simulation prevention byte. That is, each of the NAL units may include a NAL unit header and encapsulate the RBSP. The NAL unit header may include a syntax element indicating a NAL unit type code. The NAL unit type code specified by the NAL unit header of the NAL unit indicates the type of the NAL unit. An RBSP may be a grammatical structure containing an integer number of bytes encapsulated within a NAL unit. In some cases, the RBSP includes zero bits.

Different types of NAL units can encapsulate different types of RBSPs. For example, a NAL unit of a first type may encapsulate an RBSP of a picture parameter set (PPS), a NAL unit of a second type may encapsulate an RBSP of a coded block, and a NAL unit of a third type may encapsulate an SEI RBSP, and so on. The NAL unit of the RBSP (not the parameter set and the RBSP of the SEI message) encapsulating the video write data may be referred to as a Video Write Code Layer (VCL) NAL unit.

Video decoder 30 can receive the stream of bits generated by video encoder 20. In addition, video decoder 30 may parse the bitstream to decode syntax elements from the bitstream. Video decoder 30 may reconstruct an image of the video material based at least in part on the syntax elements of the bit stream decoding. The process of reconstructing the video data may be substantially reciprocal to the program executed by video encoder 20. For example, video decoder 30 may use the motion vector of the PU to determine the predictive block of the PU of the current CU. In addition, video decoder 30 may inverse quantize the transform coefficient block associated with the TU of the current CU. Video decoder 30 may perform an inverse transform on the transform coefficient block to reconstruct a transform block that is associated with the TU of the current CU. The video decoder 30 may reconstruct the code block of the current CU by adding a sample of the predictive block of the PU of the current CU to the corresponding sample of the transform block of the TU of the current CU. Video decoder 30 may reconstruct the reconstructed image by reconstructing the code blocks of each CU of the reconstructed image.

In a multiview code, there may be multiple views from different viewpoints of the same scene. The term "access unit" is used to refer to a collection of images corresponding to an individual performing at the same time. Thus, video data can be conceptualized as a series of access units that occur over time. A "view component" can be a coded representation of a view in a single access unit. In the present invention, a "view" may refer to a sequence of view components associated with the same view identifier.

Multi-view writing supports inter-view prediction. Inter-view prediction is similar to inter-frame prediction in H.264/AVC and HEVC, and the same syntax elements can be used. However, when the video codec performs inter-view prediction on the current video unit (such as a PU), the video encoder 20 can use the image in the same access unit as the current video unit but in a different view as a reference picture. image. In contrast, conventional inter-frame prediction uses only images from different access units as reference images.

In multi-view code writing, if a video decoder (eg, video decoder 30) can decode an image in a view without reference to an image in any other view, the view may be referred to as a "base view." When writing an image in one of the non-base views, if the image is in a different view from the image currently being written by the video writer, but the image is currently being coded by the video writer. Within the same time execution of the individual (i.e., access unit), a video writer (such as video encoder 20 or video decoder 30) can add the image to the reference image list. Similar to other inter-frame prediction reference images, the video codec can insert an inter-view prediction reference image at any position of the reference image list.

The video coding standard specifies the video buffer model. In H.264/AVC and HEVC, the buffer model is called "hypothetical reference decoder" or "HRD". In HEVC Working Draft 8, the HRD is described in Appendix C.

The HRD describes how the data is buffered for decoding and how the decoded data is buffered for output. For example, HRD describes the operation of a coded image buffer ("CPB"), a decoded image buffer ("DPB"), and a video decoder. The CPB is a FIFO buffer containing access units in the decoding order specified by the HRD. The DPB is a buffer that holds the decoded image for reference, output reordering, or HRD specified output delay. The behavior of CPB and DPB can be mathematically specified. HRD imposes constraints directly on timing, buffer size, and bit rate. In addition, HRD can impose constraints on various bitstream characteristics and statistics indirectly.

In H.264/AVC and HEVC, bitstream consistency and decoder consistency are specified as part of the HRD specification. In other words, the HRD model specifies a test to determine if the bitstream is consistent with the standard and specifies a test to determine if the decoder is consistent with the standard. Although HRD is named as a class of decoders, video encoders typically use HRD to ensure bitstream consistency, while video decoders typically do not require HRD.

Both H.264/AVC and HEVC specify two types of bitstream or HRD consistency (ie, Type I and Type II). The Type I bitstream stream only contains the VCL NAL unit of all access units in the bitstream and the NAL unit stream of the filler data NAL unit. The Type II bit stream system includes, in addition to the VCL NAL unit and the padding data NAL unit of all access units in the bit stream, a NAL unit stream of at least one of the following: different from the padding data NAL unit Additional non-VCL NAL units; and all leading_zero_8bits, zero_byte, start_coded_prefix_one_3bytes, and trailing_zero_8bits syntax elements forming a byte stream from the NAL unit stream.

The device may select the operating point of the bitstream when the device performs a bitstream conformance test that determines if the bitstream is consistent with the video writing standard. The device can then determine a set of HRD parameters that are applicable to the selected operating point. The device can configure the behavior of the HRD using a collection of HRD parameters suitable for the selected operating point. More specifically, the device can use the applicable set of HRD parameters to configure the behavior of particular components of the HRD, such as hypothetical stream scheduler (HSS), CPB, decoder, DPB, and the like. The HSS can then inject the coded video data of the bit stream into the CPB of the HRD according to a particular schedule.

In addition, the device can invoke a decoding process that decodes the coded video material in the CPB. The decoding program can output the decoded image to the DPB. As the device moves the data through the HRD, the device can determine if a particular set of constraints is still satisfied. For example, the device can determine whether an overflow or underrun condition occurs in the CPB or DPB while the HRD is decoding the operating point representation of the selected operating point. The device can select and process each operating point of the bitstream in this manner. If the operating point of the bit stream does not cause a violation of the constraint, the device can determine that the bit stream is consistent with the video writing standard.

Both H.264/AVC and HEVC specify two types of decoder coherency, namely output timing decoder consistency and output order decoder consistency. Decoders that claim consistency with a particular profile, level, and level are able to successfully decode all bitstreams consistent with bitstream conformance requirements such as HEVC's video coding standard. In the present invention, "profile" may refer to a subset of the meta-stream syntax. "Level" and "Level" can be specified in each profile. The level of the hierarchy may be a specified set of constraints imposed on the values of the syntax elements in the bitstream. These constraints can be a simple limit on the value. Alternatively, such constraints may take the form of a constraint on the arithmetic combination of values (eg, image width x image height x number of images decoded per second). The level specified for the lower level is more constrained than the level specified for the higher level.

When the device performs a decoder conformance test to determine if the D4.1 is consistent with the video write standard, the device can provide a bit stream consistent with the video write standard to Both HRD and DUT. The HRD can process the bitstream in the manner described above with respect to the bitstream conformance test. If the order of the decoded images output by the DUT matches the order of the decoded images output by the HRD, the device can determine that the DUT is consistent with the video writing standard. In addition, if the timing at which the DUT outputs the decoded image matches the timing of the decoded image of the HRD output, the device can determine that the DUT is consistent with the video writing standard.

In addition to bitstream conformance testing and decoder conformance testing, the device can use HRD parameters for other purposes. For example, an initial CPB removal delay can be used to guide the system to set an appropriate initial end-to-end delay, and when a video data bitstream is delivered via RTP, the DPB output time can be used to derive a Real Time Agreement (RTP) timestamp.

In the H.264/AVC and HEVC HRD models, decoding or CPB removal may be based on access units. That is, it is assumed that the HRD simultaneously decodes the complete access unit and removes the complete access unit from the CPB. Furthermore, in the H.264/AVC and HEVC HRD models, it is assumed that the image decoding is instantaneous. Video encoder 20 may signal the decoding time in the picture timing SEI message to begin decoding of the access unit. In practical applications, if a consistent video decoder strictly follows the decoding time sent to initiate decoding of the access unit, the earliest possible time to output a particular decoded image is equal to the decoding time of the particular image plus decoding. The time required for this particular image. However, in the real world, the time required to decode an image cannot be equal to zero.

The HRD parameters control various aspects of the HRD. In other words, the HRD can depend on the HRD parameters. The HRD parameters may include an initial CPB removal delay, a CPB size, a bit rate, an initial DPB output delay, and a DPB size. Video encoder 20 may signal these HRD parameters in the hrd_parameters( ) syntax structure specified in the Video Parameter Set (VPS) and/or Sequence Parameter Set (SPS). Individual VPSs and/or SPSs may include multiple hrd_parameters( ) syntax structures for different sets of HRD parameters. In some examples, video encoder 20 may signal HRD parameters in a buffering period SEI message or an image timing SEI message.

As explained above, the operating point of the bit stream is a set of layer identifiers (ie, The set of nuh_reserved_zero_6bits values is associated with the time identifier. An operational point representation can include each NAL unit associated with an operating point. The operating point representation may have a frame rate and/or a bit rate that is different from the original bit stream. This is because some of the images and/or materials in the original bitstream may not be included because of the operating point representation. Thus, if the video decoder 30 is to remove data from the CPB and/or DPB at a particular rate while processing the original bit stream, and if the video decoder 30 is to process the operating point representation at the same rate from the CPB and/or DPB When the data is removed, the video decoder 30 can remove too much or too little data from the CPB and/or DPB. Thus, video encoder 20 can signal different sets of HRD parameters for different operating points. For example, video encoder 20 may include a plurality of hrd_parameters( ) syntax structures in the VPS that include HRD parameters for different operating points.

In HEVC Working Draft 8, the set of HRD parameters needs to include a collection of information common to all sub-layers as needed. In other words, the set of HRD parameters can include a set of common syntax elements that are applicable to the operating points including any temporal sub-layers as desired. The temporal sub-layer is a time-adjustable layer of a time-adjustable bitstream, the time-adjustable bitstream consisting of VCL NAL units with associated values of TemporalId and associated non-VCL NAL units. In addition to the set of common information, the set of HRD parameters may include a collection of syntax elements specific to the individual temporal sub-layers. For example, the hrd_parameters( ) syntax structure may optionally include a collection of information that is common to all sub-layers and that always includes sub-layer specific information. Since the set of common information is common to multiple sets of HRD parameters, it is not necessary to send a set of common information in multiple sets of HRD parameters. Specifically, in the HEVC working draft 8, the common information may exist in the set of HRD parameters in the first set of HRD parameters in the set of VRD parameters, or the common information in the set of HRD parameters and the first An operation point index associated may exist in the set of HRD parameters. For example, when the hrd_parameters( ) syntax structure is the first hrd_parameters( ) syntax structure in the VPS or when the hrd_parameters( ) syntax structure is associated with the first operating point, the HEVC Working Draft 8 supports the existence of common information.

Table 1 below is an example syntax structure of the hrd_parameters( ) syntax structure in HEVC.

In the examples of Table 1 above and other grammar tables of the present invention, the syntax element having the type descriptor ue(v) may be the length encoded using the 0-order exponential Columb (Exp-Golomb) code from the left bit. Variable unsigned integer. In the examples of Table 1 and the following tables, the syntax elements of the descriptor having the form u( n ) (where n is a non-negative integer) have an unsigned value of length n .

In the example syntax of Table 1, the syntax elements in the "if(commonInfPresentFlag){...}" block are the common information of the HRD parameter set. In other words, a common set of HRD parameters of the information may include syntax elements timing_info_present_flag, num_units_in_tick, time_scale, nal_hrd_parameters_present_flag, vcl_hrd_parameters_present_flag, sub_pic_cpb_params_present_flag, tick_divisor_minus2, du_cpb_removal_delay_length_minus1, bit_rate_scale, cpb_size_scale, initial_cpb_removal_delay_length_minus1, cpb_removal_delay_length_minus1 and dpb_output_delay_length_minus1.

Furthermore, in the example of Table 1, the syntax elements fixed_pic_rate_flag[i], pic_duration_in_tc_minus1[i], low_delay_hrd_flag[i], and cpb_cnt_minus1[i] may be sets of sub-layer specific HRD parameters. In other words, such syntax elements of the hrd_parameters( ) syntax structure may only be applicable to operating points that include a particular sub-layer. Thus, the HRD parameters of the hrd_parameters( ) syntax structure may include a set of sub-layer specific HRD parameters specific to a particular sub-layer of the bit stream, in addition to the common information that needs to be included.

The fixed_pic_rate_flag[i] syntax element may indicate that when HighestTi is equal to i, the temporal distance between the HRD output times of any two consecutive images in the output order is constrained in a particular manner. The HighestTid may identify the variable of the highest temporal sub-layer (eg, of the operating point). The pic_duration_in_tc_minus1[i] syntax element may specify the time distance in hours of the HRD output time between any successive images in the output order in the coded video sequence when HighestTid is equal to i. The low_delay_hrd_flag[i] syntax element may specify the HRD mode of operation when HighestTid is equal to i, as specified in Appendix C of HEVC Working Draft 8. The cpb_cnt_minus1[i] syntax element may specify the number of alternative CPB specifications in the bitstream of the coded video sequence when HighestTid is equal to i.

Video encoder 20 may use SEI messages to include in the bitstream that are not required for the map Set the data after the correct decoding of the sample values. However, video decoder 30 or other device may use the post-data included in the SEI message for various other purposes. For example, the video decoder 30 can use the data in the SEI message for image output timing, image display, loss detection, and error concealment.

Video encoder 20 may include one or more SEI NAL units in the access unit. In other words, any number of SEI NAL units can be associated with an access unit. In addition, each SEI NAL unit may contain one or more SEI messages. The HEVC standard describes the syntax and semantics of various types of SEI messages. However, the HEVC standard does not describe the handling of SEI messages because the SEI message does not affect the standard decoding procedure. One reason for having several SEI messages in the HEVC standard is to enable interpretation of supplemental material equally in different systems using HEVC. The HEVC specification and system may require the video encoder to generate certain SEI messages or may define specific handling of a particular type of received SEI message. Table 2 below lists the SEI messages specified in HEVC and briefly describes their use.

The prior art for signaling HRD parameters and selecting HRD parameters and other parameters has several problems or disadvantages. For example, in HEVC Working Draft 8, only the set of HRD parameters in the VPS is selected for HRD operations. That is, although the HRD parameters can be provided in the SPS, the set of HRD parameters in the SPS is not selected by the HEVC video decoder for HRD operation. The video decoder always parses and decodes the VPS of the bitstream. Therefore, the video decoder always parses and decodes the set of HRD parameters of the VPS.

This is true regardless of whether the bit stream includes non-base layer NAL units. For example, the hrd_parameters( ) syntax structure that only writes code in the VPS can be selected for HRD operations, and the hrd_parameters( ) syntax structure that may exist in the SPS may never be selected. This situation may require profiling and handling of the VPS, even if the decoding does not contain a bit stream of nuh_reserved_zero_6bits greater than 0 (ie, the bit stream contains only HEVC multiview, 3DV or SVC extension base layer) Of course.

Thus, if the bitstream includes non-base layer NAL units, then parsing and handling the set of HRD parameters in the SPS can be a waste of computing resources. Furthermore, if a set of HRD parameters is present in the VPS, the set of HRD parameters in the SPS can be a waste of bits. For example, if the hrd_parameters( ) syntax structure exists in the SPS, the coded bits of the grammatical structure can be purely a waste of bits.

In accordance with one or more techniques of the present invention, video encoder 20 may generate a stream of bits comprising an SPS that is suitable for use in a sequence of images. The SPS includes a collection of HRD parameters. The set of HRD parameters applies to each operating point of the set of layer identifiers of the bitstream that match the set of target layer identifiers. Therefore, the set of HRD parameters in the SPS is not wasted, but can be used for HRD operations. For example, the operating point for the hrd_parameters( ) syntax structure written in the SPS can be clearly specified as, for example, only one value of nuh_reserved_zero_6bits in the bitstream (ie, multiview, 3DV, or Tuned video These operating points of the layer ID in the code extension.

For example, a device such as video encoder 20 or video decoder 30 may select a set of HRD parameters suitable for a particular operating point from a set of HRD parameters in the video parameter set and a set of HRD parameters in the SPS. In this example, the device can perform a bitstream conformance test based at least in part on a set of HRD parameters applicable to a particular operating point, whether the subset of bitstreams associated with a particular operating point is consistent with a video writing standard carry out testing. The bitstream conformance test verifies that the operating point representation is consistent with a video writing standard such as HEVC.

In the present invention, the operating point can be identified by a set of nuh_reserved_zero_6bits values represented as OpLayerIdSet and a TemporalId value represented as OpTid. In the case where OpTid and OpLayerIdSet are input as inputs, the associated subset of bitstreams derived as output of the sub-bitstream extraction procedure as specified in sub-clause 10.1 of HEVC Working Draft 8 can be independently decoded. Subclause 10.1 of HEVC Working Draft 8 describes an operation for extracting a sub-bitstream (i.e., an operand representation) from a bitstream. In particular, subclause 10.1 of HEVC Working Draft 8 provides that all of the layer identifiers (eg, nuh_reserved_zero_6bits) having a time identifier greater than tIdTarget (eg, TemporalID) or not a value in targetDecLayerIdSet are removed from the bitstream. The NAL unit is used to derive the sub-bit stream. tIdTarget and targetDecLayerIdSet are parameters of the bit stream extractor.

In another example problem or disadvantage of the prior art for signaling HRD parameters, a device such as a video encoder, video decoder or another type of device can perform bitstream conformance testing on an operating point representation of an operating point. . As mentioned above, the set of target layer identifiers and the time identifier can be used to identify the operating point. The set of target layer identifiers can be represented as "TargetDecLayerIdSet". The time identifier can be expressed as "TargetDecHighestTid". Problemly, HEVC Working Draft 8 does not specify how to set the TargetDecLayerIdSet or TargetDecHighestTid when performing a bitstream conformance test. For example, when calling decoding for bitstream conformance testing In the program, the semantics of the syntax elements are not clearly specified, because the values of TargetDecLayerIdSet and TargetDecHighestTid are not properly set.

One or more techniques of the present invention indicate how to set the TargetDecLayerIdSet and TargetDecHighestTid when performing a bitstream conformance test. For example, the general decoding procedure of the bitstream (or operand representation) is modified such that if the bitstream (or operand representation) is decoded in a bitstream conformance test, the TargetDecLayerIdSet is as subclause C of the HEVC standard. The settings specified in 1 are set. Similarly, the general decoding procedure for a bitstream (or operand representation) can be modified such that if a bitstream (or operand representation) is decoded in a bitstream conformance test, the TargetDecHighestTid is a subclause of HEVC Working Draft 8. Set as specified in C.1. In other words, the device can determine a target layer identifier set containing a specific operation point existing in each layer identifier of the bit stream subset, and the layer identifier set of the specific operation point is a layer identifier existing in the bit stream a subset of. In addition, the device can determine that the specific operating point is equal to the target time identifier of the largest time identifier present in the subset of bitstreams, and the target time identifier of the particular operating point is less than or equal to the maximum time identification present in the bitstream. symbol.

In subclause C.1 of HEVC Working Draft 8, TargetDecLayerIdSet is set to targetOpLayerIdSet. The targetOpLayerIdSet contains a collection of values of nuh_reserved_zero_6bits that exist in the representation of the operating point of the operating point under test. The targetOpLayerIdSet is a subset of the values of nuh_reserved_zero_6bits present in the measured bitstream.

In addition, the variable TargetDecHighestTid identifies the highest temporal sublayer to be decoded. The temporal sub-layer is a time-adjustable layer of a time-adjustable bitstream, the time-adjustable bitstream consisting of VCL NAL units with associated values of TemporalId and associated non-VCL NAL units. In subclause C.1 of the HEVC standard, TargetDecHighestTid is set to targetOpTid. targetOpTid is equal to the maximum temporal_id present in the operating point representation of the measured operating point and less than or equal to the maximum present in the measured bitstream Temporal_id. Therefore, when the decoding program is called for bit stream conformance testing, the values of TargetDecLayerIdSet and TargetDecHighestTid are set to the set of nuh_reserved_zero_6bits values present in the sub-bitstream and the maximum TemporalId value corresponding to the specific The measured operating point of the bitstream conformance test.

In this manner, a device, such as video encoder 20, video decoder 30, additional device 21, or another device, can perform the decoding process in accordance with one or more techniques of the present invention as part of performing bitstream conformance testing. Executing the decoding program can include performing a bitstream extraction program to extract an operand representation of the operating point defined by the target set of layer identifiers and the target highest time identifier from the bitstream. The target set of layer identifiers (ie, TargetDecLayerIdSet) contains the value of the layer identifier syntax element (eg, nuh_reserved_zero_6bits syntax element) present in the operation point representation. The target set of layer identifiers is a subset of the layer identifier syntax element values of the bit stream. The target highest time identifier (ie, TargetDecHighestTid) is equal to the maximum time identifier present in the operating point representation. The target highest time identifier is less than or equal to the maximum time identifier present in the bit stream. The execution of the decoding program also includes the NAL unit represented by the decoding operation point.

The decoding process is not always performed as part of performing a bitstream conformance test. Rather, the decoding program can be used in a general program for decoding bitstreams. When the decoder is not executed as part of the bitstream conformance test, the external source can specify the TargetDecLayerIdSet and TargetDecHighestTid of the operation point. An external source can be any source of information outside of the bit stream. For example, a CDN device can programmatically determine and specify a TargetDecLayerIdSet and a TargetDecHighestTid based on the configuration of a particular video decoder. The device performing the decoding process can extract the operating point representation from the bitstream using the externally specified TargetDecLayerIdSet and TargetDecHighestTid. The device performing the decoding process can then decode the NAL unit represented by the extracted operating point.

Therefore, when the decoding program is not executed as part of the bitstream conformance test, the device performing the decoding process can receive the target identifier and the highest target from the external source. Time identifier. The target set of layer identifiers contains the values of the layer identifier syntax elements that are present in the operation point representation. The target highest time identifier is equal to the maximum time identifier present in the second operating point representation. In addition, the device executable bitstream stream extractor executing the decoding process is represented by a bitstream extraction operation point. The device performing the decoding process can then decode the NAL unit represented by the operating point.

In other cases, the external source does not specify a TargetDecLayerIdSet or a TargetDecHighestTid. In such an example, the decoding program can execute on the entire bit stream. For example, the device executable bitstream stream extractor extracts an operand representation from a bitstream. In this example, 0 is the unique value of the layer identifier syntax element (ie, nuh_reserved_zero_6bits) present in the operation point representation. Moreover, in this example, the maximum time identifier present in the bitstream is equal to the maximum time identifier present in the operand representation. In this example, the device performing the decoding process can decode the NAL unit represented by the operating point.

As indicated above, the SPS may include an array of syntax elements represented as sps_max_dec_pic_buffering[i], where i is in the range of the maximum number of temporal layers in the 0-bit stream. When the highest time identifier (HighestTid) is equal to i , sps_max_dec_pic_buffering[i] indicates the maximum required size of the DPB. Sps_max_dec_pic_buffering[i] indicates the required size of the unit according to the image storage buffer. Furthermore, the SPS may comprise an array of syntax elements represented by sps_max_num_reorder_pics[i], where i is in the range of the maximum number of time layers in the 0-bit stream. Sps_max_num_reorder_pics[i] indicates the maximum allowable number of images preceding the image in the decoding order before the highest time identifier (HighestTid) is equal to i and in the output order. Furthermore, the set of HRD parameters may include an array of syntax elements denoted cpb_cnt_minus1[i], where i is in the range of the maximum number of time layers in the 0-bit stream. Cpb_cnt_minus1[i] specifies the number of alternative CPB specifications in the bitstream of the coded video sequence when the highest time identifier (HighestTid) is equal to i.

Since HEVC Working Draft 8 does not specify the meaning of the HighestTid (HEVC Working Draft 8), sps_max_dec_pic_buffering[i], sps_max_num_reorder_pics[i] are not properly selected in HRD operations, bitstream consistency operations, and level restrictions. ] and cpb_cnt_minus1[i]. In other words, the parameters sps_max_num_reorder_pics[i], sps_max_dec_pic_buffering[i], and cpb_cnt_minus1[i] in the HRD operation, the bit stream consistency requirement, and the level limit are not properly selected.

In accordance with one or more techniques of the present invention, sps_max_dec_pic_buffering[i] is defined such that sps_max_dec_pic_buffering[i] indicates the maximum required size of the DPB when TargetDecHighestTid is equal to i. The TargetDecHighestTid is determined in the manner described above. This situation can be contrasted with HEVC Working Draft 8 which does not define HighestTid. The value of sps_max_dec_pic_buffering[i] shall be in the range of 0 to MaxDpbSize (as specified in subclause A.4 of HEVC Working Draft 8) (including 0 and MaxDpbSize). When i is greater than 0, sps_max_dec_pic_buffering[i] will be equal to or greater than sps_max_dec_pic_buffering[i-1]. For each value of i, the value of sps_max_dec_pic_buffering[i] will be less than or equal to vps_max_dec_pic_buffering[i].

Similarly, according to one or more techniques of the present invention, sps_max_num_reorder_pics[i] is defined such that when TargetDecHighestTid is equal to i, sps_max_num_reorder_pics[i] indicates that any image is preceded in decoding order and after the image in output order The maximum allowed number of images. TargetDecHighestTid is determined in the manner described above. The value of sps_max_num_reorder_pics[i] should be in the range of 0 to sps_max_dec_pic_buffering[i] (including 0 and sps_max_dec_pic_buffering[i]). When i is greater than 0, sps_max_num_reorder_pics[i] will be equal to or greater than sps_max_num_reorder_pics[i- 1]. For each value of i, the value of sps_max_num_reorder_pics[i] will be less than or equal to vps_max_num_reorder_pics[i].

Moreover, in accordance with one or more techniques of the present invention, cpb_cnt_minus1[i] may specify the number of alternative CPB specifications in the bitstream of the coded video sequence when TargetDecHighestTid is equal to i, where i is in the 0-bit stream Within the range of the maximum number of time layers. The TargetDecHighestTid is determined in the manner described above. The value of cpb_cnt_minus1[i] is in the range of 0 to 31 (including 0 and 31). When low_delay_hrd_flag[i] is equal to 1, cpb_cnt_minus1[i] is equal to zero. When cpb_cnt_minus1[i] does not exist, cpb_cnt_minus1[i] is inferred to be equal to zero.

Thus, in accordance with one or more techniques of the present invention, a device can determine a particular syntax element from an array of syntax elements based on a highest time identifier. The highest time identifier is defined such that the highest time identifier always identifies the highest time layer to be decoded. Therefore, sps_max_num_reorder_pics[i], sps_max_dec_pic_buffering[i], and cpb_cnt_minus1[i] in the HRD operation, the bit stream consistency requirement, and the level limit are consistently selected, where i is equal to the clearly specified value of TargetDecHighestTid.

In this manner, the device (the video encoder 20, the video decoder 30, the additional device 21, or another device) can perform HRD operations to determine the consistency of the bitstream with the video write standard or to determine the video decoder and video write. Consistency of code standards. As part of performing the HRD operation, the device can determine the highest time identifier of the subset of bitstreams associated with the selected operating point of the bitstream. In addition, the device may determine a particular syntax element from an array of syntax elements (eg, sps_max_num_reorder_pics[i], sps_max_dec_pic_buffering[i], or cpb_cnt_minus1[i]) based on the highest temporal identifier. In addition, the device can use specific syntax elements in HRD operations.

Furthermore, in HEVC Working Draft 8, each of the hrd_parameters( ) syntax structures in the VPS can be associated with the operation_point_layer_ids( ) syntax structure. The hrd_parameters( ) syntax structure is selected for the operation_point_layer_ids( ) syntax structure for use in HRD operations. Corresponding to each selected hrd_parameters( ) syntax structure, a set of buffering period SEI messages and image timing SEI messages may also be required in the HRD operation. However, there is no way to associate a buffering period SEI message or an image timing SEI message to the hrd_parameters( ) syntax structure, where the associated operation_point_layer_ids( ) syntax structure of the hrd_parameters( ) syntax structure includes multiple values of nuh_reserved_zero_6bits (also That is, multiple views of HEVC, 3DV or multiple layer IDs in an adjustable video write extension).

The solution to this problem may be to apply a multi-view code-adjustable nested SEI message or the like as specified in Appendix H of H.264/AVC. However, multi-view code-encoded nested SEI messages or similar SEI messages can have the following disadvantages. First, since the SEI NAL unit in H.264/AVC has only one tuple NAL unit header, it is impossible to use the information in nuh_reserved_zero_6bits and temporal_id_plus1 carried in the NAL unit header of the HEVC SEI NAL unit. The buffer period or image timing SEI message is associated with the operating point. Second, each nested SEI message can be associated with only one operating point.

One or more techniques of the present invention can provide a clear designation of a buffering period SEI message, image timing, via an applicable_operation_points() syntax structure that can be carried in a buffering period SEI message, an image timing SEI message, or a sub-picture timing SEI message. The mechanism by which the SEI message or sub-picture timing SEI message is applied to the operating point. This mechanism may allow the use of information in the syntax elements nuh_reserved_zero_6bits and temporal_id_plus1 carried in the NAL unit header of the SEL NAL unit, and may allow multiple operating points to share SEI messages in the same buffer period, picture timing or sub-picture timing. Information passed in.

2 is a block diagram illustrating an example video encoder 20 that may implement the techniques of the present invention. Figure 2 is provided for purposes of explanation and should not be construed as limiting as broadly as in the present invention The technique is illustrated and described. For purposes of explanation, the present invention describes video encoder 20 in the context of the HEVC code. However, the techniques of the present invention are applicable to other writing standards or methods.

In the example of FIG. 2, the video encoder 20 includes a prediction processing unit 100, a residual generation unit 102, a transform processing unit 104, a quantization unit 106, an inverse quantization unit 108, an inverse transform processing unit 110, a reconstruction unit 112, and a filtering unit 114. The decoded image buffer 116 and the entropy encoding unit 118. The prediction processing unit 100 includes an inter-frame prediction processing unit 120 and an in-frame prediction processing unit 126. The inter-frame prediction processing unit 120 includes a motion estimation unit 122 and a motion compensation unit 124. In other examples, video encoder 20 may include more, fewer, or different functional components.

Video encoder 20 can receive video material. Video encoder 20 may encode each CTU in the tile of the image of the video material. Each of the CTUs can be associated with a similarly sized luma code tree block (CTB) and a corresponding CTB of the image. As part of encoding the CTU, prediction processing unit 100 may perform quadtree partitioning to divide the CTB of the CTU into progressively smaller blocks. The smaller block can be the code block of the CU. For example, the prediction processing unit 100 may divide the CTB associated with the CTU into four equal-sized sub-blocks, and divide one or more of the sub-blocks into four equal-sized sub-blocks, and many more.

Video encoder 20 may encode the CU of the CTU to produce an encoded representation of the CU (ie, the coded CU). As part of encoding the CU, prediction processing unit 100 may partition the write code block associated with the CU between one or more of the CUs. As such, each PU can be associated with a luma prediction block and a corresponding chroma prediction block. Video encoder 20 and video decoder 30 can support PUs of various sizes. The size of the CU may refer to the size of the CU's luma write block, and the size of the PU may refer to the size of the PU's luma prediction block. Assuming that the size of a particular CU is 2N×2N, the video encoder 20 and the video decoder 30 can support a 2N×2N or N×N PU size for intra-frame prediction, and 2N×2N for inter-frame prediction. 2N×N, N×2N, N×N or a symmetric PU size of similar size. Video encoder 20 and video decoder 30 can also Asymmetric splitting is supported to obtain PU sizes of 2N x nU, 2N x nD, nL x 2N, and nR x 2N for inter-frame prediction.

The inter-frame prediction processing unit 120 may generate predictive data for the PU by performing inter-frame prediction on each PU of the CU. The predictive data of the PU may include predictive blocks of the PU and motion information for the PU. The inter-frame prediction processing unit 120 may perform different operations on the PU of the CU by the PU in the I tile, the P tile, or the B tile. In the I tile, all PUs are predicted in-frame. Therefore, if the PU is in the I tile, the inter-frame prediction processing unit 120 does not perform inter-frame prediction on the PU. Thus, for blocks coded in I mode, predictive blocks are formed using spatial prediction from previously coded neighboring blocks within the same frame.

If the PU is in a P tile, motion estimation unit 122 may search the reference image in a list of reference images (eg, "RefPicListO") to find a reference region for the PU. The reference region for the PU may be a region within the reference picture that contains the sample block that most closely corresponds to the predicted block of the PU. Motion estimation unit 122 may generate a reference index indicating the location of the reference picture containing the reference region for the PU in RefPicListO. Moreover, motion estimation unit 122 may generate a motion vector that indicates a spatial displacement between the predicted block of the PU and a reference location associated with the reference region. For example, the motion vector can be a two-dimensional vector that provides an offset from the coordinates in the current image to the coordinates in the reference image. The motion estimation unit 122 may output a reference index and a motion vector as motion information of the PU. Motion compensation unit 124 may generate a predictive block of the PU based on actual or interpolated samples at the reference location indicated by the motion vector of the PU.

If the PU is in a B tile, motion estimation unit 122 may perform uni-prediction or bi-prediction for the PU. To perform unidirectional prediction for the PU, motion estimation unit 122 may search for a reference picture of RefPicListO or a second reference picture list ("RefPicListl") to find a reference area for the PU. The motion estimation unit 122 may output the following as the motion information of the PU: a reference index indicating the position of the reference image containing the reference region in RefPicList0 or RefPicList1; the motion vector, which refers to a spatial displacement between a sample block of the PU and a reference location associated with the reference zone; and one or more prediction direction indicators indicating that the reference image is in RefPicList0 or RefPicList1. Motion compensation unit 124 may generate a predictive block of the PU based at least in part on the actual or interpolated samples at the reference region indicated by the motion vector of the PU.

To perform bi-directional inter-frame prediction for the PU, motion estimation unit 122 may search for a reference picture in RefPicListO to find a reference region for the PU, and may also search for a reference picture in RefPicListl to find another reference for the PU. Area. Motion estimation unit 122 may generate reference indices that indicate the locations of reference pictures containing reference regions in RefPicListO and RefPicListl. Moreover, motion estimation unit 122 may generate a motion vector that indicates a spatial displacement between a reference location associated with the reference region and a sample block of the PU. The motion information of the PU may include a reference index of the PU and a motion vector. Motion compensation unit 124 may generate a predictive block of the PU based at least in part on the actual or interpolated samples at the reference region indicated by the motion vector of the PU.

In-frame prediction processing unit 126 may generate predictive material for the PU by performing in-frame prediction on the PU. Predictive data for the PU may include predictive blocks of the PU and various syntax elements. In-frame prediction processing unit 126 may perform intra-frame prediction on PUs in I-tiles, P-tiles, and B-tiles.

In order to perform intra-frame prediction on a PU, in-frame prediction processing unit 126 may use multiple in-frame prediction modes to generate multiple sets of predictive material for the PU. In order to use the intra-frame prediction mode to generate a set of predictive data for the PU, the in-frame prediction processing unit 126 may extend the sample region from the neighboring PU across the sample block of the PU in a direction associated with the in-frame prediction mode. A sample of the block. Assuming that the coding order is left to right and top to bottom for PU, CU, and CTU, the adjacent PUs may be above, to the upper right, to the upper left, or to the left of the PU. In-frame prediction processing unit 126 may use various numbers of in-frame prediction modes, for example, 33 directed in-frame prediction modes. In some examples, the number of in-frame prediction modes may depend on the size of the zone associated with the PU.

The prediction processing unit 100 may select the predictive data for the PU generated by the inter-frame prediction processing unit 120 or the predictive data for the PU of the CU from the predictive data for the PU generated by the in-frame prediction processing unit 126. In some examples, prediction processing unit 100 selects predictive data for the PU of the CU based on the rate/distortion metric of the set of predictive data. Herein, the predictive block of the selected predictive data may be referred to as the selected predictive block.

The residual generation unit 102 may generate the luma, Cb, and Cr residual blocks of the CU based on the luma of the CU, the Cb and Cr write code blocks, and the selected predictive luma, Cb, and Cr blocks of the PU of the CU. For example, the residual generation unit 102 may generate a residual block of the CU such that each sample in the residual block has a value equal to the selected prediction of the sample in the code block of the CU and the PU of the CU. The difference between the corresponding samples in the sex block.

Transform processing unit 104 may perform quadtree partitioning to partition the residual block associated with the CU into transform blocks associated with the TUs of the CU. Thus, a TU can be associated with a luma transform block and two chroma transform blocks. The size and location of the luma and chroma transform blocks of the TU of the CU may or may not be the size and location of the prediction block based on the PU of the CU. A quadtree structure called "Residual Quadtree" (RQT) may include nodes associated with each of the zones. The TU of the CU may correspond to the leaf node of the RQT.

Transform processing unit 104 may generate transform coefficient blocks for each TU of the CU by applying one or more transforms to the transform blocks of the TU. Transform processing unit 104 may apply various transforms to the transform blocks associated with the TUs. For example, transform processing unit 104 may apply a discrete cosine transform (DCT), a direction transform, or a conceptually similar transform to the transform block. In some examples, transform processing unit 104 does not apply the transform to the transform block. In such examples, the transform block can be treated as a transform coefficient block.

Quantization unit 106 may quantize the transform coefficients in the coefficient block. The quantization procedure can reduce the bit depth associated with some or all of the transform coefficients. For example, n- bit transform coefficients can be truncated to m- bit transform coefficients during quantization, where n is greater than m . Quantization unit 106 may quantize the coefficient block associated with the TU of the CU based on the quantization parameter (QP) value associated with the CU. Video encoder 20 may adjust the degree of quantization applied to the coefficient blocks associated with the CU by adjusting the QP value associated with the CU. Quantization can introduce loss of information, so quantized transform coefficients can have lower precision than the original transform coefficients.

Inverse quantization unit 108 and inverse transform processing unit 110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to reconstruct the residual block from the coefficient block. Reconstruction unit 112 may add the reconstructed residual block to a corresponding sample from one or more predictive blocks generated by prediction processing unit 100 to generate a reconstructed transform block associated with the TU. By reconstructing the transform block for each TU of the CU in this manner, video encoder 20 can reconstruct the code block of the CU.

Filtering unit 114 may perform one or more deblocking operations to reduce blockiness artifacts in the code blocks associated with the CU. After the filtering unit 114 performs one or more deblocking operations on the reconstructed code block, the decoded image buffer 116 may store the reconstructed code block. Inter-frame prediction unit 120 may use a reference picture containing reconstructed code blocks to perform inter-frame prediction of PUs of other pictures. Moreover, in-frame prediction processing unit 126 can use the reconstructed composing code blocks in decoded image buffer 116 to perform intra-frame prediction on other PUs in the same image as the CU.

Entropy encoding unit 118 may receive data from other functional components of video encoder 20. For example, entropy encoding unit 118 may receive coefficient blocks from quantization unit 106 and may receive syntax elements from prediction processing unit 100. Entropy encoding unit 118 may perform one or more entropy encoding operations on the data to produce entropy encoded material. For example, entropy encoding unit 118 may perform context adaptive variable length write code (CAVLC) operations, CABAC operations, variable-to-variable (V2V) length write code operations, grammar-based context adaptive binary arithmetic on data. Write code (SBAC) operation, probability interval partition entropy (PIPE) write code operation, exponential Golomb coding operation or another type of entropy coding operation. Video encoder 20 can output one A bit stream including the entropy encoded data generated by the entropy encoding unit 118 is included. For example, the bitstream may include data representing the RQT of the CU.

Video encoder 20 may signal the VPS in the bitstream as indicated elsewhere in the present invention. In HEVC Working Draft 8, the specific syntax elements of the VPS (ie, vps_max_dec_pic_buffering[i], vps_max_num_reorder_pics[i], and vps_max_latency_increase[i]) are defined with reference to the undefined value HighestTid. In accordance with one or more techniques of the present invention, such syntax elements of the VPS may be defined with reference to a value TargetDecHighestTid, which is defined such that the TargetDecHighestTid is as described elsewhere in the present invention. Table 3 below illustrates the syntax of a VPS in accordance with one or more techniques of the present invention.

Table 3 and the italicized portions of other grammar tables or semantic descriptions throughout the present invention may indicate differences from HEVC Working Draft 8. According to one or more techniques of the present invention, the following words of VPS The semantics of the law element can be changed as follows. The semantics of the other syntax elements of the VPS can remain the same as in HEVC Working Draft 8.

Vps_max_dec_pic_buffering [i] specifies the required size in the image storage buffer of the decoded image buffer when TargetDecHighestTid is equal to i. The value of vps_max_dec_pic_buffering[i] shall be in the range of 0 to MaxDpbSize (as specified in subclause A.4) (including 0 and MaxDpbSize). When i is greater than 0, vps_max_dec_pic_buffering[i] will be equal to or greater than vps_max_dec_pic_buffering[i-1].

Vps_max_num_reorder_pics [i] indicates the maximum allowable number of images before the image in the decoding order and after the image in the decoding order when TargetDecHighestTid is equal to i. The value of vps_max_num_reorder_pics[i] should be in the range of 0 to vps_max_dec_pic_buffering[i] (including 0 and vps_max_dec_pic_buffering[i]). When i is greater than 0, vps_max_num_reorder_pics[i] will be equal to or greater than vps_max_num_reorder_pics[i-1].

As by the MaxLatencyPictures [i] is set to vps_max_num_reorder_pics [i] + vps_max_latency_increase [i ] specified, it is not equal to vps_max_latency_increase 0 [i] is used to calculate MaxLatencyPictures [i] of value. When vps_max_latency_increase[i] is not equal to 0, the value of MaxLatencyPictures[i] is specified in the output order before the TargetDecHighestTid is equal to i, before any image in the coded video sequence and in the decoding order after the image. The maximum number of images. When vps_max_latency_increase[i] is equal to 0, the corresponding limit is not expressed. The value of vps_max_latency_increase[i] should be in the range of 0 to 2 ³² -2 (including 0 and 2 ³² -2).

As shown above, you can refer to TargetDecHighestTid to define vps_max_dec_buffering[i], vps_max_num_reorder_pics[i] and The semantics of vps_max_latency_increase[i]. In contrast, HEVC Working Draft 8 refers to HighestTid to define vps_max_dec_pic_buffering[i], vps_max_num_reorder_pics[i], and vps_max_latency_increase[i], where HighestTid is not defined.

As shown in the example syntax of Table 3, the VPS includes a number of pairs of operation_point_layer_ids( ) syntax structures and an hrd_parameters( ) syntax structure. The hrd_parameters( ) syntax structure includes syntax elements that specify a collection of HRD parameters. The operation_point_layer_ids( ) syntax structure includes syntax elements that identify a collection of operating points. The set of HRD parameters specified in the hrd_parameters( ) syntax structure may be applied to the operation points identified by the syntax elements in the corresponding operation_point_layer_ids( ) syntax structure. Table 4 below provides an example syntax for the operation_point_layer_ids( ) syntax structure.

Section 9.4.4 of HEVC Working Draft 8 describes the semantics of the op_point syntax structure. In accordance with one or more techniques of the present invention, Section 7.4.4 of HEVC Working Draft 8 may be modified as follows to provide the semantics of the operation_point_layer_ids( ) syntax structure of Table 4.

The operation_point_layer_ids(opIdx) syntax structure specifies a set of nuh_reserved_zero_6bits values included in the OpLayerIdSet of the operation point to which the opIdx hrd_parameters( ) syntax structure to which the video parameter set is applied.

Op_num_layer_id_values_minus1 [opIdx] plus 1 specifies the number of nuh_reserved_zero_6bits values included in the OpLayerIdSet of the operation point to which the opIdx hrd_parameters( ) syntax structure applied to the video parameter set is applied. Op_num_layer_id_values_minus1[opIdx] will be less than or equal to 63. In a bitstream consistent with this specification, op_num_layer_id_values_minus1[opIdx] will be equal to zero. Although the value of op_num_layer_id_values_minus1[opIdx] is required to be equal to 0 in this version of this specification, the decoder should allow other values to appear in the op_num_layer_id_values_minus1[opIdx] syntax.

Op_layer_id [opIdx][i] specifies the ith value of nuh_reserved_zero_6bits included in the OpLayerIdSet of the operation point to which the opIdx hrd_parameters( ) syntax structure to which the syntax parameter set is applied . When i is not equal to j and both i and j are in the range of 0 to op_num_layer_id_values_minus1 (including 0 and op_num_layer_id_values_minus1), the value of op_layer_id[opIdx][i] may not be equal to op_layer_id[opIdx][j]. Op_layer_id[0][0] is inferred to be equal to zero.

As indicated above, the op_num_layer_id_values_minus1[opIdx] syntax element plus 1 specifies the number of nuh_reserved_zero_6bits values included in the OpLayerIdSet of the operand to which the opIdx hrd_parameters( ) syntax structure to which the video parameter set is applied. In contrast, HEVC Working Draft 8 specifies that the op_num_layer_id_values_minus1[opIdx] syntax element plus 1 specifies the number of nuh_reserved_zero_6bits values included in the operating point identified by opIdx. Similarly, in the example of Table 4, the op_layer_id[opIdx][i] syntax element specifies the ith value of nuh_reserved_zero_6bits included in the OpLayerIdSet of the operation point to which the opIdx hrd_parameters( ) syntax structure to which the video parameter set is applied. In contrast, HEVC Working Draft 8 states that the op_layer_id[opIdx][i] syntax element specifies the ith value of nuh_reserved_zero_6bits included in the operating point identified by opIdx.

Section 7.4.2.2 of HEVC Working Draft 8 describes the semantics of SPS. In accordance with one or more techniques of the present invention, the following changes can be made to Section 7.4.2.2 of HEVC Working Draft 8. The semantics of the other syntax elements of the SPS may be the same as the semantics in HEVC Working Draft 8: sps_max_dec_pic_buffering [i] specifies the maximum required size of the decoded image buffer in units of image storage buffers when TargetDecHighestTid is equal to i. The value of sps_max_dec_pic_buffering[i] shall be in the range of 0 to MaxDpbSize (as specified in subclause A.4) (including 0 and MaxDpbSize). When i is greater than 0, sps_max_dec_pic_buffering[i] should be equal to or greater than sps_max_dec_pic_buffering[i-1]. For each value of i, the value of sps_max_dec_pic_buffering[i] should be less than or equal to vps_max_dec_pic_buffering[i].

Sps_max_num_reorder_pics [i] indicates the maximum allowable number of images before the image in the decoding order and after the image in the decoding order when TargetDecHighestTid is equal to i. The value of sps_max_num_reorder_pics[i] should be in the range of 0 to sps_max_dec_pic_buffering[i] (including 0 and sps_max_dec_pic_buffering[i]). When i is greater than 0, sps_max_num_reorder_pics[i] should be equal to or greater than sps_max_num_reorder_pics[i-1]. For each value of i, the value of sps_max_num_reorder_pics[i] should be less than or equal to vps_max_num_reorder_pics[i].

As by the MaxLatencyPictures [i] is set equal to sps_max_num_reorder_pics [i] + sps_max_latency_increase [i ] specified, it is not equal to sps_max_latency_increase 0 [i] is used to calculate MaxLatencyPictures [i] of value. When sps_max_latency_increase[i] is not equal to 0, the value of MaxLatencyPictures[i] is specified in the output order before the TargetDecHighestTid is equal to i, before any image in the coded video sequence and in the decoding order after the image. The maximum number of images. When sps_max_latency_increase[i] is equal to 0, the corresponding limit is not expressed. The value of sps_max_latency_increase[i] should be in the range of 0 to 2 ³² -2 (including 0 and 2 ³² -2). For each value of i, the value of sps_max_latency_increase[i] shall be less than or equal to vps_max_latency_increase[i].

As shown above, the semantics of sps_max_dec_pic_buffering[i], sps_max_num_reorder_pics[i], and sps_max_latency_increase[i] are defined according to TargetDecHighestTid. TargetDecHighestTid is as in this issue Determined as described elsewhere in the Ming Dynasty. In contrast, HEVC Working Draft 8 defines the semantics of sps_max_dec_pic_buffering[i], sps_max_num_reorder_pics[i], and sps_max_latency_increase[i] with reference to the undefined HighestTid.

Section 7.4.5.1 of HEVC Working Draft 8 describes general tile header semantics. The following changes can be made to section 7.4.5.1 of HEVC Working Draft 8 in accordance with one or more techniques of the present invention. The rest of Section 7.4.5.1 of HEVC Working Draft 8 may remain the same.

No_output_of_prior_pics_flag specifies how to process previously decoded images in the decoded image buffer after decoding the IDR or BLA image. See Appendix C. When the current image system CRA image or the current image is the IDR or BLA image of the first image in the bit stream, the value of no_output_of_prior_pics_flag has no effect on the decoding process. In the current image system is not the IDR or BLA image of the image in the bit stream, and the value of pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ] derived from the sequence parameter set in action is different from that of the previous image system. When the sequence parameter set derives the value of pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ], the decoder may (but should not) infer no_output_of_prior_pics_flag equal to 1 regardless of the actual value of no_output_of_prior_pics_flag.

As shown above, the semantics of no_output_of_prior_pics_flag is defined with reference to sps_max_dec_pic_buffering[TargetDecHighestTid]. TargetDecHighestTid is determined as described elsewhere in the present invention. In contrast, HEVC Working Draft 8 defines sp__max_dec_pic_buffering[HighestTid] to define the semantics of no_output_of_prior_pics_flags, where HighestTid is not defined.

Section 8.1 of HEVC Working Draft 8 describes a general decoding procedure. In accordance with one or more techniques of the present invention, the general decoding procedure of HEVC Working Draft 8 can be changed as follows.

The input to this program is a bit stream, and the output is a list of decoded images.

Specifies the set of values of nuh_reserved_zero_6bits of the VCL NAL unit to be decoded The set of TargetDecLayerIdSet is specified as follows:

- If an external component not specified in this specification can be used to set the TargetDecLayerIdSet, the TargetDecLayerIdSet is set by the external component.

- Otherwise, if the decoding procedure is called in the bitstream conformance test as specified in subclause C.1, the TargetDecLayerIdSet is set as specified in subclause C.1.

- Otherwise, the TargetDecLayerIdSet contains only one value of nuh_reserved_zero_6bits, which is equal to zero.

The variable TargetDecHighestTid identifying the highest time sublayer to be decoded is specified as follows:

- If an external component not specified in this specification can be used to set the TargetDecHighestTid, the TargetDecHighestTid is set by the external component.

- Otherwise, if the decoding procedure is called in the bitstream conformance test as specified in subclause C.1, the TargetDecHighestTid is set as specified in subclause C.1.

- Otherwise, set TargetDecHighestTid to sps_max_sub_layers_minus1.

The sub-bitstream extractor as specified in subclause 10.1 is applied with the TargetDecHighestTid and TargetDecLayerIdSet as inputs, and the output is assigned to the bitstream, commonly referred to as BitstreamToDecode.

The following applies to each coded image in BitstreamToDecode (referred to as the current image, which is represented by the variable CurrPic).

Depending on the value of chroma_format_idc, the number of sample arrays of the current image is as follows.

- If chroma_format_idc is equal to 0, the current image consists of 1 sample array S _L .

- Otherwise (chroma_format_idc is not equal to 0), the current image consists of 3 sample arrays S _L , S _Cb , S _Cr .

The decoding process of the current image uses the syntax elements from clause 7 and uppercase letter variables as input. When it comes to interpreting each NAL unit and a "bitstream" or portion thereof (eg, the semantics of each syntax element in a coded video sequence, the bitstream or portion thereof means a BitstreamToDecode or portion thereof.

The decoding procedure is specified such that all decoders should produce numerically identical results. Any decoding program that produces the same results as the programs described herein is consistent with the decoding program requirements in this specification.

When the current image is a CRA image, the following applies:

- If an external component not specified in this specification can be used to set the variable HandleCraAsBlaFlag to a value, HandleCraAsBlaFlag is set to the value provided by the external component.

- Otherwise, set the value of HandleCraAsBlaFlag to 0.

When the current image is a CRA image and HandleCraAsBlaFlag is equal to 1, the following applies during the parsing and decoding process of each coded tile NAL unit:

- Set the value of nal_unit_type to BLA_W_LP.

- Set the value of no_output_of_prior_pics_flag to 1.

Note 1 - The decoder implementation may choose to set the value of no_output_of_prior_pics_flag when setting the decoding of subsequent pictures that do not affect the current picture and in decoding order (for example, when there is always an available image storage buffer when needed) Set to 0.

Each image mentioned in this clause is a complete coded image.

The decoding procedure is structured as follows depending on the value of separate_colour_plane_flag.

- If separate_colour_plane_flag is equal to 0, the decoder is called once, where the current image is the output.

- Otherwise (separate_colour_plane_flag is equal to 1), the decoding program is called three times. The input to the decoding program is all NAL units of the coded image having the same value of colour_plane_id. The decoding procedure for a NAL unit having a particular value of colour_plane_id is specified as if only a coded video sequence in a monochrome format with that particular value of colour_plane_id would be present in the bitstream. The output of each of the three decoding procedures is assigned to a 3 sample array of the current image, wherein a NAL unit having a colour_plane_id equal to 0 is assigned to S _L , and a NAL unit having a colour_plane_id equal to 1 is assigned to S _Cb , And a NAL unit having a colour_plane_id equal to 2 is assigned to S _Cr .

Note 1 - When separate_colour_plane_flag is equal to 1 and chroma_format_idc is equal to 3, the variable ChromaArrayType is derived as 0. In the decoding process, the value of this variable is evaluated, resulting in an operation similar to the operation of a monochrome image in which chroma_format_idc is equal to zero.

The decoding program operates on the current image CurrPic as follows:

1. The decoding of the NAL unit is specified in subclause 8.2.

2. The procedure in subclause 8.3 specifies the decoding procedure using the syntax elements in the tile layer and the above syntax elements:

- Deriving variables and functions related to image order count in subclause 8.3.1 (only need to call it for the first tile of the image).

- Call the decoding procedure for the reference image set in subclause 8.3.2, where the reference image can be marked as "not for reference" or "for long-term reference" (only need to call it for the first tile of the image) ).

- when the current image is a BLA image or a CRA image of the first image in the bitstream, the decoding procedure specified in subclause 8.3.3 for generating an unavailable reference image is invoked. (You only need to call it for the first tile of the image).

- PicOutputFlag is set as follows:

- If the current image is a TFD image and the previous RAP image system BLA image in decoding order or the CRA image of the first coded image in the bit stream is set, PicOutputFlag is set equal to 0.

- Otherwise, PicOutputFlag is set equal to pic_output_flag.

- At the beginning of the decoding procedure for each P or B tile, the decoding procedure specified for the reference picture list specified in subclause 8.3.4 is invoked for deriving reference picture list 0 (RefPicList0) and is decoding The B program is called when the B block is used to derive the reference picture list 1 (RefPicList1).

- After all the tiles of the current image have been decoded, the decoded image is marked as "for short-term reference".

3. The procedures in subclauses 8.4, 8.5, 8.6, and 8.7 specify the decoding procedure using the syntax elements in the code tree unit layer and the above syntax elements.

As indicated elsewhere in the present invention, in HEVC Working Draft 8, when the decoding procedure is called for the bitstream conformance test, since the values of TargetDecLayerIdSet and TargetDecHighestTid are not properly set, the semantics of the syntax elements are not clearly defined. Specified. The modifications to the general decoding procedure shown above can remedy this problem. As shown above, when a general decoding procedure is invoked for a bitstream conformance test, the values of TargetDecLayerIdSet and TargetDecHighestTid are set as specified in subclause C.1. As described below, the modified version of subclause C.1 may set the TargetDecLayerIdSet to be a set of nuh_reserved_zero_6bits values present in the sub-bitstream corresponding to the tested operating point. The modified version of subclause C.1 may set TargetDecHighestTid to the maximum TemporalId value present in the sub-bitstream corresponding to the tested operating point.

In this manner, a device such as video decoder 30 can execute a decoding program as part of performing a bitstream conformance test. Executing the decoding program may include performing a bitstream stream extractor to extract from the bitstream stream by the target set of the layer identifier and the target highest time identifier definition The operating point representation of the operating point. The target set of layer identifiers may contain values of layer identifier syntax elements present in the operation point representation, and the target set of layer identifiers is a subset of the values of the layer identifier syntax elements of the bit stream. The target highest time identifier may be equal to the maximum time identifier present in the operation point representation, the target highest time identifier being less than or equal to the maximum time identifier present in the bit stream. In addition, the device can decode the NAL unit represented by the operating point.

As indicated above in the modification to Section 8.1, the decoding procedure does not have to be performed as part of the bitstream conformance test. In some examples where the decoding process is not performed as part of a bitstream conformance test, the device executable bitstream fetcher is represented by an operand that extracts the operand from the bitstream. In this case, 0 may be a unique value of the layer identifier syntax element (eg, nuh_reserved_zero_6bits) present in the operation point representation, and the maximum time identifier present in the bit stream is equal to the operation point representation present at the operation point The maximum time identifier in . The device can decode the NAL unit represented by the operating point of the second operating point.

Alternatively, the device may receive a target set of layer identifiers and a target highest time identifier from an external source. The target set of layer identifiers may contain values of layer identifier syntax elements present in the operand point representation of the operating point, the operation points being defined by the target set of layer identifiers and the target highest time identifier. The target highest time identifier may be equal to the maximum time identifier present in the operating point representation of the operating point. In addition, the device executable bitstream stream extractor represents an operand representation of the operand from the bitstream extraction. In addition, the device can decode the NAL unit represented by the operating point of the operating point.

Moreover, in accordance with one or more techniques of the present invention, the sub-bitstream extraction procedure described in subclause 10.1 of HEVC Working Draft 8 can be changed as follows.

The requirement for bitstream consistency is that tIdTarget included in the output of the program specified in this subclause is equal to any value in the range of 0 to 6 (including 0 and 6) and layerIdSetTarget contains only any value of 0. The bitstream should be consistent with this specification.

Remarks - Consistent bitstreams contain nuh_reserved_zero_6bits equal to 0 And one or more coded tile NAL units of TemporalId equal to zero.

At this point, the input variable tIdTarget and the set layerIdSetTarget of the program .

The output of this program is a sub-bit stream.

The sub-bitstream is derived by removing all NAL units having a TemporalId greater than tIdTarget or nuh_reserved_zero_6bits that are not in the value in layerIdSetTarget from the bitstream.

In subclause 10.1 of HEVC Working Draft 8, the variable name targetDecLayerIdSet is used, where layerIdSetTarget is used above. The change to sub-clause 10.1 of the HEVC working draft shown above using layerIdSetTarget can be used to clarify that there may be a difference between the set of layer identifiers used in the sub-bitstream extraction procedure and the targetDecLayerIdSet, as in the present invention Other places are described with specific definitions.

Moreover, in accordance with one or more techniques of the present invention, the general hierarchy and level specifications of Section A.4.1 of HEVC Working Draft 8 can be changed as follows. In the present invention, "profile" may refer to a subset of the meta-stream syntax. "Level" and "Level" can be specified in each profile. The level of the hierarchy may be a specified set of constraints imposed on the values of the syntax elements in the bitstream. These constraints can be a simple limit on the value.

Alternatively, the constraints may take the form of a constraint on the arithmetic combination of values (eg, image width x image height x number of images decoded per second). The level specified for the lower level is more constrained than the level specified for the higher level. In accordance with an example of the present invention, the "General Level Specification" section of HEVC Working Draft 8 (i.e., Section A.4.1) is re-titled as "General Class and Level Specification" and the text is changed as follows. Table A-1 can remain the same as in HEVC Working Draft 8.

In order to compare hierarchical capabilities, a hierarchy having a general_tier_flag equal to 0 should be considered a lower level than a hierarchy having a general_tier_flag equal to one.

In order to compare level capabilities, the lower level has a lower level for a particular level. General_level_idc value.

In order to express these constraints in this appendix, the following is specified.

- causing the access unit n to be the nth access unit in decoding order, wherein the first access unit is access point 0 (i.e., the 0th access unit).

- Let the image n be a coded image of the access unit n or a corresponding decoded image.

- Set the variable fR to 1÷300.

The bitstream consistent with the profile at the specified level shall obey the following constraints for each bitstream conformance test as specified in Appendix C:

a) The nominal removal time from the CPB as specified in subclause C.2.2 (where n > 0) from the CPB. The value of PicSizeInSamplesY for image n-1 satisfies the following constraints: t _r,n (n) -t _r (n-1) is equal to or greater than Max (PicSizeInSamplesY÷MaxLumaSR, fR), where MaxLumaSR is the value applied to image n-1 specified in Table A-1.

b) The difference between the continuous output times of the images specified in subclause C.3.2 from DPB for the value of PicSizeInSamplesY of image n satisfies the following constraints: Δt _{o, dpb} (n)>=Max(PicSizeInSamplesY÷MaxLumaSR , fR), where MaxLumaSR is the value specified in Table A-1 for the image n, with the constraint that the image n is the output image and is not the last image of the output of the bit stream.

c) PicSizeInSamplesY<=MaxLumaPS, where MaxLumaPS is specified in Table A-1.

d) pic_width_in_luma_samples<=Sqrt(MaxLumaPS * 8)

e) pic_height_in_luma_samples<=Sqrt(MaxLumaPS * 8)

f) sps_max_dec_pic_buffering[ TargetDecHighestTid ]<=MaxDpbSize, where MaxDpbSize is derived as specified below: (PicSizeInSamplesY<=(MaxLumaPS>>2))

Then MaxDpbSize=Min(4 * MaxDpbPicBuf,16)

Otherwise if (PicSizeInSamplesY<=(MaxLumaPS>>1))

Then MaxDpbSize=Min(2 * MaxDpbPicBuf, 16)

Otherwise if (PicSizeInSamplesY<=(MaxLumaPS<<1)/3)

Then MaxDpbSize=Min((3 * MaxDpbPicBuf)>>1,16)

Otherwise if (PicSizeInSamplesY<=((3 * MaxLumaPS)>>2))

Then MaxDpbSize=Min((4 * MaxDpbPicBuf)/3,16)

otherwise

MaxDpbSize=MaxDpbPicBuf

Where MaxLumaPS is specified in Table A-1 and MaxDpbPicBuf is equal to 6.

Table A-1 specifies the limits for each level of each level. The use of the MinCR parameter column of Table A-1 is specified in subclause A.4.2.

The hierarchy and level to which the bitstream conforms should be indicated by the syntax elements general_tier_flag and general_level_idc as follows.

- According to the hierarchical constraint specification in Table A-1, the general_tier_flag equal to 0 indicates the identity with the primary hierarchy, and the general_tier_flag equal to 1 indicates the consistency with the upper hierarchy. For levels below level 4, the general_tier_flag shall be equal to 0 (corresponding to the entry labeled "-" in Table A-1). The level limits in Table A-1 that differ from MaxBR and MaxCPB are common to both the main and high levels.

- general_level_idc should be set to a value equal to 30 times the number of levels specified in Table A-1.

As indicated in item (f) above, the bit stream consistent with the profile at the specified level is subject to the constraint of sps_max_dec_pic_buffering[TargetDecHighestTid]<=MaxDpbSize. The TargetDecHighestTid can be defined in a manner described elsewhere in the present invention. In contrast, HEVC Working Draft 8 indicates for item (f) that the bit stream consistent with the profile at the specified level is subject to the constraint of sps_max_dec_pic_buffering[sps_max_temporal_layers_minus1]<=MaxDpbSize. As indicated elsewhere in the invention, may not be at the level The parameter sps_max_dec_pic_buffering[i] is chosen appropriately. In accordance with one or more techniques of the present invention, replacing sps_max_temporal_layers_minus1 with TargetDecHighestTid as the index i of sps_max_dec_pic_buffering[i] ensures that the level limit is consistently selected, where i is equal to the clearly specified value of TargetDecHighestTid.

In this manner, the decoding program of the HRD may decode an array of syntax elements from the SPS (eg, sps_max_dec_pic_buffering[]), where each of the syntax elements in the array indicates the maximum required size of the DPB of the HRD. Moreover, when the device performs an HRD operation, the device can determine a particular syntax element in the array based on the target highest time identifier (eg, TargetDecHighestTid) (eg, sps_max_dec_pic_buffering[TargetDecHighestTid]). Moreover, when the value of a particular syntax element is greater than the maximum DPB size (eg, MaxDpbSize), the device can determine that the bitstream is not consistent with the video writing standard.

Moreover, in accordance with one or more example techniques of the present invention, Section A.4.2 of HEVC Working Draft 8 can be changed as follows. Section A.4.2 of HEVC Working Draft 8 describes the profile-specific level limits for the main profile. Table A-2 can remain the same as in HEVC Working Draft 8.

The bitstream consistent with the primary profile at the specified level and level shall obey the following constraints for the bitstream conformance test as specified in Appendix C :

a) The number of tiles in the image (where dependent_slice_flag is equal to 0 or 1) is less than or equal to MaxSlicesPerPicture, where MaxSlicesPerPicture is specified in Table A-1.

b) For VCL HRD parameters, for at least one value of SchedSelIdx, BitRate[SchedSelIdx]<=cpbBrVclFactor*MaxBR and CpbSize[SchedSelIdx]<=cpbBrVclFactor*MaxCPB, where cpbBrVclFactor is specified in Table A-2, and BitRate[SchedSelIdx ]and CpbSize[SchedSelIdx] is given as follows.

- If vcl_hrd_parameters_present_flag is equal to 1, then BitRate[SchedSelIdx] and CpbSize[SchedSelIdx] are given by using the syntax elements selected as specified in subclause C.1, respectively, by equations E-45 and E-46.

- Otherwise (vcl_hrd_parameters_present_flag is equal to 0), BitRate[SchedSelIdx] and CpbSize[SchedSelIdx] are inferred as specified in subclause E.2.3 for the VCL HRD parameter.

In Table A-1, MaxBR and MaxCPB are specified in units of cpbBrVclFactor bit/s and cpbBrVclFactor bits, respectively. For at least one value of SchedSelIdx in the range of 0 to cpb_cnt_minus1[ TargetDecHighestTid ] (including 0 and cpb_cnt_minus1[ TargetDecHighestTid ]), the bit stream should satisfy these conditions.

c) For NAL HRD parameters, for at least one value of SchedSelIdx, BitRate[SchedSelIdx]<=cpbBrNalFactor*MaxBR and CpbSize[SchedSelIdx]<=cpbBrNalFactor*MaxCPB, where cpbBrNalFactor is specified in Table A-2, and BitRate[SchedSelIdx ] and CpbSize[SchedSelIdx] are given as follows.

- If nal_hrd_parameters_present_flag is equal to 1, then BitRate[SchedSelIdx] and CpbSize[SchedSelIdx] are given by using the syntax elements selected as specified in subclause C.1, respectively, by equations E-45 and E-46.

- Otherwise (nal_hrd_parameters_present_flag is equal to 0), BitRate[SchedSelIdx] and CpbSize[SchedSelIdx] are inferred as specified in subclause E.2.3 for the NAL HRD parameter.

In Table A-1, MaxBR and MaxCPB are specified in units of cpbBrNalFactor bit/s and cpbBrNalFactor bits, respectively. For at least one value of SchedSelIdx in the range of 0 to cpb_cnt_minus1[ TargetDecHighestTid ] (including 0 and cpb_cnt_minus1[ TargetDecHighestTid ]), the bit stream should satisfy these conditions.

d) the sum of the NumBytesInNALunit variables of access unit 0 for the value of PicSizeInSamplesY of image 0 is less than or equal to 1.5 *(Max(PicSizeInSamplesY,fR * MaxLumaSR)+MaxLumaSR *(t _r (0)-t _r,n (0) )) MinCR, where MaxLumaPR and MinCR are the values applied to Image 0 as specified in Table A-1.

e) The sum of the NumBytesInNALunit variables of access unit n (where n>0) is less than or equal to 1.5 * MaxLumaSR *(t _r (n)-t _r (n-1))÷MinCR, where MaxLumaSR and MinCR are in Table A The value applied to image n specified in -1.

f) For level 5 and higher, the variable CtbSizeY should equal 32 or 64.

g) The value of NumPocTotalCurr should be less than or equal to 8.

h) The value of num_tile_columns_minus1 shall be less than MaxTileCols, and num_tile_rows_minus1 shall be less than MaxTileRows, where MaxTileCols and MaxTileRows are as specified in Table A-1.

As indicated elsewhere in the present invention, the parameter cpb_cnt_minus1[i] may not be properly selected in terms of level restrictions. HEVC Working Draft 8 specifies: "For at least one value of SchedSelIdx in the range of 0 to cpb_cnt_minus1 (including 0 and cpb_cnt_minus1), the bit stream shall satisfy these conditions...". In accordance with one or more techniques of the present invention, specifying TargetDecHighestTid as the index i of cpb_cnt_minus1[i] ensures that the level limit is consistently selected, where i is equal to the clearly specified value of TargetDecHighestTid.

Moreover, in accordance with one or more techniques of the present invention, general subclause C.1 in Appendix C of HEVC Working Draft 8 may be modified. Figures C-1 and C-2 of subclause C.1 of HEVC Working Draft 8 may remain the same as those of HEVC WD8. The text of subclause C.1 of HEVC Working Draft 8 can be changed as follows.

This appendix specifies the hypothetical reference decoder (HRD) and its check bit stream is consistent with the decoder. Sexual use.

Two types of bitstreams are subject to the HRD consistency check of this specification. A first type of bit stream called a Type I bit stream contains only VCL NAL units of all access units in the bit stream and NAL units with NAL units (filler data NAL units) equal to nal_unit_type of FD_NUT flow. A second type of bitstream, referred to as a Type II bitstream, contains at least one of the following: in addition to the VCL NAL unit and the padding data NAL unit of all access units in the bitstream: - Unlike padding Additional non-VCL NAL units of the data NAL unit, - forming all leading_zero_8bits, zero_byte, start_code_prefix_one_3bytes, and trailing_zero_8bits syntax elements from the byte stream of the NAL unit stream (as specified in Appendix B).

Figure C-1 shows the type of bit stream consistency point checked by HRD.

The syntax elements (or default values for some of the syntax elements) required by the HRD for non-VCL NAL units are specified in the semantic subclauses of Appendix D and E clause 7.

Two types of HRD parameters (NAL HRD parameters and VCL HRD parameters) are used. The HRD parameters are sent via the video parameter set syntax structure or via video availability information as specified in subclauses E.1 and E.2, which is part of the sequence structure of the sequence parameter set.

Multiple tests are required to check the consistency of the bitstream. For each test, apply the following steps in the order listed:

1. Select the operated operating point indicated as TargetOp. TargetOp is identified by an OpLayerIdSet equal to targetOpLayerIdSet and an OpTid equal to targetOpTid. The targetOpLayerIdSet contains a set of values of nuh_reserved_zero_6bits present in the subset of bitstreams associated with TargetOp, and should be a subset of the values of nuh_reserved_zero_6bits present in the measured bitstream. targetOpTid is equal to the maximum TemporalId present in the subset of bitstreams associated with TargetOp and should be less than or equal to The greatestTemporalId that exists in the measured bitstream.

2. TargetDecLayerIdSet is set to targetOpLayerIdSet, TargetDecHighestTid is set to targetOpTid, and BitstreamToDecode is set to the output of the sub-bitstream extractor as specified in subclause 10.1, where TargetDecHighestTid and TargetDecLayerIdSet are used as inputs.

3. Select the hrd_parameters( ) syntax structure and the sub_layer_hrd_parameters( ) syntax structure for TargetOp. If the TargetDecLayerIdSet contains only the value 0, then the hrd_parameters( ) syntax structure in the active sequence parameter set is selected. Otherwise, select the hrd_parameters( ) syntax structure that satisfies the following conditions: it is in the active parameter set (or via external components) and for i in the range 0 to op_num_layer_id_values_minus1[opIdx] (including 0 and op_num_layer_id_values_minus1[opIdx]) The set of values specified by op_layer_id[opIdx][i] is the same as TargetDecLayerIdSet. In the selected hrd_parameters( ) syntax structure, if BitstreamToDecode is a type I bit stream, the sub_layer_hrd_parameters(TargetDecHighestTid) syntax structure immediately following the condition "if(vcl_hrd_parameters_present_flag)" is selected (in this case, the variable NalHrdModeFlag is set equal to 0), otherwise (BitstreamToDecode is a type II bit stream), the selection is immediately followed by the condition "if(vcl_hrd_parameters_present_flag)" (in this case, the variable NalHrdModeFlag is set equal to 0) or the condition "if(nal_hrd_parameters_present_flag)" (in this case) Next, the variable NalHrdModeFlag is set to be equal to the sub_layer_hrd_parameters (TargetDecHighestTid) syntax structure after 1), and all non-VCL NAL units except the filler data NAL unit are discarded from BitstreamToDecode in the previous case, and the result is assigned to BitstreamToDecode.

4. Select the access unit associated with the buffer period SEI message for TargetOp As the HRD initialization point, it is called access unit 0.

5. Select the SEI message that includes the timing information. A buffering period SEI message is selected, which is coded in access unit 0 and applied to TargetOp as indicated by the intellectual_operation_points( ) syntax structure. For each access unit of BitstreamToDecode starting from access unit 0, the selection system is associated with the access unit and applied to the image timing SEI message as TargetOp as indicated by the applicable_operation_points( ) syntax structure, and is equal to 1 in SubPicCpbFlag and When sub_pic_cpb_params_in_pic_timing_sei_flag is equal to 0, the selection is associated with the decoding unit in the access unit and applied to the sub-picture timing SEI message of TargetOp as indicated by the applicable_operation_points() syntax structure.

6. Select the value of SchedSelIdx. The selected SchedSelIdx should be in the range of 0 to cpb_cnt_minus1[TargetDecHighestTid] (including 0 and cpb_cnt_minus1[TargetDecHighestTid]), wherein cpb_cnt_minus1[TargetDecHighestTid] can be found in the sub_layer_hrd_parameters(TargetDecHighestTid) syntax structure as selected above.

7. The initial CPB removal delay and delay offset are selected, and the TFD access unit associated with access unit 0 can be discarded from BitstreamToDecode. If the coded picture in access unit 0 has nal_unit_type equal to CRA_NUT or BLA_W_LP, and rap_cpb_params_present_flag in the selected buffer period SEI message is equal to 1, the selection is represented by initial_cpb_removal_delay[SchedSelIdx] and initial_cpb_removal_delay_offset[SchedSelIdx] corresponding to NalHrdModeFlag Preset initial CPB removal delay and delay offset (in this case, the variable DefaultInitCpbParamsFlag is set equal to 1) or an alternative initial represented by initial_alt_cpb_removal_delay[SchedSelIdx] and initial_alt_cpb_removal_delay_offset[SchedSelIdx] corresponding to NalHrdModeFlag CPB removal delay and delay offset (in this case, the variable DefaultInitCpbParamsFlag is set equal to 0), and in the latter case, the TFD access unit associated with access unit 0 is discarded from BitstreamToDecode and the result is assigned to BitstreamToDecode. Otherwise, the preset initial CPB removal delay and delay offset are selected (in this case, the variable DefaultInitCpbParamsFlag is set equal to 1).

The number of bit stream conformance tests performed is equal to N1 * N2 * N3 * (N4 * 2 + N5), where the values of N1, N2, N3, N4, and N5 are specified as follows.

- N1 is the number of operating points contained in the measured bit stream.

- If Bitstream ToDecode is a type I bit stream, then N2 is equal to 1, otherwise (BitstreamToDecode is a type II bit stream) and N2 is equal to 2.

- N3 is equal to cpb_cnt_minus1[TargetDecHighestTid] + 1.

- N4 is the number of access units associated with a buffering period SEI message for TargetOp in BitstreamToDecode, wherein the coded image in each of the access units has a nal_unit_type equal to CRA_NUT or BLA_W_LP, And the associated buffering period SEI message applicable to TargetOp has rap_cpb_params_present_flag equal to one.

- N5 is the number of access units associated with a buffering period SEI message for TargetOp in BitstreamToDecode, wherein the coded image in each of the access units has a value other than CRA_NUT and BLA_W_LP The nal_unit_type of one, or the associated buffering period SEI message applicable to TargetOp has rap_cpb_params_present_flag equal to zero.

When BitstreamToDecode is a type II bit stream, if the sub_layer_hrd_parameters(TargetDecHighestTid) syntax structure immediately following the condition "if(vcl_hrd_parameters_present_flag)" is selected, the test is performed at the type I coincidence point shown in Figure C-1. And for input bit rate and CPB Stores only the VCL and padding data NAL units; otherwise, the sub_layer_hrd_parameters(TargetDecHighestTid) syntax structure immediately following the condition "if(nal_hrd_parameters_present_flag)" is selected, and the test is performed at the type II coincidence point shown in Figure C-1. And for all NAL units or (of a byte stream) all bytes of the input bit rate and CPB store count (of type II NAL unit streams).

NOTE 3 – For the same values of InitCpbRemovalDelay[SchedSelIdx], BitRate[SchedSelIdx] and CpbSize[SchedSelIdx] for the VBR condition (cbr_flag[SchedSelIdx] equal to 0), by the type II coincidence point shown in Figure C-1 The NAL HRD parameter established by the value of SchedSelIdx is sufficient to also establish the VCL HRD consistency of the type I consistency point shown in Figure C-1. This is because the data stream in the type I consistency point is tied to a subset of the data stream in the Type II consistency point, and because the CPB is allowed to become empty for the VBR condition and remains empty until the next graph Like the time of scheduling to start arriving. For example, when using a decoding program specified in clauses 2 through 9 to decode a coded video sequence that is consistent with one or more of the profiles specified in Appendix A, when providing not only for type II consistency points In the sub-clause A.4.2, item c), for the purpose of setting the file consistency, within the limits set for the NAL HRD parameter and belonging to sub-clause A.4.2, b), the VCL HRD parameter is set for the purpose of setting the file. The NAL HRD parameters within the limits also ensure that the conformance of the VCL HRD of the Type I consistency point falls within the bounds of item b) of subclause A.4.2.

All video parameter sets , sequence parameter sets, and image parameter sets involved in the VCL NAL unit and corresponding buffer period and picture timing SEI messages shall be in the bit stream (by non-VCL NAL units) or by specification Other components not specified in the time are passed to the HRD in a timely manner.

In Appendix C, D, and E, when non-VCL NAL units (or only some of the NAL units) are passed to the decoder (or to the HRD) by other components not specified by this specification, The specification of the "presence" of the NAL unit is also satisfied. In order to count the bits, only the appropriate bits actually present in the bitstream are counted.

NOTE 1 As an example, if the encoder decides to pass a non-VCL NAL unit in the bitstream, it indicates two points in the bitstream (non-VCL NAL units will exist in the bitstream between the two points) Synchronization of non-VCL NAL units communicated by means other than the presence in the bit stream with NAL units present in the bit stream can be achieved.

When the content of a non-VCL NAL unit is passed for delivery by some means other than the presence within the bitstream, the representation of the content of the non-VCL NAL unit is not required to use the same syntax as specified in this specification.

NOTE 2 – When HRD information is contained in a bitstream, it is possible to verify the consistency of the bitstream with the requirements of this subclause based solely on the information contained in the bitstream. When the HRD information does not exist in the bitstream, such as for all "independent" type I bitstream conditions, the consistency can be verified only when the HRD data is supplied by some other means not specified in the specification.

The HRD contains a coded image buffer (CPB), an instantaneous decoding program, a decoded image buffer (DPB), and an output crop as shown in Figure C-2.

For each bitstream conformance test, the CPB size (number of bits) is as specified by equation E-46, CpbSize[SchedSelIdx], where the SchedSelIdx and HRD parameters are selected as specified above in these subclauses. . The DPB size (the number of image storage buffers) is sps_max_dec_pic_buffering[TargetDecHighestTid].

The variable SubPicCpbPreferredFlag is specified by an external component or is set to 0 when not specified by an external component.

The variable SubPicCpbFlag is derived as follows: SubPicCpbFlag=SubPicCpbPreferredFlag && sub_pic_cpb_params_present_flag (C-1)

If SubPicCpbFlag is equal to 0, CPB operates at the access unit level, and each solution The code unit is an access unit. Otherwise, the CPB operates at the sub-picture level and each decoding unit is a subset of the access units.

HRD operates as follows. The data associated with the decoding unit flowing into the CPB according to the specified arrival schedule is delivered by the HSS. The data associated with each decoding unit is instantaneously removed and decoded by the instantaneous decoding procedure at the CPB removal time of the decoding unit. Each decoded image is placed in the DPB. The decoded image is removed from the DPB as specified in subclause C.3.1 or subclause C.5.2.

The operation of the CPB for each bitstream conformance test is specified in subclause C.2. The instantaneous decoder operation is specified in clauses 2 through 9. The operation of the DPB for each bitstream conformance test is specified in subclause C.3. The output cropping for each bitstream conformance test is specified in subclause C.3.2 and subclause C.5.2.

The HSS and HRD information relating to the number of listed delivery schedules and their associated bit rate and buffer size is specified in subclauses E.1.1, E.1.2, E.2.1 and E.2.2. The HRD is initialized as specified by the buffering period SEI message specified in subclauses D.1.1 and D.2.1. The timing of the removal of the decoding unit from the CPB and the output timing of the decoded image from the DPB are specified in the image timing SEI message as specified in subclauses D.1.2 and D.2.1. All timing information associated with a particular decoding unit should arrive before the CPB removal time of the decoding unit.

The requirement for bitstream consistency is specified in subclause C.4, and HRD is used to check the consistency of the decoder as specified in subclause C.5.

Note 3 - Although it is assumed that all image rate and time streams used to generate the bit stream are consistent with the value of the signal transmitted in the bit stream, in real systems, these image rates are timely and timely. Each of them may be different from the value sent or specified.

All of the arithmetic in this appendix is done with real values so that rounding errors cannot be propagated. For example, the number of bits in the CPB just before or after the removal of the decoding unit is not necessarily an integer.

The variable t _c is derived as follows and is called the tick: t _c = num_units_in_tick÷time_scale (C-1)

The variable t _{c_sub} is derived as follows and is referred to as the sub-image clock scale: t _{c_sub} =t _c ÷(tick_divisor_minus2+2) (C-2)

In this appendix, specify the following to express constraints:

- causing the access unit n to be the nth access unit in decoding order, wherein the first access unit is an access unit 0 (i.e., the 0th access unit).

- Let the image n be a coded image or a decoded image of the access unit n.

- causing the decoding unit m to be the mth decoding unit in decoding order, wherein the first decoding unit is decoding unit 0.

The above modification of Section C.1 of HEVC Working Draft 8 clarifies the bitstream conformance test. As indicated above, when the decoding program is called for the bitstream conformance test in HEVC Working Draft 8, since the values of TargetDecLayerIdSet and TargetDecHighestTid are not properly set, the semantics of the syntax elements are not clearly specified. The modification of Section C.1 clarifies the definitions of TargetDecLayerIdSet and TargetDecHighestTid.

As shown above in the modification of section C.1 of HEVC Working Draft 8, the device may perform an HRD operation (such as a bitstream conformance test) that selects an operating point and determines the layer identifier of the operating point. The target set (TargetDecLayerIdSet) and the highest time identifier (TargetDecHighestTid). Further, in HRD operation, the device may select a set of HRD parameters suitable for the operating point and configure the HRD to execute the decoding program using the selected set of HRD parameters. The set of HRD parameters applicable to a particular operating point may include parameters specifying an initial CPB removal delay, a CPB size, a bit rate, an initial DPB output delay, a DPB size, and the like. The HRD operation can include performing a decoding process.

In some examples, the device may select a set of HRD parameters suitable for the operating point from one or more of the HRD parameters (eg, hrd_parameters( ) syntax structure) in the VPS and the set of HRD parameters in the SPS. In some instances, when specific operations When the set of layer identifiers contains a set of all layer identifiers present in the coded video sequence associated with the SPS, the device can determine that the set of HRD parameters in the SPS is applicable to the operating point. Moreover, in some examples, the device may select a set of HRD parameters in the SPS in response to determining that the target layer identifier set (eg, TargetDecLayerIdSet) of the operating point contains only a value of zero. In some examples, the device may select a set of HRD parameters in the SPS in response to the set of decision layer identifiers (eg, op_layer_id[ ][ ]) being the same as the target layer identifier set of the operating point (eg, TargetDecLayerIdSet).

Moreover, as shown above in the modification of section C.1 of HEVC Working Draft 8 and elsewhere in the present invention, the device may decode an array of syntax elements (sps_max_dec_pic_buffering[ ]) from the SPS, each of which indicates HRD The maximum required size of the DPB. The device can determine a particular syntax element in the array based on the target highest time identifier (ie, sps_max_dec_pic_buffering[TargetDecHighestTid]). As indicated above, the number of image storage buffers in the DPB is indicated by a particular syntax element (ie, the DPB size (the number of image storage buffers) is sps_max_dec_pic_buffering[TargetDecHighestTid]).

In addition, the decoding program can decode the HRD parameter syntax structure (hrd_parameters( )) that includes the selected set of HRD parameters. The selected set of HRD parameters includes an array of syntax elements (cbp_cnt_minus1[]), each of which indicates the number of alternative CPB specifications in the bitstream. A modification of Section C.1 of HEVC Working Draft 8 clarifies that when the device performs an HRD operation, the device can select a particular syntax element (cpb_cnt_minus1[TargetDecHighestTid]) in the array based on the target highest time identifier (TargetDecHighestTid), and Select a scheduler selection index (SchedSelIdx) from 0 to the value of the specific syntax element. The device can determine an initial CPB removal delay for the CPB of the HRD based at least in part on the scheduler selection index.

Section C.2.1 of HEVC Working Draft 8 relates to the removal of images from the DPB for bitstream consistency. According to the present invention, one or more examples of techniques HEVC Working Draft Section C.2.1 8 of changes made as follows: in this subclause is applied independently to the specifications as specified in subclause C.1 selected parameters of DPB Every collection.

Removing the image from the DPB before the decoding of the current image (but after parsing the tile header of the first tile of the current image) occurs instantaneously in the first decoding unit of the access unit n (containing the current image) The CPB removal time is continued as follows.

The decoding procedure for the reference image set as specified in subclause 8.3.2 is invoked.

If the current image is an IDR or BLA image, the following applies:

1. The value of pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ] derived from the IDR or BLA image that is not the decoded first image and derived from the active sequence parameter set is different from the sequence parameter in the previous image system. When the value of the derived pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ] is set, the HRD infers no_output_of_prior_pics_flag equal to 1 regardless of the actual value of the no_output_of_prior_pics_flag.

Note - In terms of changes to pic_width_in_luma_samples, pic_height_in_luma_samples, or sps_max_dec_pic_buffering[TargetDecHighestTid] , the decoder implementation should attempt to handle image or DPB size changes more appropriately than HRD.

2. When no_output_of_prior_pics_flag is equal to 1 or is inferred to be equal to 1, all image storage buffers in the DPB are emptied without outputting the image it contains, and the DPB saturation is set to zero.

Remove all images k from the DPB that are true for both of the following conditions:

- Image k is marked as "not for reference".

- PicOutputFlag image having k equal to 0, the DPB output time, or less than or equal to the current image n of the first decoding unit (decoding unit denoted as m) CPB removal time; _i.e., t _{o, dpb} (k) <=t _r (m)

When an image is removed from the DPB, the DPB saturation is decremented by one.

As indicated elsewhere in the present invention, the parameter sps_max_dec_pic_buffering[i] may not be properly selected in the HRD operation. As shown above, HEVC Working Draft 8 only indicates sps_max_dec_pic_buffering[i] instead of sps_max_dec_pic_buffering[TargetDecHighestTid]. The HEVC Working Draft 8 does not indicate the semantics of the index i in Section C.2.1. According to one or more techniques of the present invention, specifying the TargetDecHighestTid as the index i of sps_max_dec_pic_buffering[i] ensures that i is equal to the clearly specified value of TargetDecHighestTid in sps_max_dec_pic_buffering[i] when performing the HRD operation of the DPB removal image. .

As demonstrated above in the modification of section C.2.1 of HEVC Working Draft 8, the device may be self-instructed from the first array of SPS decoding syntax elements (sps_max_dec_pic_buffering[]) for the current image in effect, each of which indicates HRD The maximum required size of the DPB. In addition, the device may decode a second array of SPS decoding syntax elements (sps_max_dec_pic_buffering[]) from the active SPS for the previous image, each of the syntax elements indicating the maximum required size of the DPB of the HRD. The device may determine a first syntax element (sps_max_dec_pic_buffering[TargetDecHighestTid]) in the first array based on the target highest time identifier (TargetDecHighestTid). Additionally, the device can determine a second syntax element in the second array (sps_max_dec_pic_buffering[TargetDecHighestTid]) based on the target highest time identifier. The device may infer a third syntax element when the current image is an Instantaneous Decoding (IDR) image or a Broken Link Access (BLA) image and the value of the first syntax element is different from the value of the second syntax element ( The value of no_output_of_prior_pics_flag) is independent of the value indicated by the third syntax element. The third syntax element may specify how to process the previously decoded image in the DPB after decoding the IDR image or the BLA image.

The IDR image may have equal to IDR_W_LP or IDR_N_LP for each tile segment. Random access point (RAP) image of nal_unit_type. The IDR image contains only I tiles and may be the first image in the bitstream in decoding order, or may appear later in the bitstream. An IDR image having nal_unit_type equal to IDR_N_LP does not have an associated preamble image present in the bitstream. The pre-image is an image that precedes the associated RAP image in output order. An IDR image having a nal_unit_type equal to IDR_W_LP does not have an associated mark-off (TFD) image present in the bitstream, but may have an associated DLP image in the bitstream.

The BLA image system has a RAP image of nal_unit_type equal to BLA_W_TFD, BLA_W_DLP or BLA_N_LP per tile segment. A BLA image having a nal_unit_type equal to BLA_W_TFD may have an associated TFD image present in the bitstream. A BLA image having a nal_unit_type equal to BLA_N_LP does not have an associated preamble image present in the bitstream. A BLA image having a nal_unit_type equal to BLA_W_DLP does not have an associated TFD image present in the bitstream, but may have an associated DLP image in the bitstream.

Section C.3 of HEVC Working Draft 8 describes bitstream consistency operations. In accordance with one or more example techniques of the present invention, Section C.3 of HEVC Working Draft 8 may be modified as follows: The bitstream of the coded material consistent with this specification shall meet all of the requirements specified in this subclause.

The bitstream should be constructed according to the syntax, semantics, and constraints specified in this specification outside of this appendix.

The first coded image in the bitstream should be a RAP image, ie an IDR image, a CRA image or a BLA image.

For each current image decoded, the variables maxPicOrderCnt and minPicOrderCnt are set to be equal to the maximum and minimum values of the PicOrderCntVal values of the following images:

- The current image.

- A previous image in decoding order with a TemporalId equal to zero.

- A short-term reference image in the reference image set of the current image.

- all images n with a PicOutputFlag equal to 1 and t _r (n) < t _r (currPic) and t _o,dpb (n)>=t _r (currPic), where currPic is the current image.

All of the following conditions should be met for each of the bitstream conformance tests:

1. for each access unit n associated with a buffering period SEI message (where n > 0), where Δt _g,90 (n) is specified by

Δt _g,90 (n)=90000 *(t _r,n (n)-t _af (n-1)) (C-18)

The value of InitCpbRemovalDelay[SchedSelIdx] shall be subject to the following constraints.

- If cbr_flag[SchedSelIdx] is equal to 0, then InitCpbRemovalDelay[SchedSelIdx]<=Ceil(△t _g,90 (n))(C-19)

- Otherwise (cbr_flag[SchedSelIdx] is equal to 1), Floor(Δt _g,90 (n))<=InitCpbRemovalDelay[SchedSelIdx]<=Ceil(△t _g,90 (n)) (C-20)

NOTE 4 - The exact number of bits in the CPB at the removal time of each image may be based on which buffer period SEI message is selected to initialize the HRD. The encoder must take this into account to ensure that all specified constraints must be taken so that the HRD is initialized regardless of which buffering SEI message is selected, since the HRD can be initialized at any of the buffering period SEI messages.

2. Specify the CPB overflow as the condition that the total number of bits in the CPB is greater than the CPB size. The CPB should never overflow.

3. The CPB under-bit is specified as at least one value for m, the nominal CPB removal time t _r,n (m) of the decoding unit m is less than the condition of the final CPB arrival time t _af (m) of the decoding unit m. When low_delay_hrd_flag is equal to 0, CPB should never be under-bited.

4. When low_delay_hrd_flag is equal to 1, the CPB undershoot can occur at decoding unit m. In this case, the final CPB arrival time t _af (n) of the access unit n containing the decoding unit m should be greater than the nominal CPB removal time t _r,n (n) of the access unit n containing the decoding unit m.

5. The nominal removal time of the image from the CPB (starting from the second image in the decoding order) shall satisfy the pair of _{r r,n} (n) and t expressed in subclauses A.4.1 to A.4.2. The constraint of _r (n).

6. For each current image that is decoded, after calling a program that removes the image from the DPB as specified in subclause C.3.1, includes a PicOutputFlag labeled "for reference" or having a value equal to 1 and _{o, dpb} (n)>=t _r (currPic) (where currPic is the current image) The number of decoded images in the DPB of all images n should be less than or equal to Max (0, sps_max_dec_pic_buffering[TargetDecHighestTid]-1 ).

7. All reference images should be present in the DPB when needed for prediction. Each image with an OutputFlag equal to 1 shall be present in the DPB at its DPB output time, unless the image is removed from the DPB by one of the procedures specified in subclause C.3 before its output time. .

8. For each current image decoded, the value of maxPicOrderCnt-minPicOrderCn should be less than MaxPicOrderCntLsb/2.

9. The Δ _to given by the difference between the output time of the image and the output time of the first image after the image and having a PicOutputFlag equal to 1 in the output order, as given by Equation C-17 _, The value of _dpb (n) shall satisfy the constraints expressed in subclause A.4.1 for the profile, hierarchy and level specified in the bitstream using the decoding procedures specified in clauses 2-9.

As indicated elsewhere in the present invention, the parameter sps_max_dec_pic_buffering[i] may not be properly selected in the bitstream consistency operation. In item 6 of Section C.3, HEVC Working Draft 8 indicates that "the number of decoded pictures in the DPB ... should be less than or equal to Min (0, sps_max_dec_pic_buffering [TemporalId] - 1)", where TemporalId is not defined. In accordance with one or more techniques of the present invention, specifying the TargetDecHighestTid as the index i of sps_max_dec_pic_buffering[i] ensures that i equal to the clearly specified value of TargetDecHighestTid is used in sps_max_dec_pic_buffering[i] when performing the bitstream consistency operation.

When the device performs a decoding process that is part of the HRD operation, the device can decode an array of syntax elements (sps_max_dec_pic_buffering[]) from the SPS, each of which indicates the maximum required size of the DPB of the HRD. In addition, as part of performing the HRD operation, the device can determine a particular syntax element in the array based on the target highest time identifier (TargetDecHighestTid). Furthermore, as shown in the above amendment to section C.3 of HEVC Working Draft 8, the device may be based, at least in part, on whether the number of decoded pictures in the DPB is less than or equal to zero and the value of the specific syntax element minus one. The maximum value determines whether the bit stream is consistent with the video writing standard.

Section C.4 of HEVC Working Draft 8 describes decoder consistency. In accordance with one or more example techniques of the present invention, Section C.4 of HEVC Working Draft 8 may be modified as follows: A decoder consistent with this specification shall meet all of the requirements specified in this subclause.

A decoder claiming consistency with a particular profile, level, and level should be able to successfully decode all bitstreams consistent with the bitstream conformance requirements specified in subclause C.4, as specified in Appendix A. The limitation is that the VCL NAL unit and all video parameter sets , sequence parameter sets, and image parameter sets involved in the appropriate buffering period and image timing SEI message are instantaneously in the bit stream (by non-VCL NAL units). Or pass to the decoder by an external component not specified in this specification.

When a bitstream contains a number of syntax elements having values specified as reserved and specifies that the decoder should ignore the values of the syntax elements or NAL units containing the syntax elements with reserved values and the bitstream is otherwise When this specification is consistent, the consistent decoder shall decode the bitstream in the same manner as the decoder shall consistently decode the bitstream, and if specified, ignore the value of the syntax element or contain such syntax elements with reserved values. NAL unit.

There are two types of consistency that can be claimed by the decoder: output timing consistency and output order consistency.

In order to check the consistency of the decoder, a test bit stream consistent with the claimed profile, hierarchy and level (as specified in subclause C.4) is delivered to the HRD and tested by a hypothetical stream scheduler (HSS). Both decoders (DUTs). All images output by the HRD should also be output by the DUT, and for each image of the HRD output, the value of all samples output by the DUT for the corresponding image should be equal to the value of the sample output by the HRD.

For output timing decoder consistency, the HSS operates as described above with a subset of only the value of the SchedSelIdx selected from the bit rate and CPB size for the specified profile, level, and level as specified in Appendix A. Delivery schedule, or "interpolated" delivery schedule as specified below, with bit rate and CPB size as specified in Appendix A. The same delivery schedule is used for both HRD and DUT.

When there is an HRD parameter and a buffer period SET message with cpb_cnt_minus1[ TargetDecHighestTid ] greater than 0, the decoder should be able to decode as follows: self-use is specified to have peak bit rate r, CPB size c(r), and initial CPB removal delay ( The bit stream of the HSS delivery of the "interpolation" delivery scheduling operation of f(r)÷r): α=(r-BitRate[SchedSelIdx-1])÷(BitRate[SchedSelIdx]-BitRate[SchedSelIdx-1]) , (C-22)

c(r)=α * CpbSize[SchedSelIdx]+(1-α)* CpbSize[SchedSelIdx-1], (C-23)

f(r)=α * InitCpbRemovalDelay[SchedSelIdx]* BitRate[SchedSelIdx]+(1-α)* InitCpbRemovalDelay[SchedSelIdx-1]* BitRate[SchedSelIdx-1] (C-24)

For any SchedSelIdx>0 and r, make BitRate[SchedSelIdx-1]<=r<=BitRate[SchedSelIdx], so that r and c(r) are the largest in the profile, hierarchy and level as specified in Appendix A. Within the limits specified by the bit rate and buffer size.

Note 1 - InitCpbRemovalDelay[SchedSelIdx] is not between buffer cycles Same and must be recalculated.

For output timing decoder consistency, the HRD as described above is used, and up to a fixed delay, the timing of the image output (relative to the delivery time of the first bit) is the same for both the HRD and the DUT.

For output order decoder consistency, the following applies.

- HSS "delivers the bitstream ToDecode to the DUT "by command from the DUT", meaning that the HSS delivers the bits (in decoding order) only when the DUT needs more bits to continue its processing.

NOTE 2 – This situation means that for this test, the DUT's coded image buffer can be as small as the maximum decoding unit size.

- using a modified HRD as described below, and the HSS delivers the bitstream to the HRD by one of the schedules specified in the bitstream ToDecode such that the bitrate and CPB size are as specified in Appendix A limit. The order of image output should be the same for both HRD and DUT.

- For output order decoder consistency, the CPB size is as specified by Equation E-46, CpbSize[SchedSelIdx], where the SchedSelIdx and HRD parameters are selected as specified above in subclause C.1. The DPB size is sps_max_dec_pic_buffering[TargetDecHighestTid]. The removal time from the CPB for the HRD is equal to the final bit arrival time, and the decoding is immediate. The operation of the DPB of this HRD is as described in subclauses C.5.1 to C.5.3.

As indicated elsewhere in the present invention, the parameters cpb_cnt_minus1[i] and sps_max_dec_pic_buffering[i] may not be properly selected in the decoder conformance requirements. For example, section C.4 of HEVC Working Draft 8 does not specify an index for cpb_cnt_minus1. According to one or more techniques of the present invention, specifying the index i of TargetDecHighestTid to cpb_cnt_minus1[i] and sps_max_dec_pic_buffering[i] ensures that the decoder coherency operation is consistently performed, where i is equal to the clear of TargetDecHighestTid Chu specified value.

Furthermore, Section C.4.2 of HEVC Working Draft 8 describes the removal of images from the DPB for decoder consistency. In accordance with one or more example techniques of the present invention, the title of Section C.4.2 can be changed from "Remove Image from DPB" to "Export and Remove Image from DPB". The text of section C.4.2 of HEVC Working Draft 8 can be changed as follows: before decoding the current image (but after parsing the tile header of the first tile of the current image), the image is output and removed from the DPB. This occurs when the first decoding unit of the access unit containing the current picture is removed from the CPB and continues as follows.

The decoding procedure for the reference image set as specified in subclause 8.3.2 is invoked.

- If the current image is an IDR or BLA image, the following applies.

1. The IDR or BLA image is not the decoded first image, and the values of pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ] derived from the active sequence parameter set are different from the sequences in the previous image system. When the parameter set derives the value of pic_width_in_luma_samples or pic_height_in_luma_samples or sps_max_dec_pic_buffering[ TargetDecHighestTid ], the HRD infers no_output_of_prior_pics_flag to be equal to 1 regardless of the actual value of no_output_of_prior_pics_flag.

Note - In terms of changes to pic_width_in_luma_samples, pic_height_in_luma_samples, or sps_max_dec_pic_buffering[TargetDecHighestTid] , the decoder implementation should attempt to handle image or DPB size changes more appropriately than HRD.

2. When no_output_of_prior_pics_flag is equal to 1 or is inferred to be equal to 1, all image storage buffers in the DPB are emptied without outputting the images they contain.

3. When no_output_of_prior_pics_flag is not equal to 1 and is not inferred to be equal to 1, the image storage buffer containing images marked as "not required for output" and "not used for reference" is cleared (no output), And by repeatedly calling the sub-clause The "Up" program in C.5.2.1 clears all non-empty image storage buffers in the DPB.

- Otherwise (the current image is not an IDR or BLA image), the image storage buffer containing images labeled "Not required for output" and "Not for reference" is cleared (no output). When one or more of the following conditions are true, the "lift" program specified in subclause C.5.2.1 is repeatedly called until there is an empty image storage buffer for storing the currently decoded image.

1. The number of images marked as "need for output" in the DPB is greater than sps_max_num_reorder_pics[ TargetDecHighestTid ].

2. The number of images in the DPB is equal to sps_max_dec_pic_buffering[ TargetDecHighestTid ].

"upgrade" procedure

The "lift" program is called under the following conditions.

- The current picture is an IDR or BLA picture as specified in subclause C.5.2, and no_output_of_prior_pics_flag is not equal to 1 and is not inferred to be equal to 1.

- As specified in subclause C.5.2, the current image is not an IDR or BLA image, and the number of images labeled "need for output" in the DPB is greater than sps_max_num_reorder_pics[ TargetDecHighestTid ].

- As specified in subclause C.5.2, the current image is not an IDR or BLA image, and the number of images in the DPB is equal to sps_max_dec_pic_buffering[TargetDecHighestTid].

The "Upgrade" program consists of the following sorted steps:

1. The first output image is selected as the image with the minimum value of PicOrderCntVal in all images labeled "need for output" in the DPB.

2. The image is cropped using the crop rectangle specified in the sequence of parameters in the image, the cropped image is output, and the image is labeled "Not required for output."

3. If the image storage buffer including the cropped and output image contains an image labeled "Not for Reference", the image storage buffer is cleared.

As indicated elsewhere in the present invention, the parameters sps_max_dec_pic_buffering[i] and sps_max_num_reorder_pics[i] may not be properly selected in the HRD operation (such as removing images from the DPB). According to one or more techniques of the present invention, specifying the TargetDecHighestTid as the index i of sps_max_dec_pic_buffering[i] and sps_max_num_reorder_pics[i] ensures that in the sps_max_dec_pic_buffering[i] and sps_max_num_reorder_pics[i] when performing the HRD operation of removing the image from the DPB Use an i equal to the clearly specified value of TargetDecHighestTid.

When the device performs a decoding process during HRD operation, the device can decode an array of syntax elements (sps_max_dec_pic_buffering[]) from the SPS, each of which indicates the maximum required size of the DPB of the HRD. In addition, when the device performs an HRD operation, the device can determine a particular syntax element in the array based on the target highest time identifier (sps_max_dec_pic_buffering[TargetDecHighestTid]). In addition, the device may perform one or more image storage buffers for clearing the DPB when the current image is not an IDR image or a BLA image and the number of images in the DPB that are marked for output is greater than a value of a particular syntax element. Lifting program.

Similarly, when the device performs a decoding process during HRD operation, the device can decode an array of syntax elements (sps_max_dec_pic_buffering[]) from the SPS, each of which indicates the maximum required size of the DPB of the HRD. In addition, when the device performs an HRD operation, the device can determine a particular syntax element in the array based on the target highest time identifier (sps_max_dec_pic_buffering[TargetDecHighestTid]). Further, when the current image is not an IDR image or a BLA image and the number of images in the DPB is equal to the number indicated by the particular syntax element, the device may perform a lifting procedure to clear one or more image storage buffers of the DPB.

Furthermore, according to one or more techniques of the present invention, the applicable_operation_points( ) syntax structure and associated semantics can be added to HEVC Working Draft 8. Table 5 below shows an example syntax for the approximate structure of the applicable_operation_points( ).

The applicable_operation_point() syntax structure shown in Table 5 specifies the point of operation to which the SEI message associated with this syntax structure is applied. The SEI message (also referred to as the associated SEI message) associated with the applicable_operation_point( ) syntax structure is an SEI message containing the applicable structure of the apply_operation_point( ). The SEI message associated with the applicable_operation_point( ) syntax structure may be a buffering period SEI message, an image timing SEI message, or a sub-picture timing SEI message.

The preset operating point may be defined as an operating point identified by an OpLayerIdSet containing a value of 0 to nuh_reserved_zero_6 bits (including 0 and nuh_reserved_zero_6 bits), where the code nuh_reserved_zero_6bits is written in the NAL unit header of the SEI NAL unit containing the associated SEI message, and OpTid Equal to the TemporalId value of the SEI NAL unit containing the associated SEI message. Alternatively, the preset operating point may be defined as an operating point identified by an OpLayerIdSet of nuh_reserved_zero_6bits in the NAL unit header containing only the SEI NAL unit, the SEI NAL unit containing the associated SEI message, and the OpTid being equal to the associated SEI The TemporalId value of the SEI NAL unit of the message. Alternatively, the preset operating point may be defined as an operating point identified by an OpLayerIdSet containing only a value of 0, and the OpTid is equal to the TemporalId value of the SEI NAL unit containing the associated SEI message.

If default_op_applicable_flag is equal to 1, the operating point to which the associated SEI message is applied is the preset operating point, and is operated by OpLayerIdSet as specified by operation_point_layer_ids(i) and num_applicable_ops_minus1 identified by OpTid equal to op_temporal_id[i], where i is in 0 to the range of num_applicable_ops_minus1 (including 0 and num_applicable_ops_minus1). Otherwise (default_op_applicable_flag is equal to 0), the operating point to which the associated SEI message is applied may be the num_applicable_ops_minus1+1 operating point as identified by OpLayerIdSet specified by operation_point_layer_ids(i) and OpTid equal to op_temporal_id[i], where i is at 0 to Within the range of num_applicable_ops_minus1+1 (including 0 and num_applicable_ops_minus1+1).

Furthermore, in the example syntax of Table 5, the num_applicable_ops_minus1 syntax element plus 1 specifies the number of operating points to which the associated SEI message is applied. The value of num_applicable_ops_minus1 can be in the range of 0 to 63 (including 0 and 63). In the example of Table 5, the default_op_applicable_flag syntax element equal to 1 specifies that the associated SEI message is applied to the preset operating point. The default_op_applicable_flag syntax element equal to 0 specifies that the associated SEI message is not applied to the preset operating point. The op_temporal_id[i] syntax element specifies the ith OpTid value that is explicitly sent in the applicable_operation_point( ) syntax structure. The value of op_temporal_id[i] can be in the range of 0 to 6 (including 0 and 6).

As indicated above, HEVC Working Draft 8 does not provide any way to associate a buffering period SEI message or an image timing SEI message to the hrd_parameters( ) syntax structure for which the associated operation_point_layer_ids( ) syntax structure includes Multiple values of nuh_reserved_zero_6bits (ie, multiple layers of HEVC, 3DV or multiple layer IDs in an adjustable video write extension). Including the applicable_operation_point() syntax structure can at least partially solve this problem. The applicable_operation_point( ) syntax structure clearly specifies the operation point to which the buffer period SEI message, the picture timing SEI message, or the sub-picture timing SEI message is applied. This scenario may allow the use of information in the syntax elements nuh_reserved_zero_6bits and temporal_id_plus1 carried in the NAL unit header of the SEL NAL unit, and may allow sharing of information conveyed in the same buffer period, picture timing or sub-picture timing SEI message. To process video material associated with multiple operating points.

Section D.1.1 of HEVC Working Draft 8 describes the syntax of the buffering period SEI message. In accordance with one or more example techniques of the present invention, the buffering period SEI message syntax can be changed as shown in Table 6 below. A change to the buffer period SEI message syntax may enable the buffer period SEI message to include an applicable_operation_points( ) syntax structure.

Section D.2.1 of HEVC Working Draft 8 describes the semantics of the syntax elements of the buffering period SEI message. According to one or more techniques of the present invention, the semantics of the buffering_period (payloadSize) syntax structure can be changed as follows. The semantics of the grammatical elements not mentioned are the same as those in HEVC Working Draft 8.

The Buffer Period SEI message provides information on the initial CPB removal delay and the initial CPB removal delay offset.

The following applies to the buffer period SEI message syntax and semantics:

The syntax elements initial_cpb_removal_delay_length_minus1 and sub_pic_cpb_params_present_flag and the variables NalHrdBpPresentFlag, VclHrdBpPresentFlag, CpbSize[SchedSelIdx], BitRate[SchedSelIdx], and CpbCnt can be seen or derived from the hrd_parameters( ) syntax structure applicable to any of the operating points to which the buffer period SEI message is applied. And in the syntax element in the sub_layer_hrd_parameters( ) syntax structure.

- any two operating point indications to which the buffering period SEI message with different OpTid values tIdA and tIdB is applied: writing cpb_cnt_minus1[tIdA] and cpb_cnt_minus1[tIdB] in the hrd_parameters( ) syntax structure applicable to the two operating points The values are the same.

- Any two operating point points to which the buffering period SEI message with different OpLayerIdSet values of layerIdSetA and layerIdSetB is applied indicates that the values of the respective nal_hrd_parameters_present_flag and vcl_hrd_parameters_present_flag of the two hrd_parameters( ) syntax structures applicable to the two operating points are the same.

The bit stream (or a portion thereof) refers to a subset of bitstreams (or a portion thereof) associated with any of the operating points to which the buffering period SEI message is applied.

If NalHrdBpPresentFlag or VclHrdBpPresentFlag is equal to 1, the buffer period SEI message applicable to the specified operation point may exist in any access unit in the coded video sequence, and the buffer period SEI message applicable to the specified operation point shall exist in each In the RAP access unit, and in each access unit associated with the recovery point SEI message. Otherwise (NalHrdBpPresentFlag and VclHrdBpPresentFlag are both equal to 0), the access unit in the code sequence should not have a buffer period SEI message suitable for the specified operating point.

Note - For some applications, the frequent presence of buffered SEI messages can be desirable.

When a SEI NAL unit containing a buffer period SEI message and having nuh_reserved_zero_6 bits equal to 0 exists, the SEI NAL unit shall precede the first VCL NAL unit in the access unit in decoding order.

The buffer period is specified as a set of access units between two execution entities of a consecutive buffering period SEI message in decoding order.

The variable CpbCnt is derived equal to cpb_cnt_minus1[tId] + 1, where cpb_cnt_minus1[tId] is written to the hrd_parameters( ) syntax structure applicable to any of the operating points applied to the buffer period SEI message and having an OpTid equal to tId in.

Seq_parameter_set_id refers to the set of active sequence parameters. The value of seq_parameter_set_id shall be equal to the value of seq_parameter_set_id in the image parameter set of the coded image reference associated with the buffering period SEI message. The value of seq_parameter_set_id should be in the range of 0 to 31 (including 0 and 31).

The rap_cpb_params_present_flag equal to 1 specifies the existence of the initial_alt_cpb_removal_delay[SchedSelIdx] and initial_alt_cpb_removal_delay_offset[SchedSelIdx] syntax elements. When not present, the value of alt_cpb_params_present_flag is inferred to be equal to zero. When the associated image is neither a CRA image nor a BLA image, the value of alt_cpb_params_present_flag should be equal to zero.

the initial_cpb_removal_delay [SchedSelIdx] and initial_alt_cpb_removal_delay [SchedSelIdx] specify the preset first SchedSelIdx th CPB and CPB alternative initial removal delay. The syntax element has a length in bits given by initial_cpb_removal_delay_length_minus1+1 and is in units of 90 kHz clock. The value of the syntax element should not be equal to 0, and should be less than or equal to 90000 *(CpbSize[SchedSelIdx]÷BitRate[SchedSelIdx]), which is the time equivalent of the CPB size in units of 90 kHz clock.

initial_cpb_removal_delay_offset [SchedSelIdx] and initial_alt_cpb_removal_delay_offset [SchedSelIdx] specify the preset first SchedSelIdx th CPB and CPB removal alternative initial offset. The syntax element has a length in bits given by initial_cpb_removal_delay_length_minus1+1 and is in units of 90 kHz clock. These syntax elements are not used by the decoder and may only be used for the Delivery Scheduler (HSS) specified in Appendix C.

The buffering period SEI message may include HRD parameters (eg, initial_cpb_removal_delay[SchedSelIdx], initial_cpb_removal_delay_offset[SchedSelIdx], initial_alt_cpb_removal_delay[SchedSelIdx], and initial_alt_cpb_removal_delay_offset[SchedSelIdx]). As indicated above, HEVC Working Draft 8 does not provide any way to associate a buffering period SEI message to the hrd_parameters( ) syntax structure in the VPS for which the associated operation_point_layer_ids( ) syntax structure includes more nuh_reserved_zero_6bits Values (ie, multiple views of HEVC, 3DV or multiple layer IDs in an adjustable video write extension). Thus, in accordance with one or more techniques of the present invention, the apply_operation_points( ) syntax element in the buffering period SEI message specifies the operating point to which the buffering period SEI message is applied.

Section D.1.2 of HEVC Working Draft 8 indicates the syntax of the image timing SEI message. In accordance with one or more techniques of the present invention, the syntax of the image timing SEI message can be changed as shown in Table 7 below. Changes to the image timing SEI message syntax can enable image timing SEI messages to Enough to include the applicable_operation_points() syntax structure.

In addition, the semantics of the image timing SEI message can be changed as follows. The semantics of the syntax elements of the pic_timing(payloadSize) syntax structure not mentioned below may be identical to their semantics in HEVC Working Draft 8.

The picture timing SEI message provides information on the CPB removal delay and DPB output delay of the access unit associated with the SEI message.

The following applies to image timing SEI message syntax and semantics:

The syntax elements sub_pic_cpb_params_present_flag, cpb_removal_delay_length_minus1, dpb_output_delay_length_minus1, and du_cpb_removal_delay_length_minus1 and the variable CpbDpbDelaysPresentFlag may be obtained or derived from the syntax of the hrd_parameters( ) syntax structure and the sub_layer_hrd_parameters( ) syntax structure applicable to the operation point to which the image timing SEI message is applied. In the element.

The bit stream (or a portion thereof) refers to a subset of bitstreams (or a portion thereof) associated with any of the operating points to which the image timing SEI message is applied.

Note 1 - Image Timing The syntax of the SEI message depends on the content of the hrd_parameters( ) syntax structure that applies to the operation point to which the image timing SEI message is applied. These hrd_parameters( ) syntax structures are in a set of video parameter sets and/or sequence parameters in the action of the coded image system associated with the image timing SEI message. When the picture timing SEI message is associated with a CRA access unit, an IDR access unit, or a BLA access unit of a first access unit in a bit stream, unless a buffer period SEI message within the same access unit Prior to the image timing SEI message, otherwise the video parameter set and the sequence parameter set are initiated (and for IDR or BLA images that are not the first image in the bitstream, the coded image IDR image or The BLA image does not occur until the first coded block NAL unit of the coded image is decoded. Since the coded block NAL unit of the coded picture is after the picture timing SEI message in the NAL unit order, there may be a situation in which it is necessary for the decoder to store the RBSP containing the picture timing SEI message until the decision is active. The video parameter set and/or the active sequence parameter set are up, and then the analysis of the image timing SEI message is performed.

The presence of the picture timing SEI message in the bit stream is specified as follows.

- If CpbDpbDelaysPresentFlag is equal to 1, then an image timing SEI message suitable for the specified operating point shall be present in each access unit of the coded video sequence.

- Otherwise (CpbDpbDelaysPresentFlag is equal to 0), the image timing SEI message for the specified operating point should not be present in any access unit of the coded video sequence.

When an SEI NAL unit containing an image timing SEI message and having nuh_reserved_zero_6 bits equal to 0 exists, the SEI NAL unit shall precede the first VCL NAL unit in the access unit in decoding order.

Au_cpb_removal_delay_minus1 plus 1 specifies that the access unit associated with the most recent buffering period SEI message in the previous access unit is removed from the CPB when the HRD is operating at the access unit level, after the self-buffer removal and image timing SEI How many clock ticks to wait before the access unit data associated with the message. This value is also used to calculate the earliest possible time for the access unit data to reach the CPB of the HSS. The syntax element is a fixed length code whose length in bits is given by cpb_removal_delay_length_minus1+1.

Note 2 - The value of cpb_removal_delay_length_minus1 of the length (in bits) of the decision syntax element au_cpb_removal_delay_minus1 is written in the video parameter set or sequence parameter set in the action of the coded picture system associated with the image timing SEI message. The value of cpb_removal_delay_length_minus1, although au_cpb_removal_delay_minus1 plus 1 specifies the number of clock scales relative to the removal time of the previous access unit containing the buffer period SEI message, the previous access unit may be stored in different coded video sequences Take the unit.

Pic_dpb_output_delay is used to calculate the DPB output time of the image. It specifies how many clock scales to wait after outputting the decoded image from the DPB after the last decoding unit in the access unit has been removed from the CPB.

Note 3 - When the image is still marked as "for short-term reference" or "for long-term reference", the image is not removed from the DPB at the output time of the image.

Note 4 - Specify only one pic_dpb_output_delay for the decoded image.

The length of the syntax element pic_dpb_output_delay is given in bits by dpb_output_delay_length_minus1+1. When sps_max_dec_pic_buffering[minTid] is equal to 1, pic_dpb_output_delay should be equal to 0, where minTid is the minimum of the OpTid values applied to all operating points by the image timing SEI message.

The output time derived from pic_dpb_output_delay of any image output from the output timing-consistent decoder shall be prior to the output time derived from pic_dpb_output_delay of all images in any subsequent code-coded video sequence in decoding order.

The order in which the images are output by the values of the syntax elements should be in the same order as the order established by the value of PicOrderCntVal.

For images that are not output by the "lift" program, because the images precede the no_output_of_prior_pics_flag with a value equal to 1 or inferred to be equal to 1 in decoding order. The IDR or BLA image, so the output time derived from pic_dpb_output_delay should increase with increasing value of PicOrderCntVal relative to all images within the same coded video sequence.

Du_common_cpb_removal_delay_flag equal to 1 specifies the presence of the syntax element du_common_cpb_removal_delay_minus1. Du_common_cpb_removal_delay_flag equal to 0 specifies that the syntax element du_common_cpb_removal_delay_minus1 does not exist.

Du_common_cpb_removal_delay_minus1 plus 1 specifies how many sub-images to wait before each decoding unit in the access unit associated with the image timing SEI message is removed from the CPB after the previous decoding unit in the decoding order is removed from the CPB Pulse scale (see subclause E.2.1). As specified in Appendix C, this value is also used to calculate the earliest possible time for the decoding unit data to reach the CPB of the HSS. The syntax element is a fixed length code whose length is given in units of bits by du_cpb_removal_delay_length_minus1+1.

As indicated above, HEVC Working Draft 8 does not provide any way to associate an image timing SEI message to the hrd_parameters( ) syntax structure in the VPS for which the associated operation_point_layer_ids( ) syntax structure includes nuh_reserved_zero_6bits Multiple values (ie, multiple views of HEVC, 3DV or multiple layer IDs in an adjustable video write extension). Thus, in accordance with one or more techniques of the present invention, the apply_operation_points( ) syntax element in the image timing SEI message specifies the operating point to which the buffering period SEI message is applied.

Moreover, in accordance with one or more techniques of the present invention, the syntax of the sub-picture timing SEI message can be changed as shown in Table 8 below. The change to the sub-picture timing SEI message syntax may enable the sub-picture timing SEI message to include the applicable_operation_points( ) syntax structure. In HEVC Working Draft 8, the sub-picture timing SEI message does not include the applicable_operation_points() syntax structure.

Table 8 - Sub-image timing SEI messages

Section D.2.2.2 of HEVC Working Draft 8 describes the semantics of sub-picture timing SEI messages. In accordance with one or more techniques of the present invention, Section D.2.2.2 of HEVC Working Draft 8 may be modified as follows: Sub-picture timing The SEI message provides CPB removal delay information for the decoding unit associated with the SEI message.

The following applies to sub-picture timing SEI message syntax and semantics:

The syntax elements sub_pic_cpb_params_present_flag and cpb_removal_delay_length_minus1 and the variable CpbDpbDelaysPresentFlag may be found or derived from syntax elements in the hrd_parameters( ) syntax structure and the sub_layer_hrd_parameters( ) syntax structure applicable to any of the operation points to which the sub-picture timing SEI message is applied.

The bit stream (or a portion thereof) refers to a subset of bitstreams (or a portion thereof) associated with any of the operating points to which the sub-picture timing SEI message is applied.

The presence of the sub-picture timing SEI message in the bit stream is specified as follows.

- If CpbDpbDelaysPresentFlag is equal to 1 and sub_pic_cpb_params_present_flag is equal to 1, then a sub-picture timing SEI message suitable for the specified operating point may be present in each decoding unit in the coded video sequence.

- Otherwise (CpbDpbDelaysPresentFlag is equal to 0 or sub_pic_cpb_params_present_flag is equal to 0), sub-picture timing SEI messages applicable to the specified operating point shall not be present in the coded video sequence.

The decoding unit associated with the sub-picture timing SEI message is in decoding order by the SEL NAL unit containing the sub-picture timing SEI message followed by one or more NAL units (including access) that do not contain the sub-picture timing SEI message Up to (but not including) sub-image timing in the unit Composed of all subsequent NAL units of any subsequent SEI NAL unit of the SEI message. There should be at least one VCL NAL unit in each decoding unit. All non-VCL NAL units associated with a VCL NAL unit shall be included in the same decoding unit.

Du_spt_cpb_removal_delay_minus1 plus 1 specifies that after the CPB removes the last decoding unit in the access unit associated with the most recent buffering period SEI message in the previous access unit, the removal from the CPB is associated with the sub-picture timing SEI message How many sub-image clock scales to wait before decoding the unit. As specified in Appendix C, this value is also used to calculate the earliest possible time for the decoding unit data to reach the CPB of the HSS. The syntax element is represented by a fixed length code whose length is given by cpb_removal_delay_length_minus1+1 in bits.

Remarks - The value of cpb_removal_delay_length_minus1 of the length (in bits) of the decision syntax element du_spt_cpb_removal_delay_minus1 is in the video parameter set or sequence parameter set in the access unit system for the decoding unit associated with the sub-picture timing SEI message. The value of the coded cpb_removal_delay_length_minus1, although du_spt_cpb_removal_delay_minus1 plus 1 specifies the number of sub-image clock scales relative to the removal time of the last decoding unit in the previous access unit containing the buffer period SEI message, the previous save The fetch unit can be an access unit in a different coded video sequence.

Section E.2.2 of HEVC Working Draft 8 describes HRD parameter semantics. In accordance with one or more of the techniques of the present invention, Section E.2.2 of HEVC Working Draft 8 can be changed as follows. The semantics of the grammatical elements of the HRD parameters not mentioned below may be identical to their semantics in HEVC Working Draft 8.

The hrd_parameters( ) syntax structure provides HRD parameters for use in HRD operations. When the hrd_parameters( ) syntax structure is included in the video parameter set, the set of numbers of nuh_reserved_zero_6bits values included in the OpLayerIdSet applied to the operation point of the syntax structure is determined by the corresponding operation_point_layer_ids() syntax structure in the video parameter set. Specify or implicitly export, as specified in subclause 7.4.4. When the hrd_parameters( ) syntax structure is included in the sequence parameter set, the applicable operating point has all the operating points with only the OpLayerIdSet of value 0. Alternatively, when the hrd_parameters( ) syntax structure is included in the sequence parameter set, the applicable operating point has all the operating points of the OpLayerIdSet that are the same as the TargetDecLayerIdSet.

One of the requirements for bitstream consistency is that for all hrd_parameters() syntax structures in a coded video sequence (video parameter set or sequence parameter set), there should be no more than one hrd_parameters() syntax structure applicable to the same operating point. . Alternatively, it is required that there should be no more than one hrd_parameters( ) syntax structure applicable to the same operating point in the video parameter set. Alternatively, the set of video parameters is required to not include the hrd_parameters( ) syntax structure applicable to an operation point having an OpLayerIdSet containing only a value of zero.

Du_cpb_removal_delay_length_minus1 plus 1 specifies the length of the du_cpb_removal_delay_minus1[i] and du_common_cpb_removal_delay_minus1 syntax elements of the image timing SEI message in bits.

Cpb_removal_delay_length_minus1 plus 1 specifies the length of the au_cpb_removal_delay_minus1 syntax element in the image timing SEI message and the du_spt_cpb_removal_delay_minus1 syntax element in the sub-picture timing SEI message in bits. When the cpb_removal_delay_length_minus1 syntax element does not exist, it is inferred to be equal to 23.

Dpb_output_delay_length_minus1 plus 1 specifies the length of the pic_dpb_output_delay syntax element in the image timing SEI message in bits. When the dpb_output_delay_length_minus1 syntax element does not exist, it is inferred to be equal to 23.

A fixed_pic_rate_flag [i] equal to 1 indicates that when TargetDecHighestTid is equal to i, the time distance between the HRD output times of any two consecutive images in the output order is constrained as follows. A fixed_pic_rate_flag[i] equal to 0 indicates that there is no such time constraint as the time between the HRD output times of any two consecutive images on the output order.

When fixed_pic_rate_flag[i] does not exist, it is inferred to be equal to zero.

For a coded video sequence containing image n, when TargetDecHighestTid is equal to i and fixed_pic_rate_flag[i] is equal to 1, the calculated value of Δt _{o, dpb} (n) as specified in Equation C-17 shall be equal to t. _c *(pic_duration_in_tcs_minus1[i]+1), wherein when one or more of the following conditions are true for the following image nn specified for use in Equation C-17, t _c is as Equation C-1 Specified (using the t _c value of the coded video sequence containing image n):

- Image nn is in the same coded video sequence as image n.

- Image nn is in a different coded video sequence, and fixed_pic_rate_flag[i] is equal to 1, in the coded video sequence containing image nn, the value of num_units_in_tick÷time_scale is the same for both coded video sequences And the value of pic_duration_in_tc_minus1[i] is the same for the two coded video sequences.

Pic_duration_in_tc_minus1 [i] plus 1 specifies the time distance in hours of the HRD output time between any two consecutive images in the output order in the coded video sequence when TargetDecHighestTid is equal to i. The value of pic_duration_in_tc_minus1[i] should be in the range of 0 to 2047 (including 0 and 2047).

As specified in Appendix C, when TargetDecHighestTid is equal to i, low_delay_hrd_flag [i] specifies the HRD mode of operation. When fixed_pic_rate_flag[i] is equal to 1, low_delay_hrd_flag[i] should be equal to zero.

Note 3 - When low_delay_hrd_flag[i] is equal to 1, a "large image" that violates the nominal CPB removal time due to the number of bits used by the access unit is permitted. It is expected that such "big images" will not be required to occur only occasionally.

Cpb_cnt_minus1 [i] plus 1 specifies the number of alternative CPB specifications in the bitstream of the coded video sequence when TargetDecHighestTid is equal to i. The value of cpb_cnt_minus1[i] should be in the range of 0 to 31 (including 0 and 31). When low_delay_hrd_flag[i] is equal to 1, cpb_cnt_minus1[i] should be equal to zero. When cpb_cnt_minus1[i] does not exist, it is inferred to be equal to zero.

As described elsewhere in the present invention, in HEVC Working Draft 8, only the hrd_parameters( ) syntax structure in the VPS can be selected for HRD operations, and the hrd_parameters( ) syntax structure in the SPS is never selected. The changes to the semantics of the hrd_parameters( ) syntax structure shown above clarify that when the hrd_parameters( ) syntax structure is included in the SPS, the hrd_parameters( ) syntax structure applies to all operating points that have the same OpLayerIdSet as the TargetDecLayerIdSet. . As indicated above in the modified general decoding procedure, if an external component can be used to set the TargetDecLayerIdSet, the TargetDecLayerIdSet can be specified by the external component. Otherwise, if the decoder is called in the bitstream conformance test, the TargetDecLayerIdSet can be the set of layer identifiers of the tested operating points. Otherwise, the TargetDecLayerIdSet may contain only one layer identifier (ie, only one value of nuh_reserved_zero_6bits), which is equal to zero. In one example, the external component can be an API that is part of the terminal implementation and provides the functionality to set the value of the TargetDecLayerIdSet. In this example, the terminal implementation may include certain functions of the decoder implementation and not part of the decoder implementation.

In this manner, the device (such as video encoder 20, video decoder 30, additional device 21, or another device) can be selected from a set of HRD parameters in the video parameter set and a set of HRD parameters in the SPS for a particular operating point. A collection of HRD parameters. In addition, the device can perform a bitstream conformance test based at least in part on a set of HRD parameters suitable for a particular operating point, the test testing whether a subset of bitstreams associated with a particular operating point is consistent with a video writing standard.

As indicated above, Section E.2.2 of HEVC Working Draft 8 may be modified to indicate The hrd_parameters( ) syntax structure is included in the sequence parameter set. The operating points are all operating points with the same OpLayerIdSet of TargetDecLayerIdSet. Furthermore, as described above, the TargetDecLayerIdSet is set to targetOpLayerIdSet, which contains a set of values of nuh_reserved_zero_6bits present in the subset of bitstreams associated with TargetOp. TargetOp is the measured operating point in HRD operation. In addition, HRD operations (eg, bitstream conformance testing and decoder conformance testing) can invoke a general decoding procedure.

As explained above, Section 8.1 of HEVC Working Draft 8 can be modified to specify that the sub-bitstream extractor as specified in subclause 10.1 is applied with TargetDecHighestTid and TargetDecLayerIdSet as inputs, and the output is assigned to a call called BitstreamToDecode. Bit stream. Therefore, the only nuh_reserved_zero_6bits value present in BitstreamToDecode is the value of nuh_reserved_zero_6bits in TestDecLayerIdSet (ie, the set of values of nuh_reserved_zero_6bits present in the subset of bitstreams associated with TargetOp). Section 8.1 further explains that when it comes to interpreting the semantics of each syntax element in each NAL unit and "bitstream" or part thereof (eg, a coded video sequence), the bitstream or part thereof means BitstreamToDecode. Or part thereof.

Thus, when interpreting a section describing the semantics of an HRD parameter (eg, section E.2.2 of HEVC Working Draft 8), the term "coded video sequence" means part of BitstreamToDecode. TargetDecLayerIdSet is equivalent to a collection of all values of nuh_reserved_zero_6bits present in BitstreamToDecode. Therefore, it is concluded that the phrase "When the hrd_parameters( ) syntax structure is included in the sequence parameter set, the applicable operation point has all the operation points of the OpLayerIdSet identical to the TargetDecLayerIdSet" is equivalent to "when hrd_parameters (when the hrd_parameters() syntax structure is included in the sequence parameter set" When the grammatical structure is included in the sequence parameter set, the applicable operation point has the same set of values as nuh_reserved_zero_6bits existing in the BitstreamToDecode. All operating points of OpLayerIdSet".

Since the "coded video sequence" is part of BitstreamToDecode, the set of nuh_reserved_zero_6bits present in the coded video sequence is present in a subset of the set of nuh_reserved_zero_6bits in BitstreamToDecode. Therefore, the phrase "When the hrd_parameters() syntax structure is included in the sequence parameter set, the applicable operation point is all the operation points of the OpLayerIdSet having the same set of values of nuh_reserved_zero_6bits existing in the BitstreamToDecode"" when the hrd_parameters() syntax is required. When the structure is included in the sequence parameter set, the applicable operating point has all the operating points of the OpLayerIdSet containing all the values of nuh_reserved_zero_6bits present in the coded video sequence. In other words, if the set of nuh_reserved_zero_6bits of the operation point is the same as the set of nuh_reserved_zero_6bits existing in BitstreamToDecode, the set of nuh_reserved_zero_6bits of the operation point necessarily contains all the nuh_reserved_zero_6bits values existing in the coded video sequence of BitstreamToDecode. In this phrase, a "coded video sequence" may refer to a coded video sequence associated with a particular SPS.

When performing the HRD operation, the device can determine the hrd_parameters() syntax structure applicable to TargetOp from the hrd_parameters() syntax structure in the VPS and the hrd_parameters() syntax structure indicated in the SPS. If the layer id set of TargetOp matches the set of layer identifiers specified in the VPS for a particular hrd_parameters() syntax structure, then the particular hrd_parameters() syntax structure in the VPS applies to TargetOp. If the TargetOp layer id set (ie, TargetDecHighestTid) (ie, the set of nuh_reserved_zero_6bits present in BitstreamToDecode) contains all of the nuh_reserved_zero_6bits (which are the set of nuh_reserved_zero_6bits in the BitstreamToDecode) present in the SPS coded video sequence. Set), the hrd_parameters() syntax structure in SPS can be applied to TargetOp. Because the set of nuh_reserved_zero_6bits of TargetOp may necessarily contain the existence of the association associated with SPS. All nuh_reserved_zero_6bits values in the coded video sequence are written, so the hrd_parameters() syntax structure in SPS can always be applied to TargetOp. However, not all SPSs have the hrd_parameters() syntax structure. If the SPS does not have the hrd_parameters() syntax structure and the set of nuh_reserved_zero_6bits present in the BitstreamToDecode contains all of the nuh_reserved_zero_6bits present in the SPS codec video sequence, the SPS's hrd_parameters() syntax structure should be used. Since not all SPSs have the hrd_parameters() syntax structure, the VPS can still be selected.

Furthermore, as shown above in the modification of section E.2.2 of HEVC Working Draft 8, when the device performs bitstream conformance testing, the video decoder can be in all sets of HRD parameters in the coded video sequence. In the case where one or more sets of HRD parameters are applied to the same operating point, the bit stream is determined not to conform to the video writing standard. In addition, when the device performs bitstream conformance testing, the video decoder can determine that the bitstream is not consistent with the video writing standard when more than one set of HRD parameters in the VPS is applied to the same operating point. In addition, when the device performs a bitstream decoding test, the device can determine that the bitstream does not conform to the video writing standard when the VPS includes a set of HRD parameters applicable to an operating point having a layer id set containing only a value of zero.

3 is a block diagram illustrating an example video decoder 30 that is configured to implement the techniques of the present invention. 3 is provided for purposes of explanation and does not limit the techniques as broadly illustrated and described in the present invention. For purposes of explanation, the present invention describes video decoder 30 in the context of the HEVC code. However, the techniques of the present invention are applicable to other writing standards or methods.

In the example of FIG. 3, video decoder 30 includes an entropy decoding unit 150, a prediction processing unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, a reconstruction unit 158, a filtering unit 160, and a decoded image buffer 162. The prediction processing unit 152 includes a motion compensation unit 164 and an in-frame prediction processing unit 166. In other examples, video decoder 30 may include more, fewer, or different functional components.

The coded image buffer (CPB) 151 can receive and store the encoded video material (e.g., NAL unit) of the bit stream. Entropy decoding unit 150 may receive NAL units from CPB 151 and parse the NAL units to decode syntax elements. Entropy decoding unit 150 may entropy decode the entropy encoded syntax elements in the NAL unit. Prediction processing unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filtering unit 160 may generate decoded video material based on the extracted syntax elements from the bitstream.

The NAL unit of the bitstream may include a coded tile NAL unit. As part of the decoded bitstream, entropy decoding unit 150 may extract the syntax elements from the coded tile NAL unit and entropy decode the syntax elements. Each of the coded tiles may include a tile header and tile data. A tile header can contain syntax elements about a tile. The syntax element in the tile header may include a syntax element that identifies the PPS associated with the image containing the tile.

In addition to decoding syntax elements from the bitstream, video decoder 30 may also perform reconstruction operations on the non-segmented CUs. In order to perform a reconstruction operation on a non-segmented CU, video decoder 30 may perform a reconstruction operation on each TU of the CU. By performing a reconstruction operation for each TU of the CU, video decoder 30 may reconstruct the residual block of the CU.

As part of performing a reconstruction operation on the TU of the CU, inverse quantization unit 154 may inverse quantize (ie, dequantize) the coefficient block associated with the TU. The inverse quantization unit 154 may determine the degree of quantization using the QP value associated with the CU of the TU, and similarly determine the degree of inverse quantization to be applied by the inverse quantization unit 154. That is, the compression ratio (i.e., the ratio of the number of bits used to represent the initial sequence to the number of bits of the compressed sequence) can be controlled by adjusting the value of the QP used when quantizing the transform coefficients. The compression ratio may also depend on the method of entropy writing used.

After inverse quantization unit 154 inverse quantizes the coefficient block, inverse transform processing unit 156 can apply one or more inverse transforms to the coefficient block to generate a residual block associated with the TU. For example, the inverse transform processing unit 156 can perform inverse DCT, inverse integer transform, Karhunen-Loeve transform (KLT), inverse rotation transform, and inverse Apply a transform or another inverse transform to the coefficient block.

If the PU is encoded using intra-frame prediction, in-frame prediction processing unit 166 may perform in-frame prediction to generate a predictive block for the PU. In-frame prediction processing unit 166 may use the in-frame prediction mode to generate predictive luma, Cb, and Cr blocks for the PU based on the predicted blocks of spatially neighboring PUs. In-frame prediction processing unit 166 may determine an intra-frame prediction mode for the PU based on decoding one or more syntax elements from the bit stream.

The prediction processing unit 152 may construct a first reference image list (RefPicList0) and a second reference image list (RefPicList1) based on the syntax elements extracted from the bit stream. Furthermore, if the PU is encoded using inter-frame prediction, the entropy decoding unit 150 may extract motion information for the PU. Motion compensation unit 164 can determine one or more reference regions of the PU based on the motion information of the PU. Motion compensation unit 164 may generate predictive luma, Cb, and Cr blocks for the PU based on the one or more reference blocks at the PU.

Reconstruction unit 158 may use the luma, Cb, and Cr transform blocks associated with the TU of the CU and the predictive luma, Cb, and Cr blocks of the PU of the CU (ie, in-frame prediction data or inter-frame prediction data, When applicable, to reconstruct the CU's brightness, Cb, and Cr code blocks. For example, reconstruction unit 158 may add samples of luma, Cb, and Cr transform blocks to corresponding samples of predictive luma, Cb, and Cr blocks to reconstruct the luma, Cb, and Cr code blocks of the CU.

Filtering unit 160 may perform a deblocking operation to reduce blockiness artifacts associated with the CU's brightness, Cb, and Cr code blocks. Video decoder 30 may store the CU's luma, Cb, and Cr code blocks in decoded image buffer 162. The decoded image buffer 162 can provide a reference image for subsequent motion compensation, in-frame prediction, and rendering on a display device, such as display device 32 of FIG. For example, video decoder 30 may perform intra-frame prediction or inter-frame prediction operations on PUs of other CUs based on the luma, Cb, and Cr blocks in decoded image buffer 162. In this manner, video decoder 30 can decode the transform coefficient level of the effective luma coefficient block from the bit stream, inverse quantize the transform coefficient level, and apply the transform to the transform system. The number level is to generate a transform block, the write code block is generated based at least in part on the transform block, and the write code block is output for display.

4 is a flow diagram illustrating an example operation 200 of a device in accordance with one or more techniques of the present invention. Operation 200 may be performed by video encoder 20, video decoder 30, additional device 21, or another device. As illustrated in the example of FIG. 4, the device may select an HRD parameter suitable for a particular operating point of the bitstream from a set of hypothetical HRD parameters (eg, hrd_parameters syntax structure) in the VPS and a set of HRD parameters in the SPS. Collection (202). Moreover, the device can perform an HRD operation on the subset of bitstreams associated with a particular operating point based at least in part on a set of HRD parameters suitable for a particular operating point (204). For example, the device may perform bitstream conformance testing or decoder conformance testing.

FIG. 5 is a flow diagram illustrating an example operation 250 of a device in accordance with one or more techniques of the present invention. Operation 200 may be performed by video encoder 20, video decoder 30, additional device 21, or another device. As illustrated in the example of FIG. 5, the device can perform a bitstream conformance test (252) that determines if the bitstream is consistent with the video writing standard. The device can execute a decoding program (254) as part of performing a bitstream conformance test.

As illustrated in the example of FIG. 5, when performing the decoding process, the device executable bitstream extraction program extracts the operation point representation of the operation point defined by the target set of the layer identifier and the target highest time identifier from the bit stream. (256). The target set of layer identifiers may contain values of layer identifier syntax elements that are present in the operation point representation. The target set of layer identifiers may be a subset of the layer identifier syntax element values of the bit stream. The target highest time identifier may be equal to the maximum time identifier present in the operation point representation, the target highest time identifier being less than or equal to the maximum time identifier present in the bit stream. In addition, the device can decode the NAL unit represented by the operating point (258).

6 is a flow diagram illustrating an example HRD operation 300 of a device in accordance with one or more techniques of the present invention. HRD operation 300 may be performed by video encoder 20, video decoder 30, additional device 21, or another device. Other devices may include a consistent bitstream checker, which checks The inspector takes the bit stream as an input and outputs an indication of whether the input bit stream is a consistent bit stream. In some examples, HRD operation 300 can determine the consistency of the bitstream with the video coding standard. In other examples, HRD operation 300 can determine the consistency of the decoder with the video writing standard. As part of performing HRD operation 300, the device can determine the highest time identifier (302) of the subset of bitstreams associated with the selected operating point of the bitstream. In addition, the device may determine a particular syntax element (304) from an array of syntax elements (eg, sps_max_num_reorder_pics[i], sps_max_dec_pic_buffering[i], and cpb_cnt_minus1[i]) based on the highest temporal identifier. The device may use a particular syntax element (306) in the HRD operation.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on or transmitted through a computer readable medium and executed by a hardware-based processing unit. The computer readable medium can include a computer readable storage medium (which corresponds to a tangible medium such as a data storage medium) or communication medium including, for example, any medium that facilitates transfer of the computer program from one location to another in accordance with a communication protocol . In this manner, computer readable media generally may correspond to (1) a tangible, tangible computer readable storage medium, or (2) a communication medium such as a signal or carrier. The data storage medium can be any available media that can be accessed by one or more computers or one or more processors to capture instructions, code, and/or data structures for use in carrying out the techniques described in the present invention. The computer program product can include a computer readable medium.

By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, flash memory, or may be used for storage Any other medium that is in the form of an instruction or data structure and that is accessible by a computer. Also, any connection is properly termed a computer-readable medium. For example, if you use coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technology (such as infrared, radio, and microwave) Coax, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of the media when the station, server, or other remote source transmits instructions. However, it should be understood that computer readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead are directed to non-transitory tangible storage media. As used herein, magnetic disks and optical disks include compact discs (CDs), laser compact discs, optical compact discs, digital audio and video discs (DVDs), flexible magnetic discs, and Blu-ray discs, in which magnetic discs are typically magnetically regenerated, while optical discs are used. Optically regenerating data by laser. Combinations of the above should also be included in the context of computer readable media.

One or more of such equivalent integrated or discrete logic circuits, such as one or more digital signal processors (DSPs), general purpose microprocessors, special application integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits Processors to execute instructions. Thus, the term "processor" as used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Moreover, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. . Moreover, the techniques can be implemented entirely in one or more circuits or logic elements.

The techniques of the present invention can be implemented in a wide variety of devices or devices, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., wafer sets). Various components, modules or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but are not necessarily required to be implemented by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit, or by interoperable hardware units incorporating suitable software and/or firmware (including one as described above or A collection of multiple processors) to provide such units.

Various examples have been described. These and other examples are within the scope of the following claims.

200‧‧‧Instance operations

Claims (26)

A method for processing video data, the method comprising: selecting from a set of hypothetical reference decoder (HRD) parameters in a video parameter set (VPS) and a set of HRD parameters in a sequence parameter set (SPS) a set of HRD parameters for a particular operating point of a one-bit stream; and one of the bitstreams associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point The set performs an HRD operation. The method of claim 1, wherein the set of HRD parameters selected for the particular operating point is included when a set of layer identifiers of the particular operating point contains a set of all layer identifiers present in the subset of bitstreams It is determined that the set of HRD parameters in the SPS is applicable to the particular operating point. The method of claim 1, wherein the set of HRD parameters applicable to the particular operating point comprises parameters specifying: an initial write code image buffer (CPB) removal delay, a CPB size, a bit element Rate, an initial decoded image buffer (DPB) output delay, and a DPB size. The method of claim 1, further comprising: determining that the specific operating point contains a target layer identifier set of each layer identifier present in the subset of bit streams, wherein the target layer identifier of the specific operating point A set is a subset of the layer identifiers present in the bitstream, and a target time identifier equal to one of the maximum time identifiers present in the subset of bitstreams is determined to be the specific operating point, wherein the specific The target time identifier of the operating point is less than or equal to the maximum time identifier present in the bit stream. The method of claim 4, wherein the selecting the set of HRD parameters comprises selecting the SPS in response to determining that the target layer identifier set of the particular operating point contains only a value of 0 This set of HRD parameters. The method of claim 4, wherein the selecting the set of HRD parameters comprises selecting the HRD in the SPS in response to determining that the set of layer identifiers specified in the SPS is the same as the target layer identifier set of the particular operating point This collection of parameters. The method of claim 1, wherein performing the HRD operation comprises performing a one-bit stream conformance test that determines whether the bit stream is consistent with a video write standard. The method of claim 7, wherein performing the bitstream conformance test comprises determining the bitstream when more than one set of HRD parameters are applied to the same operating point for all sets of HRD parameters in a coded video sequence It is inconsistent with the video writing standard. The method of claim 7, wherein performing the bitstream conformance test comprises determining that the bitstream is inconsistent with the video coding standard when one or more sets of HRD parameters in the VPS are applied to the same operating point. The method of claim 7, wherein performing the bitstream conformance test comprises determining the bitstream and the VPS when the VPS includes applying to a set of HRD parameters having an operation point having only a layer identifier set of value 0 The video writing standard is inconsistent. A device comprising one or more processors configured to: from a set of hypothetical reference decoder (HRD) parameters in a video parameter set (VPS) and a sequence of parameter sets ( Selecting a set of HRD parameters applicable to a particular operating point of one of the meta-streams in a set of HRD parameters in the SPS); and at least partially based on the selected set of HRD parameters applicable to the particular operating point A subset of the bitstreams associated with the particular operating point performs an HRD operation. The device of claim 11, wherein the one or more processors are configured to have a layer identifier set at the particular operating point containing all of the layers present in the subset of bit streams The set of HRD parameters in the SPS is determined to be applicable to the particular operating point. The device of claim 11, wherein the set of HRD parameters applicable to the particular operating point includes parameters specifying: an initial write code image buffer (CPB) removal delay, a CPB size, a bit element Rate, an initial decoded image buffer (DPB) output delay, and a DPB size. The device of claim 11, wherein the one or more processors are configured to: determine that the particular operating point contains a set of target layer identifiers present in each of the layer identifiers in the subset of bitstreams, wherein The target layer identifier set of the specific operation point is a subset of the layer identifiers present in the bit stream, and determining that the specific operation point is equal to one of the maximum time identifiers present in the bit stream subset A target time identifier, wherein the target time identifier of the particular operating point is less than or equal to the maximum time identifier present in the bit stream. The device of claim 14, wherein the one or more processors are configured to select the set of HRD parameters in the SPS in response to determining that the target layer identifier set for the particular operating point contains only a value of zero. The device of claim 14, wherein the one or more processors are configured to select the set in response to determining that the set of layer identifiers specified in the SPS is the same as the target layer identifier set for the particular operating point This set of HRD parameters in the SPS. The device of claim 11, wherein the one or more processors are configured to perform a one-bit stream conformance test that determines whether the bit stream is consistent with a video write standard. The device of claim 17, wherein the one or more processors are configured to determine the bit when more than one set of HRD parameters are applied to the same operating point for all sets of HRD parameters in a coded video sequence The meta stream is inconsistent with the video writing standard. The device of claim 17, wherein the one or more processors are configured to determine that the bitstream is inconsistent with the video write code criteria when one or more sets of HRD parameters in the VPS are applied to the same operating point. The device of claim 17, wherein the one or more processors are configured to determine the bitstream when the VPS includes a set of HRD parameters applied to an operating point having a set of layer identifiers having only a value of 0 It is inconsistent with the video writing standard. A device comprising: for selecting from a set of hypothetical reference decoder (HRD) parameters in a video parameter set (VPS) and a set of HRD parameters in a sequence parameter set (SPS) for one bit a component of one of a set of HRD parameters of a particular operating point; and a bitstream associated with the particular operating point based at least in part on the selected set of HRD parameters applicable to the particular operating point A subset of components that perform an HRD operation. The device of claim 21, comprising: determining, for a set of HRD parameters in the SPS, when the set of layer identifiers of the particular operating point contains a set of all layer identifiers present in the subset of bitstreams The component at this particular operating point. The device of claim 21, further comprising: means for determining that the particular operating point contains a target layer identifier set of each layer identifier present in the subset of bitstreams, wherein the specific operating point The target layer identifier set is a subset of the layer identifiers present in the bit stream; a target time identification for determining that the specific operating point is equal to one of the maximum time identifiers present in the bit stream subset a component, wherein the target time identifier of the particular operating point is less than or equal to the maximum time identifier present in the bitstream; and a set responsive to determining a layer identifier assigned to the SPS Same as this The set of target layer identifiers for a particular operating point and the components of the set of HRD parameters in the SPS are selected. A computer readable data storage medium having stored thereon instructions for configuring the device when executed by one or more processors of a device: from a hypothetical reference decoder in a video parameter set (VPS) ( Selecting one of a set of HRD parameters in a set of HRD) parameters and a set of HRD parameters in a sequence of parameter sets (SPS) for selecting a set of HRD parameters for a particular operating point of a one-bit stream; and based at least in part on the particular operation The selected set of HRD parameters of the point performs an HRD operation on a subset of bitstreams associated with the particular operating point. The computer readable material storage medium of claim 24, wherein the instructions further configure the device to have a set of layer identifiers at a particular operating point containing a set of all layer identifiers present in the subset of bitstreams It is determined that the set of HRD parameters in the SPS is applicable to the particular operating point. The computer readable data storage medium of claim 24, wherein the instructions configure the device to: determine that the particular operating point contains a target layer identifier set of each layer identifier present in the subset of bitstreams, The target layer identifier set of the specific operation point is a subset of the layer identifiers present in the bit stream; determining that the specific operation point is equal to one of the maximum time identifiers present in the bit stream subset a target time identifier, wherein the target time identifier of the specific operating point is less than or equal to the maximum time identifier present in the bit stream; and responsive to determining one of the layer identifiers specified in the VPS The set of HRD parameters in the VPS is selected by collecting the set of target layer identifiers that are identical to the particular operating point.