WO2022222656A1 - 码流处理方法、装置、终端设备及存储介质 - Google Patents

码流处理方法、装置、终端设备及存储介质 Download PDF

Info

Publication number
WO2022222656A1
WO2022222656A1 PCT/CN2022/081593 CN2022081593W WO2022222656A1 WO 2022222656 A1 WO2022222656 A1 WO 2022222656A1 CN 2022081593 W CN2022081593 W CN 2022081593W WO 2022222656 A1 WO2022222656 A1 WO 2022222656A1
Authority
WO
WIPO (PCT)
Prior art keywords
code stream
layer
stream
encoding
encoded
Prior art date
Application number
PCT/CN2022/081593
Other languages
English (en)
French (fr)
Inventor
高莹
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to US18/555,912 priority Critical patent/US20240223794A1/en
Priority to EP22790765.6A priority patent/EP4329306A1/en
Publication of WO2022222656A1 publication Critical patent/WO2022222656A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present application relates to the field of communication technologies, for example, to a code stream processing method, apparatus, terminal device and storage medium.
  • the present application provides a code stream processing method, apparatus, terminal device and storage medium, which reduce decoding overhead.
  • the present application provides a code stream processing method, including:
  • the encoded code stream includes encoding indication information
  • the encoding indication information indicates a code stream type of the encoded code stream
  • obtain the encoding indication information decode the encoded code stream according to the encoding indication information Code stream after encoding.
  • the present application provides a code stream processing method, including:
  • the application provides a code stream processing device, including:
  • the first obtaining module is configured to obtain the encoded code stream, the encoded code stream includes encoding indication information, and the encoding indication information indicates the code stream type of the encoded code stream; the second obtaining module is configured to obtain the encoded code stream.
  • the encoding indication information; the decoding module is configured to decode the encoded code stream according to the encoding indication information.
  • a code stream processing device provided by this application includes:
  • a generation module is set to generate an image encoding code stream and the encoding instruction information corresponding to the image encoding code stream;
  • the adding module is configured to add the encoding indication information to the image encoding code stream to obtain the encoded code stream; the sending module is configured to send the encoded code stream.
  • This application provides a terminal device, including:
  • one or more processors a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the Any of the code stream processing methods provided in the application embodiments.
  • the present application provides a storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, any one of the code stream processing methods in the embodiments of the present application is implemented.
  • FIG. 1 is a schematic flowchart of a code stream processing method provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of another code stream processing method provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of another code stream processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a code stream processing apparatus provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a code stream processing apparatus provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
  • FIG. 1 is a schematic flowchart of a code stream processing method provided by an embodiment of the present application.
  • the method may be suitable for reducing decoding overhead, the method may be executed by a code stream processing apparatus, the apparatus may be implemented by software and/or hardware, and the apparatus may be integrated in a terminal device.
  • the related technology can create video content of different resolution levels and identify the video resources of each resolution level.
  • the user requests the resolution description information of the video resources of each resolution level before acquiring the video resources.
  • video resources with appropriate resolution levels are selected for selective reception or active acquisition of corresponding video resources.
  • the main role of video content is not only for real-time monitoring, but also for finding clues after the fact, so network bandwidth consumption and storage time need to be considered.
  • Using a higher resolution will consume a lot of network bandwidth, and will also lead to shorter storage time of video content and reduced video traceability under the condition that the storage space on the user side remains unchanged.
  • using a lower resolution can reduce network bandwidth consumption and user-side storage space overhead, it may also lose details in some key images, resulting in loss of important clues.
  • the video stream may include a base layer (Base Layer) sub-stream and at least one enhancement layer (Enhancement Layer) sub-stream, and the enhancement layer sub-stream may be an independent enhancement layer sub-stream or a dependent enhancement layer sub-stream flow.
  • the base layer substream can be decoded independently to provide basic quality video content
  • the independent enhancement layer substream can be independently decoded to provide higher quality video content.
  • the dependent enhancement layer substream needs to be decoded together with the base layer substream to provide higher quality video content. quality video content.
  • the decoding end must perform deep decoding on the received multi-layer encoded code stream (must parse at least the video parameter set in the code stream Video Parameter Set, VPS)) before it can parse out the necessary information and extract the user side
  • VPS Video Parameter Set
  • the code stream type can indicate the type to which the encoded code stream belongs.
  • the decoding overhead is reduced by including encoding indication information in the encoded code stream to indicate decoding.
  • the code stream type is not limited here.
  • the encoded codestream may include a multi-layer encoded codestream and a termination codestream.
  • the corresponding types of the multi-layer encoded code stream and the termination code stream can be indicated by the code stream type.
  • the termination code stream can be regarded as a code stream that terminates the continuous range indicated by the previous coding indication information.
  • the termination code stream can be a multi-layer coded code stream or a non-multi-layer coded code stream.
  • the code stream type may be indicated by at least one of the following in the encoding indication information: cancellation identification indication information, encoding indication information itself, and key frame identification information.
  • Whether the termination code stream is a multi-layer encoded code stream or a non-multi-layer code stream may be indicated by at least one of the following information: cancellation identification indication information, encoding indication information itself and key frame identification information.
  • the terminal device in this embodiment may be regarded as a device at the decoding end.
  • the encoded code stream at the encoding end can be obtained. It is also possible to obtain the encoded code stream obtained by encoding at the local end.
  • the encoding indication information is not limited.
  • the SEI message or the encoding indication information is included to indicate in the file encapsulation of the code stream.
  • the encoded code stream may be parsed to obtain the encoding indication information included in the encoded code stream. How to obtain the encoding indication information is not limited here, and may be determined based on the location of the encoding indication information in the encoded code stream.
  • this embodiment may perform decoding based on encoding indication information. Since the encoding indication information indicates the code stream type of the encoded code stream, the decoding can be performed in combination with the code stream type during decoding, which reduces the decoding overhead. How to decode is not limited here.
  • the code can be decoded in combination with the information indicating decoding in the encoding instruction information. postcode stream.
  • the information indicating decoding includes but is not limited to one of the following: enhancement type parameter, sub-stream parameter information, key frame identification information and persistent identification information.
  • a code stream processing method provided by the present application obtains an encoded code stream, the encoded code stream includes encoding indication information, and the encoding indication information indicates the code stream type of the encoded code stream; obtains the encoded code stream. encoding indication information; and decoding the encoded code stream according to the encoding indication information. This method effectively reduces the decoding overhead.
  • the encoding indication information includes a Supplemental Enhancement Information (SEI) message
  • SEI Supplemental Enhancement Information
  • the code stream corresponding to the code stream type includes a multi-layer encoded code stream and a termination code stream
  • the termination code stream is Terminates a codestream encoded as a multi-layer encoded codestream.
  • SEI Supplemental Enhancement Information
  • the SEI message may include the SEI message in the standard and the newly added SEI message.
  • the SEI message includes at least one of the following:
  • the extended user data SEI message The extended user data unregistered SEI message; the newly added SEI message.
  • the SEI message includes at least one of the following:
  • the sub-code stream refers to the base layer sub-code stream and the enhancement layer sub-code stream.
  • the enhanced type parameter and the enhanced layer type parameter are unified into the enhanced type parameter, and the enhanced type parameter is a parameter carried in the SEI message.
  • enhancement-layer sub-stream types in this application may be unified into enhancement-layer sub-stream types, that is, the types corresponding to independent enhancement-layer sub-streams and dependent enhancement-layer sub-streams.
  • the previous SEI message may be regarded as the previous SEI message sent by the encoding end corresponding to the current encoded code stream, and the message is included in the corresponding encoded code stream.
  • Cancelling the persistence restriction of the previous SEI message can be considered as canceling the encoding to the multi-layer encoding code stream at the beginning of the code stream corresponding to the cancellation identification indication information, or canceling the encoding to the multi-layer encoding code stream after canceling the code stream corresponding to the identification indication information.
  • the persistent identification information sets a persistent restriction
  • the sub-stream parameter information includes at least one of the following: an enhancement layer quantity parameter, a quality level, and a layer identifier of the sub-stream corresponding to the quality level, where the number of sub-streams corresponding to the quality level is one or wherein, the enhancement layer quantity parameter indicates the number of enhancement layer sub-streams included in the encoded code stream, and the quality level indicates a quality level that can be provided by the sub-streams in the encoded code stream.
  • the decoding the encoded stream according to the encoding indication information includes:
  • the layer identifier of the sub-stream to be extracted from the encoded stream may be determined based on the encoding indication information, and the determination is not limited here, such as determination based on the quality level.
  • the obtaining a sub-stream from the encoded stream according to the encoding indication information includes:
  • FIG. 2 is a schematic flowchart of another code stream processing method provided by an embodiment of the present application. Referring to FIG. 2, the method includes:
  • the sub-stream parameter information can be queried based on the quality level required by the user, and the layer identifier of the sub-stream to be extracted corresponding to the required quality level can be determined.
  • the required quality level can be regarded as the picture quality level required after decoding.
  • the present application further provides a code stream processing method
  • FIG. 3 is a schematic flowchart of another code stream processing method provided by an embodiment of the present application, and the method can be applied to the case of reducing decoding overhead , the method can be integrated in a code stream processing device, the device can be implemented by software and/or hardware, and the device can be integrated on a terminal device.
  • the details of this embodiment refer to the above-mentioned embodiment.
  • a code stream processing method provided by this embodiment of the present application includes:
  • the image encoding code stream in this embodiment may be a multi-layer encoding code stream, and may also be a non-multi-layer encoding code stream.
  • the image encoding code stream may be a termination code stream.
  • the multi-layer encoding code stream is generated for the image, including but not limited to the following three types: 1) The image contains an indication that multi-layer encoding needs to be performed, for example, indicating that the image is a key picture, and the image needs to be encoded. Multi-layer encoding; 2) The encoding end analyzes the image characteristics, and determines that multi-layer encoding needs to be performed according to the image characteristics; 3) The image is included in the persistent scope (persistent_scope) of the SEI message corresponding to the previous multi-layer encoded image.
  • the image When it is determined that a termination code stream is generated for the image, the image should be included in the persistent scope (persistent_scope) of the SEI message corresponding to the previous multi-layer encoded image and/or one of the following conditions should be satisfied at the same time: 1) Encoding The terminal analyzes the characteristics of the image, and determines according to the characteristics of the image that multi-layer coding is not required; 2) the image contains an instruction to terminate the multi-layer coding.
  • persistent_scope persistent scope
  • This embodiment does not limit the sequence of generating the coding indication information and generating the image coding code stream.
  • coding indication information is generated so as to reduce overhead during decoding.
  • the coding indication information includes an SEI message.
  • the location where the encoding indication information is added to the image encoding code stream is not limited, as long as the decoding end can obtain it.
  • the terminal device in this embodiment may be regarded as a device at the encoding end, and in this step, the encoded code stream may be sent to the device at the decoding end.
  • an image coding code stream and coding indication information corresponding to the image coding code stream are generated; the coding indication information is added to the image coding code stream to obtain an encoded code stream; sending In the encoded code stream, decoding overhead is reduced by adding encoding indication information to the code stream.
  • the encoding indication information includes at least one of the following: enhancement type parameter, sub-code stream parameter information, key frame identification information, cancellation Identification indication information and persistent identification information.
  • the encoding indication information includes at least one of the following enhancement type parameters, substream parameter information, key frame identification information, cancellation identification indication information, and persistent identification information may be: Included in the SEI message.
  • the encoding instruction The information includes cancellation identification indication information, and the image encoding code stream is a multi-layer encoding code stream or a termination code stream.
  • the cancellation identification indication information included in the coding indication information may be included in the SEI message.
  • the encoding indication information is used to generate a multi-layer encoded code stream, or the encoding indication information is obtained after the multi-layer encoded code stream is generated.
  • the present application is exemplarily described below, and the code stream processing method provided by the present application can be considered as a method for processing a multi-layer encoded video code stream.
  • the solution of the present application provides the basic encoding code stream for the regular picture, provides the multi-layer encoding code stream for the key picture, and provides the indication of the encoding code stream layer information corresponding to the key picture in the high-level semantics such as SEI information, so that the decoding end can be used without
  • the appropriate encoding code stream layer or combination of encoding code stream layers in the multi-layer encoding code stream can be selected for decoding and playback, so as to provide low-quality content for conventional monitoring images and high-quality key monitoring images. The effect of quality content.
  • the user side can select different multi-layer encoding code streams for different display terminals according to the instructions in the code stream.
  • the encoding code stream layer or the combination of the encoding code stream layers is used for decoding and playback.
  • the SEI message is used as an example for description in the subsequent scheme, but other encoding code stream layer information that can be used to indicate the corresponding key pictures is not excluded. other ways, including but not limited to indicating in the file encapsulation of the code stream.
  • the code stream processing method applied to the decoding end may include:
  • Step 1 Receive a code stream to be decoded, and obtain a cross-layer indication SEI message corresponding to the code stream to be decoded.
  • the decoding end receives the code stream to be decoded, that is, the code stream after encoding, and the code stream to be decoded includes a cross-layer indication SEI message (that is, cross_layer_indication SEI message) proposed in this application, and the cross-layer indication SEI message is used. Indicates that the code stream associated with it is a multi-layer encoded code stream.
  • the cross-layer indication SEI message may be regarded as a standard newly added SEI message.
  • the decoder Before obtaining the SEI message, the decoder first obtains the Network Abstract Layer unit (NAL unit) from the code stream to be decoded.
  • NAL unit is the most basic syntax structure in the code stream, and each NAL unit contains a header Information (nal_unti_header) and raw byte sequence payload (Raw Byte Sequence Payload, RBSP) information.
  • the NAL unit header information includes the NAL unit type information (nal_unti_type).
  • the cross-layer indication SEI message is not limited to be PREFIX_SEI_NUT or SUFFIX_SEI_NUT.
  • the RBSP data in the NAL unit is parsed according to the raw byte sequence payload (Raw Byte Sequence Payload, RBSP) syntax structure corresponding to the SEI message, and an SEI RBSP may include one or more SEI message (sei_message).
  • the syntax element payload_type included in the syntax structure indicates the type of the SEI message.
  • the parameter information to be transmitted by the SEI message can be obtained by parsing according to the sei_payload() syntax structure corresponding to the SEI message of this type.
  • the value of the payloadType of the cross-layer indication SEI message may be different from the defined SEI information in the current H.266/VVC version 1 (for example, the value may be 8).
  • the decoding end can obtain the cross-layer indication SEI message through the above analysis. Since the grammatical structure and parsing process before obtaining a specific type of SEI message is a standard process, the description will not be repeated in the subsequent description, and the description will directly start with the payload grammatical structure of the relevant SEI message.
  • Step 2 Determine the code stream to be decoded as the code stream to be extracted through the cross-layer indication SEI message, and determine the enhancement layer sub-code stream type contained in the code stream to be extracted.
  • the cross-layer indication SEI message appears in the code stream, it means that the received code stream to be decoded is a multi-layer encoded code stream, and the sub-stream extraction process needs to be performed on the code stream to be decoded. It can also be called the code stream to be extracted.
  • the video code stream When the video code stream is a multi-layer coded code stream, it may include a base layer (Base Layer) sub-code stream and at least one enhancement layer (Enhancement Layer) sub-code stream, and the enhancement layer sub-code stream may be an independent enhancement layer sub-code stream. codestream or dependent enhancement layer sub-codestream.
  • the base layer substream can be decoded independently to provide basic quality video content, and the independent enhancement layer substream can be independently decoded to provide higher quality video content.
  • the dependent enhancement layer substream needs to be decoded together with the base layer substream to provide higher quality video content. quality video content. Therefore, in order to better perform the sub-stream extraction process, it is also necessary to obtain the enhancement type (enhancement_type) parameter from the cross-layer indication SEI message.
  • the enhancement type (enhancement_type) parameter is used to indicate the enhancement layer sub-codestream type contained in the to-be-extracted codestream.
  • Table 1 is the payload syntax structure table indicating the SEI message across
  • cross_layer_indication(payloadSize) ⁇ describe cli_enhancement_type u(3) ⁇
  • the enhancement type parameter that is, cli_enhancement_type, is used to indicate the type of the enhancement layer sub-code stream contained in the to-be-extracted code stream, and Table 2 is a corresponding table of values and corresponding meanings of the types of the enhancement layer sub-code stream.
  • cli_enhancement_type When the value of cli_enhancement_type is equal to 0, it indicates that only the independent enhancement layer type is included in the enhancement layer sub-stream of the code stream to be extracted. When the value of cli_enhancement_type is equal to 1, it indicates that the enhancement layer sub-stream of the to-be-extracted code stream only contains the dependent enhancement layer type. When the value of cli_enhancement_type is equal to 2, it indicates that the enhancement layer sub-stream of the code stream to be extracted contains both types of dependent enhancement layers and independent enhancement layers.
  • Step 3 According to the enhancement layer sub-stream type contained in the code stream to be extracted, obtain the sub-stream parameter information corresponding to the type, and determine the sub-stream identifier to be extracted according to the sub-stream parameter information.
  • the sub-stream parameter information may include a parameter of the number of enhancement layers (num_enhancement_layers), that is, the number of enhancement layer sub-streams, a quality level (quality_level_idx) parameter, and a layer identifier of the sub-stream corresponding to each quality level.
  • the number of enhancement layer sub-streams (num_enhancement_layers) is used to indicate the number of enhancement layers contained in the to-be-extracted code stream, and the quality level (quality_level_idx) parameter is used to indicate the to-be-extracted code stream.
  • Table 3 is a schematic table of the relationship between a quality level and a corresponding set provided by this embodiment of the present application.
  • the quality level parameter that is, quality_level_idx
  • the quality_level_idx when the quality level parameter, that is, quality_level_idx, is equal to 0, it indicates that the sub-stream in the to-be-extracted code stream can only be Provides basic quality video.
  • the quality_level_idx is equal to 1
  • quality_level_idx is equal to 2
  • the sub-stream in the to-be-extracted code stream can simultaneously provide basic quality, medium quality and high quality video.
  • Table 3 A schematic table of the relationship between a quality level and a corresponding set provided by the embodiment of the present application
  • the user side After determining the set of quality levels that can be provided by the sub-streams in the code stream to be extracted, the user side selects an appropriate quality level from the set of quality levels according to the actual situation (input instructions, pre-configuration, device capabilities, etc.). , obtain a list of layer identifiers corresponding to the selected quality level, where the list of layer identifiers corresponding to the quality level is the layer identifier of the substream to be extracted.
  • step 2 there are three types of enhancement layer sub-streams contained in the to-be-extracted code stream.
  • the following describes the list of layer identifiers corresponding to each quality level in the quality level set and the determination of the three different types. The process of identifying the substream to be extracted.
  • Example 1 cli_enhancement_type is equal to 0, and the code stream to be extracted contains only the independent enhancement layer sub-code stream.
  • the base layer sub-stream and the enhancement layer sub-stream in the to-be-extracted code stream can be decoded independently, so there is only one code stream layer identifier in the code stream layer identifier list corresponding to the quality level, and Table 4 is as follows: This application provides a correspondence table between quality levels and layer identifiers.
  • the corresponding code stream identification list only contains layer 0, so layer 0 is the sub-stream layer identification to be extracted, and the code stream layer identification is the sub-stream of layer 0. It is the sub-stream to be extracted.
  • the corresponding stream ID list only contains layer 1, so layer 1 is the sub-stream layer ID to be extracted, and the sub-stream whose code stream layer ID is layer 1 is to be extracted substream.
  • the corresponding stream ID list only contains layer 2, so layer 0 is the sub-stream layer ID to be extracted, and the sub-stream whose code stream layer ID is layer 2 needs to be extracted substream.
  • Example 2 when cli_enhancement_type is equal to 1, the to-be-extracted code stream only contains the dependent enhancement layer sub-code stream.
  • the dependent enhancement layer sub-stream cannot be decoded alone, the dependent enhancement layer sub-stream must be combined with the base layer sub-stream to achieve decoding, so the code stream identification list corresponding to the quality level (except the basic quality) should be At least two code stream layer identifiers are included, and the basic quality corresponding code stream identifier list may only contain one code stream layer identifier.
  • Table 5 is yet another correspondence table between quality levels and layer identifiers provided by this application.
  • the corresponding code stream identification list only contains layer 0, so layer 0 is the sub-stream layer identification to be extracted, and the sub-stream whose code stream layer identification is layer 0 is Substream to be extracted.
  • the corresponding code stream identification list contains both layer 0 and layer 1, so layer 0 and layer 1 are the sub-stream layer identifications to be extracted, and the code stream layer identification is layer 0 and layer 1
  • the sub-stream is the sub-stream to be extracted.
  • the corresponding code stream identifier list contains both layer 0 and layer 2, so layer 0 and layer 2 are the sub-stream layer identifiers to be extracted, and the code stream layer identifiers are layer 0 and layer 2
  • the sub-stream is the sub-stream to be extracted.
  • the corresponding code stream identification list includes layer 0, layer 1 and layer 2 at the same time, so layer 0, layer 1 and layer 2 are the sub-stream layer identifications to be extracted, and the code The stream layer is identified as layer 0, and the sub-streams of layer 1 and layer 2 are the sub-streams to be extracted.
  • the basic quality corresponding code stream identifier list may also include more than one code stream layer identifier.
  • Table 6 is yet another correspondence table between quality levels and layer identifiers provided by this application.
  • the corresponding code stream identifier list contains both layer 0 and layer 1, so layer 0 and layer 1 are the sub-stream layer identifiers to be extracted, and the code stream layer identifier is the layer The sub-streams of 0 and layer 1 are the sub-streams to be extracted.
  • Example 3 cli_enhancement_type is equal to 2, the code stream to be extracted contains both the independent enhancement layer sub-stream and the dependent enhancement layer sub-stream.
  • the code stream identification list corresponding to the quality level may include one or more code stream layer identifications.
  • Table 7 is another quality level and layer identification correspondence table provided by the present application.
  • Tier 1 (medium quality picture)
  • Tier 2 (high quality picture)
  • Tier 1 (high quality picture)
  • Tier 1 (high quality picture)
  • Tier 2 (high quality picture)
  • the corresponding code stream identification list only contains layer 1, so layer 1 is the sub-stream layer identification to be extracted, and the sub-stream whose code stream layer identification is layer 1 is Substream to be extracted.
  • the corresponding stream identifier list can include layer 1 and layer 2, so layer 1 and layer 2 are the sub-stream layer identifiers to be extracted, and the stream layer identifiers are layer 1 and layer 2
  • the sub-stream is the sub-stream to be extracted.
  • the number of sub-code stream identifiers to be extracted contained in the code stream identifier list corresponding to the quality level should not be greater than the number indicated by the aforementioned enhancement layer sub-code stream quantity (num_enhancement_layers) parameter.
  • Step 4 According to the identifier of the sub-code stream to be extracted, extract the sub-code stream to be extracted from the to-be-extracted code stream.
  • Extracting a sub-stream from a complete code stream according to the sub-stream identifier is a related technology that has been defined in the standard, and can be implemented by those skilled in the art according to the content of the standard protocol, and the extraction process is not described in detail in this solution.
  • the parameter set (Parameter Set) is an important syntax structure in the video coding and decoding standard, it can contain the public information required in the coding and decoding process, such as the flag bit (flag), public parameter information (such as the maximum number of layers (max layers). )Wait).
  • flag bit flag bit
  • maximum number of layers maximum layers
  • the value of the vps_all_independent_layers_flag parameter in the video parameter set associated with the to-be-extracted code stream associated with the cross-layer indication SEI message should be 1;
  • the value of the cli_enhancement_type in the SEI message is not equal to 0, the value of the vps_all_independent_layers_flag parameter in the video parameter set associated with the to-be-extracted code stream associated with the SEI message should be 0.
  • the value of the maximum number of layers parameter vps_max_layers_minus1 in the video parameter set should be consistent with the value of the number of enhancement layer sub-streams num_enhancement_layers in the cross-layer indication SEI message.
  • Table 8 is a syntax structure table of a cross-layer indication SEI message provided by this application.
  • Table 8 Syntax structure table of a cross-layer indication SEI message provided by this application
  • the parameter of the number of enhancement layers namely cli_num_enhancement_layers: indicates the number of enhancement layer sub-streams in the to-be-decoded code stream associated with the SEI message.
  • Layer ID that is, cli_layer_id[i]: indicates the layer ID of the i-th substream.
  • the quality level parameter also known as the quality level, that is, cli_quality_level_idx: indicates the quality level (set) that can be provided by the sub-codestream in the to-be-extracted codestream, see the description in the first embodiment.
  • the layer identifier corresponding to the quality level indicates the layer identifier of the corresponding extractable substream when the quality level is i.
  • cli_extractable_layer_id[i] may also be expressed as cli_extractable_layer_id[i][0], indicating the layer identifier of the corresponding extractable substream when the quality level is i.
  • cli_extractable_layer_flag[i][j] indicates the extractable flag bit of the jth layer substream when the quality level is i. When its value is equal to 1, it indicates that the jth layer substream is the extractable substream when the quality level is i.
  • Code stream when its value is equal to 0, it indicates that the sub-stream of the j-th layer is not an extractable sub-stream when the quality level is i. All sub-stream layer flags corresponding to j satisfying cli_extractable_layer_flag[i][j] equal to 1 form a list of extractable sub-stream flags corresponding to the quality level i.
  • This embodiment only provides an example of a feasible cross-layer indication SEI message syntax structure, and all SEI messages conforming to all or part of the solution described in the first embodiment are within the protection scope of the solution of this application.
  • a key frame flag is added, and the key frame flag in the cross-layer indication SEI message indicates whether the code stream to be decoded needs to involve cross-layer operations.
  • This specific type of SEI message ie, The SEI message in the second embodiment
  • the cross-layer indication SEI message may further include a cancellation flag (cli_cancel_flag) and a persistence flag (cli_persistence_flag).
  • Table 9 is another syntax structure table of cross-layer indication SEI message provided by this application.
  • the cancellation flag indication information ie cli_cancel_flag: when the value is equal to 1, indicates that the cross-layer indication SEI message cancels the persistence of any previous cross-layer indication SEI message and does not use related SEI functions. On the contrary, when the value is equal to 0, it means that the following is cross-layer indication information such as the first multi-layer coding parameter information.
  • Persistence identification information ie cli_persistence_flag: used to indicate the persistence of the current cross-layer indication SEI message.
  • persistence flag bit 0
  • persistence flag bit cli_persistence_flag
  • the persistence scope (peisistence_scope) of the cross-layer indication SEI message may be an access unit (Access Unite, AU), a coded video sequence (Coded Video Sequence, CVS), or an unspecified (unspecified), this There are no restrictions on the application.
  • cli_key_frame_flag When the value is equal to 1, it indicates that the code stream to be decoded is a multi-layer encoded code stream, and the sub-stream extraction process needs to be performed on the code stream to be decoded according to the type of the enhancement layer. Therefore, the code stream to be decoded is also It can be expressed as the code stream to be extracted. On the contrary, when its value is equal to 0, it indicates that the code stream to be decoded is not a multi-layer encoded code stream, and corresponding operations are performed on the code stream to be decoded according to the original basic decoding process in the standard.
  • cli_extractable_layer_flag[i][j] indicates the extractable flag bit of the jth layer substream when the quality level is i. When its value is equal to 1, it indicates that the jth layer substream is the extractable substream when the quality level is i.
  • Code stream when its value is equal to 0, it indicates that the sub-stream of the j-th layer is not an extractable sub-stream when the quality level is i. All sub-stream layer flags corresponding to j satisfying cli_extractable_layer_flag[i][j] equal to 1 form a list of extractable sub-stream flags corresponding to the quality level i.
  • cli_extractable_layer_flag[i][j] is uniformly used to indicate the extractable flag bit of the jth layer substream when the quality level is i.
  • the enhancement layer sub-stream type in the to-be-decoded stream associated with the SEI message should be an independent enhanced sub-stream, so the corresponding extractable sub-stream under each quality level is only 1 Therefore, as long as there is one j such that the value of cli_extractable_layer_flag[i][j] is 1, the substream of the j layer is the corresponding extractable substream when the level is i, and there is no need to continue executing the do loop.
  • This embodiment provides another example of indicating the syntax structure and semantic information of the SEI message across layers.
  • the solution of the present application can be implemented through the syntax element user_data_payload_byte in the SEI message with unregistered user data.
  • the syntax element user_data_payload_byte can include the first embodiment
  • the mentioned capability may include all or part of the content in the SEI message exemplified in the second embodiment, so as to achieve the purpose of implementing the solution of the present application.
  • the user data SEI message registered with the existing ITU-T T.35 recommendation in the H.266/VVC version1 standard and the user data unregistered SEI message can be implemented.
  • any other extension of the solution of the present application using these two SEI messages is also supported.
  • This embodiment is a video encoding method, which is applied to a terminal device that encodes a video.
  • the input of the device is an image contained in the video, and the output is an image code stream or a transport stream or media file containing the image code stream.
  • Step S1 Read an image, the image is a complete image in the video sequence, or a sub-image in a complete image.
  • Step S2. Determine whether the image involves multi-layer coding, if yes, execute step S3, otherwise, execute step S5.
  • the methods for judging whether the image involves multi-layer coding include but are not limited to the following 3 methods: 1) The read image contains an indication that multi-layer coding needs to be performed, for example, indicating that the image is a key picture and needs to be encoded Multi-layer encoding; 2) The encoding end analyzes the image characteristics, and determines that multi-layer encoding needs to be performed according to the image characteristics; 3) The read image is included in the continuous range ( persistent_scope).
  • Multi-layer coding can be thought of as encoding an image into a multi-layer code stream.
  • Step S3. Obtain first multi-layer coding parameter information.
  • the first multi-layer coding parameter information includes an enhancement type parameter, an enhancement layer quantity parameter, a quality level (quality_level_idx) parameter, and a bitstream layer identifier corresponding to the quality level, and the like.
  • the ITU-T Rec. T.35 Registered User Data SEI message contains user data registered as specified in ITU-T Rec. T.35.
  • the solution of the present application may be implemented through the syntax element itu_t_t35_payload_byte in the SEI message.
  • the syntax element itu_t_t35_payload_byte may contain the capabilities mentioned in the first embodiment or all or part of the content in the SEI message exemplified in the second embodiment. , so as to achieve the purpose of the solution of the present application.
  • Step S4 Generate a multi-layer coded code stream for image coding, and perform step S8.
  • the encoding method can be implemented by the related art, which is not limited in this application, and the generated multi-layer encoding code stream should satisfy the first multi-layer encoding parameter information.
  • the present application does not limit the sequence between S3 and S4.
  • Step S5. Determine whether the image is used to end multi-layer coding, if so, execute step S6, otherwise execute step S7'.
  • the methods for judging whether the image is used to end the multi-layer encoding include but are not limited to the following 3 kinds: 1) the read image contains an instruction that needs to end the multi-layer encoding; 2) the encoding end analyzes the image characteristics, according to the image characteristics It is determined that the multi-layer encoding needs to be terminated; 3) the encoding end analyzes the image characteristics and determines that multi-layer encoding is not required, but the read image is still included in the continuous range of the cross-layer indication SEI message corresponding to the previous indication of the multi-layer encoded image. (persistent_scope).
  • Step S6 Obtain second multi-layer coding parameter information.
  • the second multi-layer coding parameter information includes at least a cancellation flag bit (cancel_flag), that is, cancellation flag indication information.
  • Step S7/S7' Generate a code stream for image coding.
  • the encoding method can be implemented by related technologies, which is not limited in this application.
  • the present application does not limit the sequence between S6 and S7.
  • Step S8 Write the SEI message carrying the multi-layer coding parameter information into the code stream.
  • the SEI message carrying the first multi-layer encoding parameter information or the second multi-layer encoding parameter information may be any one of the SEI message examples described in Embodiments 2 to 4.
  • the multi-layer encoding parameter information includes first multi-layer encoding parameter information and second multi-layer encoding parameter information.
  • the first multi-layer encoding parameter information is added to the multi-layer encoding code stream generated based on step S4, and the second multi-layer encoding parameter information is added to the code stream generated based on step S7.
  • the encoding end may also check the first multi-layer encoding Whether the parameter information is exactly the same as the third multi-layer encoding parameter information in the previous SEI message carrying the multi-layer encoding parameter information, if the first multi-layer encoding parameter information is exactly the same as the third multi-layer encoding parameter information and carries the third
  • the persistence scope (peisistence_scope) of the SEI message of the layer coding parameter information includes the current image, you may choose not to write the SEI message carrying the first multi-layer coding parameter information into the code stream.
  • Step S9 Output an image code stream or a transport stream or media file containing the image code stream.
  • FIG. 4 is a schematic structural diagram of a code stream processing apparatus provided by an embodiment of the present application.
  • the apparatus may be integrated in a terminal device. As shown in FIG. 4 , the apparatus includes:
  • the first obtaining module 31 is configured to obtain the encoded code stream, the encoded code stream includes encoding indication information, and the encoding indication information indicates the code stream type of the encoded code stream; the second obtaining module 32 is configured as Obtain the encoding indication information; the decoding module 33 is configured to decode the encoded stream according to the encoding indication information.
  • the code stream processing apparatus provided in this embodiment is used to implement the code stream processing method in the embodiment shown in FIG. 1 .
  • the implementation principle and technical effect of the code stream processing apparatus provided in this embodiment are the same as the code stream processing in the embodiment shown in FIG. 1 .
  • the method is similar and will not be repeated here.
  • the encoding indication information includes an SEI message
  • the code stream corresponding to the code stream type includes a multi-layer encoded code stream and a termination code stream
  • the termination code stream is a multi-layer encoded code stream whose termination code is terminated. code stream.
  • the SEI message includes at least one of the following:
  • the extended user data SEI message The extended user data unregistered SEI message; the newly added SEI message.
  • the SEI message includes at least one of the following:
  • Enhancement type parameter indicates the enhancement type of the enhancement layer substream contained in the encoded codestream
  • the enhancement type parameter indicates the enhancement type of the enhancement layer substream contained in the encoded codestream
  • the sub-code stream parameter information is used to determine the sub-code stream that needs to be extracted from the encoded code stream
  • the key frame identification information indicates whether the encoded code stream is a multi-layer encoded code stream
  • the cancellation identifier indicates whether the corresponding SEI message cancels the persistence restriction of the previous SEI message
  • the persistence identification information indicates whether the persistence restriction is set.
  • the sub-stream parameter information includes at least one of the following: an enhancement layer quantity parameter, a quality level, and a layer identifier of the sub-stream corresponding to the quality level, where the number of sub-streams corresponding to the quality level is one or wherein, the enhancement layer quantity parameter indicates the number of enhancement layer sub-streams included in the encoded code stream, and the quality level indicates a quality level that can be provided by the sub-streams in the encoded code stream.
  • the decoding module 33 is set to:
  • the decoding module 33 obtains a sub-stream from the encoded stream according to the encoding indication information, including:
  • FIG. 5 is a schematic structural diagram of a code stream processing apparatus provided by an embodiment of the present application, and the apparatus is integrated in a terminal device. As shown in Figure 5, the device includes:
  • the generating module 41 is configured to generate an image encoding code stream and the encoding instruction information corresponding to the image encoding code stream; the adding module 42 is configured to add the encoding instruction information to the image encoding code stream to obtain the post-encoding code stream; the sending module 43 is configured to send the encoded stream.
  • the code stream processing apparatus provided in this embodiment is used to implement the code stream processing method in the embodiment shown in FIG. 3 .
  • the implementation principle and technical effect of the code stream processing apparatus provided in this embodiment are the same as the code stream processing in the embodiment shown in FIG. 3 .
  • the method is similar and will not be repeated here.
  • the encoding indication information includes at least one of the following: enhancement type parameter, sub-code stream parameter information, key frame identification information, cancellation Identification indication information and persistent identification information.
  • the encoding instruction The information includes cancellation identification indication information, and the image encoding code stream is a multi-layer encoding code stream or a termination code stream.
  • the encoding indication information is used to generate a multi-layer encoded code stream, or the encoding indication information is obtained after the multi-layer encoded code stream is generated.
  • FIG. 6 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device provided by this application includes one or more processors 51 and a storage device 52 ; the number of processors 51 in the terminal device may be one or more, and one processor 51 in FIG. 6 is used as the
  • the storage device 52 is used to store one or more programs; the one or more programs are executed by the one or more processors 51 , so that the one or more processors 51 implement the implementation as in the embodiments of the present application The code stream processing method.
  • the terminal device further includes: a communication device 53 , an input device 54 and an output device 55 .
  • the processor 51 , the storage device 52 , the communication device 53 , the input device 54 and the output device 55 in the terminal device may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 6 .
  • the input device 54 can be used to receive input numerical or character information, and generate key signal input related to user setting and function control of the terminal device.
  • the output device 55 may include a display device such as a display screen.
  • the communication device 53 may include a receiver and a transmitter.
  • the communication device 53 is configured to transmit and receive information according to the control of the processor 51 .
  • Information includes but is not limited to encoded stream
  • the storage device 52 can be configured to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the code stream processing method described in the embodiments of the present application (for example, a code stream processing device).
  • the first acquisition module 31 , the second acquisition module 32 and the decoding module 33 in the code stream processing device are also for example the acquisition module 41 , the adding module 42 and the sending module 43 in the code stream processing device).
  • the storage device 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like.
  • storage device 52 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.
  • storage device 52 may include memory located remotely from processor 51, and these remote memories may be connected to the terminal device through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • Embodiments of the present application further provide a storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, any one of the methods described in the present application is implemented, and the storage medium stores a computer program, and the computer When the program is executed by the processor, any one of the code stream processing methods described in the embodiments of the present application is implemented.
  • Stream processing methods include:
  • the encoded code stream includes encoding indication information
  • the encoding indication information indicates a code stream type of the encoded code stream
  • obtain the encoding indication information decode the encoded code stream according to the encoding indication information Code stream after encoding.
  • the code stream processing method including:
  • the computer storage medium of the embodiments of the present application may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above.
  • Examples (non-exhaustive list) of computer readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (Read Only Memory) Memory, ROM), Erasable Programmable Read Only Memory (EPROM), Flash Memory, Optical Fiber, Portable Compact Disc Read-Only Memory (CD-ROM), Optical Storage Devices, Magnetic memory device, or any suitable combination of the above.
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • suitable medium including but not limited to: wireless, wire, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out the operations of the present application may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional Procedural programming language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to an external computer (eg, use an internet service provider to connect via the internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • terminal equipment encompasses any suitable type of wireless user equipment, such as mobile telephones, portable data processing devices, portable web browsers or vehicle mounted mobile stations.
  • the various embodiments of the present application may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof.
  • some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor or other computing device, although the application is not limited thereto.
  • Embodiments of the present application may be implemented by the execution of computer program instructions by a data processor of a mobile device, eg in a processor entity, or by hardware, or by a combination of software and hardware.
  • Computer program instructions may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or written in any combination of one or more programming languages source or object code.
  • ISA Instruction Set Architecture
  • the block diagrams of any logic flow in the figures of the present application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions.
  • Computer programs can be stored on memory.
  • the memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, Read-Only Memory (ROM), Random Access Memory (RAM), optical Memory devices and systems (Digital Video Disc (DVD) or Compact Disk (CD)), etc.
  • Computer-readable media may include non-transitory storage media.
  • the data processor may be of any type suitable for the local technical environment, such as, but not limited to, a general purpose computer, a special purpose computer, a microprocessor, a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC) ), programmable logic devices (Field-Programmable Gate Array, FPGA) and processors based on multi-core processor architecture.
  • a general purpose computer such as, but not limited to, a general purpose computer, a special purpose computer, a microprocessor, a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC) ), programmable logic devices (Field-Programmable Gate Array, FPGA) and processors based on multi-core processor architecture.
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本文公开了一种码流处理方法、装置、终端设备及存储介质。该码流处理方法包括：获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；获取所述编码指示信息；根据所述编码指示信息解码所述编码后码流。

Description

码流处理方法、装置、终端设备及存储介质 技术领域
本申请涉及通信技术领域，例如涉及一种码流处理方法、装置、终端设备及存储介质。
背景技术
随着数字媒体技术的快速发展，各类监控摄像机部署的越来越密集，市面上所使用的大多数监控摄像机都已具备基础的智能分析功能，能够在捕捉监控视频画面的同时对画面内容进行智能分析，判断出当前画面是否存在异常事情或者其他设定的需要关注的事件发生。判定有异常事情或其他设定的需要关注的事情发生的画面可以认为是关键画面。
如何降低解码开销是亟待解决的技术问题。
发明内容
本申请提供一种码流处理方法、装置、终端设备及存储介质，降低了解码开销。
本申请提供了一种码流处理方法，包括：
获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；获取所述编码指示信息；根据所述编码指示信息解码所述编码后码流。
本申请提供了一种码流处理方法，包括：
生成图像编码码流和所述图像编码码流对应的编码指示信息；将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；发送所述编码后码流。
本申请提供了一种码流处理装置，包括：
第一获取模块，设置为获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；第二获取模块，设置为获取所述编码指示信息；解码模块，设置为根据所述编码指示信息解码所述编码后码流。
本申请提供的一种码流处理装置，包括：
生成模块，设置为生成图像编码码流和所述图像编码码流对应的编码指示 信息；
添加模块，设置为将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；发送模块，设置为发送所述编码后码流。
本申请提供了一种终端设备，包括：
一个或多个处理器；存储装置，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本申请实施例中提供的任一种码流处理方法。
本申请提供了一种存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现本申请实施例中的任一种码流处理方法。
附图说明
图1为本申请实施例提供的一种码流处理方法的流程示意图；
图2为本申请实施例提供的又一种码流处理方法的流程示意图；
图3为本申请实施例提供的又一种码流处理方法的流程示意图；
图4为本申请实施例提供的一种码流处理装置的结构示意图；
图5为本申请实施例提供的一种码流处理装置的结构示意图；
图6为本申请实施例提供的一种终端设备的结构示意图。
具体实施方式
下文中将结合附图对本申请的实施例进行说明。
在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且，虽然在流程图中示出了逻辑顺序，但是在一些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
在一个示例性实施方式中，图1为本申请实施例提供的一种码流处理方法的流程示意图。该方法可以适用于降低解码开销的情况，该方法可以由码流处理装置执行，该装置可以由软件和/或硬件实现，该装置可以集成在终端设备中。
随着数字媒体技术的快速发展，城市安防中的各类监控摄像机部署的越来越密集，使得监控视频业务呈现爆炸式的增长，同时由于视频采集能力的提升，图像分辨率不断提高，需要传输的视频数据增加，在给网络造成沉重负担的同时，对客户端的存储能力也带来很大的挑战。
市面上所使用的大多数监控摄像机都已具备基础的智能分析功能，能够在 捕捉监控视频画面的同时对画面内容进行智能分析，判断出当前画面是否存在异常事情或者其他设定的需要关注的事件发生。这类画面往往才是监控视频中需要重点关注的关键画面，因此如何在带宽和存储能力不变的情况下，尽可能地提高关键画面质量，保留关键画面细节的同时又能保证视频能够满足需要的存储时长就是本申请方案需要重点解决的问题。
针对普通视频，相关技术可以制作不同分辨率等级的视频内容并对各分辨率等级的视频资源进行标识，用户在获取视频资源前先请求对各分辨率等级的视频资源的分辨率描述信息，根据不同网络带宽状况及终端呈现能力，选择合适的分辨率等级的视频资源，进行选择性接收或主动获取相应的视频资源。但是对于监控视频来说，视频内容的主要作用不仅是用于实时监控，还包括用于事后查找线索等用途，所以网络带宽消耗和存储时长都是需要考虑的。采用较高分辨率会对网络带宽消耗比较大，还会导致在用户侧存储空间不变的情况下，视频内容的可保存时间变短，视频可追溯性降低。而采用较低的分辨率虽然可以降低网络带宽消耗及用户侧存储空间开销，但是也可能因此丢失一些关键画面中的细节信息，造成重要线索丢失。
而随着360度全景视频和超高分辨率视频业务的广泛应用，一种区别于普通视频的全新视频内容类型，例如全景视频中包含主视场的非均匀全景视频内容类型、超高分辨率视频中不同质量分块视频组成的超高分辨率视频内容类型，可以实现仅对用户观看的主视场画面采用较高的分辨率，而对其他视角画面可以采用较低分辨率，从而实现降低网络开销的目的。但是上述方案不仅对编码端的能力要求比较高，价格也比较昂贵，对于数量庞大的视频监控场景来说，编码器的编码能力往往做不到同一画面中进行不同质量分块编码。由国际标准化组织(International Organization for Standardization，ISO)/国际电工委员会(International Electrotechnical Commission，IEC)和国际电信联盟(International Telecommunication Union，ITU)下属的联合视频专家组(Joint Video Expert Teams，JVET)制定的高性能视频编码(H.265/High Efficiency Video Coding，HEVC)标准和通用视频编码(H.266/Versatile Video Coding，VVC)标准都已经支持“多层视频编码/可伸缩视频编码”的概念。
视频码流中可以包括一个基本层(Base Layer)子码流和至少一个增强层(Enhancement Layer)子码流，所述增强层子码流可以是独立增强层子码流或依赖增强层子码流。基本层子码流可以独立解码提供基础质量的视频内容，独立增强层子码流可以独立解码提供较高质量的视频内容，依赖增强层子码流需要与基本层子码流一起解码提供较高质量的视频内容。将上述多层视频编码思想应用到视频监控场景中，将常规视频监控画面编码成基本层子码流，提供基础质量的视频内容，当检测出关键画面(异常发生、人员入侵、特定事件等)时利用增 强层子码流提供较高质量的视频内容，以提供更多的事件细节)。但是相关技术中，解码端必须要对收到的多层编码码流进行深度解码(必须至少解析码流中的视频参数集Video Parameter Set，VPS))以后才能解析出必要的信息，提取用户侧所需要的相关层的码流，浪费不必要的解码开销。
为解决上述技术问题，如图1所示，本申请提供的码流处理方法，包括：
S110.获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型。
码流类型可以指示编码后码流所属的类型。本实施例通过在编码后码流中包括编码指示信息以指示解码，从而降低了解码开销。此处不对码流类型进行限定。
在一个示例中，编码后码流可以包括多层编码码流和终止码流。多层编码码流和终止码流对应的类型可以由码流类型指示。终止码流可以认为是一种终止前一个编码指示信息所指示持续范围的码流。终止码流可以为多层编码码流，也可以为非多层编码码流。
本实施例可以通过编码指示信息中的如下至少一个指示码流类型：取消标识指示信息、编码指示信息本身和关键帧标识信息。
终止码流为多层编码码流或为非多层码流可以通过如下至少一个信息指示：取消标识指示信息、编码指示信息本身和关键帧标识信息。
本实施例中的终端设备可以认为是解码端的设备。本步骤可以获取编码端的编码后码流。也可以获取本端编码得到的编码后码流。
在本实施例中编码指示信息不作限定，如包括SEI消息或编码指示信息在码流的文件封装中进行指示。
S120.获取所述编码指示信息。
在获取编码后码流后，本步骤可以解析编码后码流获取编码后码流中所包括的编码指示信息。此处不对如何获取编码指示信息进行限定，可以基于编码指示信息在编码后码流中的位置确定。
S130.根据所述编码指示信息解码所述编码后码流。
在解码编码后码流的情况下，本实施例可以基于编码指示信息进行解码。由于编码指示信息指示了编码后码流的码流类型，故在进行解码时，能够结合码流类型进行解码，降低了解码开销。此处不对如何解码进行限定。
在编码后码流为多层编码码流的情况下或在编码后码流为终止码流，终止码流为多层编码码流的情况下，可以结合编码指示信息中指示解码的信息解码 编码后码流。其中，指示解码的信息包括但不限于如下之一：增强类型参数、子码流参数信息、关键帧标识信息和持久性标识信息。
本申请提供的一种码流处理方法，该方法获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；获取所述编码指示信息；根据所述编码指示信息解码所述编码后码流。该方法有效地降低了解码开销。
在上述实施例的基础上，提出了上述实施例的变型实施例，为了使描述简要，在变型实施例中仅描述与上述实施例的不同之处。
在一个实施例中，所述编码指示信息包括补充增强信息(Supplemental Enhancement Information，SEI)消息，所述码流类型对应的码流包括多层编码码流和终止码流，所述终止码流为终止编码为多层编码码流的码流。
本实施例可以SEI消息可以包括标准中的SEI消息和新增的SEI消息。
在一个实施例中，所述SEI消息包括如下至少一项：
扩展后的用户数据SEI消息；扩展后的用户数据未注册的SEI消息；新增的SEI消息。
在一个实施例中，所述SEI消息，包括如下至少一项：
增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息；所述增强类型参数指示所述编码后码流中包含的增强层子码流的增强类型，所述子码流参数信息用于确定所需从所述编码后码流中抽取的子码流，所述关键帧标识信息指示所述编码后码流是否为多层编码码流，所述取消标识指示信息指示所对应SEI消息是否取消了前一SEI消息的持久性限制，所述持久性标识信息指示是否设置持久性限制。
本申请中子码流是指基本层子码流和增强层子码流。增强类型参数和增强层类型参数统一为增强类型参数，增强类型参数是SEI消息中携带的一个参数。
本申请中的子码流类型和增强层子码流类型，可以统一为增强层子码流类型，也即独立增强层子码流和依赖增强层子码流所对应的类型。
前一SEI消息可以认为是与当前编码后码流对应的编码端发送的前一个SEI消息，该消息包含在对应的编码后码流。
取消前一SEI消息的持久性限制可以认为是在该取消标识指示信息对应的码流开始取消编码至多层编码码流，或从取消标识指示信息对应的码流后取消编码至多层编码码流。
在持久性标识信息设置持久性限制的情况下，可以限制对应的编码后码流 后设定个数的编码码流均为多层编码码流。设定个数可以用于确定持久性的持续范围。
在一个实施例中，子码流参数信息包括如下至少一项：增强层数量参数、质量等级、质量等级所对应子码流的层标识，所述质量等级所对应子码流的数量为一个或多个；其中，所述增强层数量参数指示所述编码后码流中包括的增强层子码流的数量，所述质量等级指示所述编码后码流中子码流可以提供的质量等级。
在一个实施例中，所述根据所述编码指示信息解码所述编码后码流，包括：
根据所述编码指示信息从所述编码后码流中获取子码流；解码所述子码流。
本实施例中可以基于编码指示信息确定编码后码流中所需提取的子码流的层标识，此处不对如何确定进行限定，如基于质量等级确定。
在一个实施例中，所述根据所述编码指示信息从所述编码后码流中获取子码流，包括：
获取所述编码指示信息所包括的增强类型参数；获取对应所述增强类型参数的子码流参数信息；根据所述子码流参数信息和所需质量等级，确定需提取的子码流的层标识；根据所述子码流的层标识，从所述编码后码流中提取子码流。
图2为本申请实施例提供的又一种码流处理方法的流程示意图，参见图2，该方法包括：
S1.获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型。
S2.获取所述编码指示信息。
S3.获取所述编码指示信息所包括的增强类型参数。
S4.获取对应所述增强类型参数的子码流参数信息。
S5.根据所述子码流参数信息和所需质量等级，确定需提取的子码流的层标识。
S6.根据所述子码流的层标识，从所述编码后码流中提取子码流。
S7.解码所述子码流。
本实施例中，增强类型参数和子码流参数信息存在对应关系，质量等级和层标识存在对应关系。本实施例可以基于用户所需质量等级查询子码流参数信息，确定对应所需质量等级的需提取的子码流的层标识。其中，所需质量等级 可以认为是解码后需要的画面质量等级。
在一个示例性实施方式中，本申请还提供了一种码流处理方法，图3为本申请实施例提供的又一种码流处理方法的流程示意图，该方法可以适用于降低解码开销的情况，该方法可以集成在码流处理装置中，该装置可以由软件和/或硬件实现，该装置集成在终端设备上。本实施例尚未详尽之处参见上述实施例。
如图3所示，本申请该实施例提供的一种码流处理方法，包括：
S210.生成图像编码码流和所述图像编码码流对应的编码指示信息。
本实施例中的图像编码码流可以为多层编码码流，也可以非多层编码码流。在图像编码码流为非多层编码码流的情况下，图像编码码流可以为终止码流。
确定对图像生成多层编码码流的情况，包括但不限于以下3种：1)所述图像中包含需要进行多层编码的指示，例如指示所述图像为关键画面，需要对所述图像进行多层编码；2)编码端分析图像特性，根据图像特性确定需要进行多层编码；3)所述图像包含在前一指示多层编码图像对应的SEI消息的持续范围(persistent_scope)内。
确定对所述图像生成终止码流的情况，应满足所述图像包含在前一指示多层编码图像对应的SEI消息的持续范围(persistent_scope)内和/或同时满足以下条件之一：1)编码端分析图像特性，根据图像特性确定不需要进行多层编码；2)所述图像中包含终止多层编码的指示。
本实施例不限定生成编码指示信息和生成图像编码码流的先后顺序。本步骤生成编码指示信息，以用于在解码时能够降低开销。
本实施例中，编码指示信息包括SEI消息。
S220.将所述编码指示信息添加至所述图像编码码流中，得到编码后码流。
编码指示信息添加至图像编码码流中的位置不作限定，只要解码端能够获取即可。
S230.发送所述编码后码流。
本实施例中的终端设备可以认为是编码端设备，本步骤可以将编码后码流发送至解码端的设备。
本申请提供的码流处理方法，生成图像编码码流和所述图像编码码流对应的编码指示信息；将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；发送所述编码后码流，通过将编码指示信息添加至码流中降低了解码开销。
在上述实施例的基础上，提出了上述实施例的变型实施例，为了使描述简 要，在变型实施例中仅描述与上述实施例的不同之处。
在一个实施例中，在所述图像编码码流为多层编码码流的情况下，所述编码指示信息包括如下至少一项：增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息。
在所述编码指示信息包括SEI消息的情况下，所述编码指示信息所包括的如下至少一项增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息可以包含在SEI消息中。
在一个实施例中，在所述图像编码码流在前一个编码指示信息的持续范围内，且所述图像编码码流对应的图像结束编码为多层编码码流的情况下，所述编码指示信息包括取消标识指示信息，所述图像编码码流为多层编码码流或终止码流。
在所述编码指示信息包括SEI消息的情况下，所述编码指示信息包括的取消标识指示信息可以包含在SEI消息中。
在一个实施例中，所述编码指示信息用于生成多层编码码流，或所述编码指示信息在生成多层编码码流后获取。
以下对本申请进行示例性描述，本申请提供的码流处理方法可以认为是一种多层编码视频码流的处理方法。本申请方案通过对常规画面提供基础编码码流，而对关键画面提供多层编码码流并在高层语义如SEI信息中提供关键画面对应的编码码流层信息的指示，从而实现解码端在不用对码流进行深层解码的情况下就可以选择多层编码码流中合适的编码码流层或编码码流层的组合进行解码播放，实现对常规监控画面提供低质量内容，关键监控画面提供高质量内容的效果。针对在本地存在多个能力不同的显示终端时，用户侧可以在收到一路多层编码监控码流后，根据码流中的指示，为不同的显示终端分别选择多层编码码流中不同的编码码流层或编码码流层的组合进行解码播放。
本申请为了实现在高层语义中提供关键画面对应的编码码流层信息的指示，后续方案中以SEI消息为例进行说明，但是并不排除其他可以用来指示关键画面对应的编码码流层信息的其他方式，包括但不限于在码流的文件封装中进行指示。
实施例一
应用于解码端的码流处理方法可以包括：
步骤1.接收待解码码流，获取所述待解码码流对应的跨层指示SEI消息。
解码端接收到待解码码流，即编码后码流，所述待解码码流中包含了本申 请中提出的一种跨层指示SEI消息(即cross_layer_indication SEI message)，该跨层指示SEI消息用于指示和它关联的码流为一个多层编码码流。该跨层指示SEI消息可以认为是标准新增的SEI消息。
为了能够更好地说明SEI消息解析过程，以下先补充H.266/VVC version1标准中与SEI消息封装相关的内容，以便更好地理解本申请方案内容。解码端在获取SEI消息前，首先会从待解码码流中获得网络抽象层单元(Network Abstract Layer unit，NAL unit)，NAL单元是码流中最基础的语法结构，每个NAL单元包含一个头信息(nal_unti_header)和原始字节序列负荷(Raw Byte Sequence Payload，RBSP)信息。NAL单元头信息中包含NAL单元类型信息(nal_unti_type)，当nal_unit_type取值为23时，指示该NAL单元为PREFIX_SEI_NUT，即前置的SEI信息，当nal_unit_type取值为24时，指示该NAL单元为SUFFIX_SEI_NUT，即后置的SEI信息。在本申请方案中，并不限定跨层指示SEI消息为PREFIX_SEI_NUT或SUFFIX_SEI_NUT。当确定一NAL单元类型为SEI消息时，按照SEI消息对应的原始字节序列载荷(Raw Byte Sequence Payload，RBSP)语法结构解析NAL单元中的RBSP数据，一个SEI RBSP中可以包括1个或者多个SEI消息(sei_message)。语法结构中所包括的语法元素payload_type，表示SEI消息的类型。
确定SEI消息的类型后，可以根据该类型SEI消息对应的sei_payload()语法结构，解析得到该SEI消息所要传递的参数信息。跨层指示SEI消息的payloadType取值与当前H.266/VVC version 1中的已经定义的SEI信息不同即可(例如，可取值为8)。解码端通过上述解析可以获取到跨层指示SEI消息。由于获取到一特定类型SEI消息前的语法结构及解析过程是标准流程，后续说明中将不再进行赘叙，直接从相关SEI消息的payload语法结构开始描述。
步骤2.通过所述跨层指示SEI消息确定所述待解码码流为待抽取码流并确定待抽取码流中包含的增强层子码流类型。
当码流中出现所述跨层指示SEI消息时，就表示接收到的待解码码流为多层编码码流，需要对待解码码流进行子码流抽取过程，因此所述待解码码流此时也可以称为待抽取码流。
当视频码流为多层编码码流时，可以包括一个基本层(Base Layer)子码流和至少一个增强层(Enhancement Layer)子码流，所述增强层子码流可以是独立增强层子码流或依赖增强层子码流。基本层子码流可以独立解码提供基础质量的视频内容，独立增强层子码流可以独立解码提供较高质量的视频内容，依赖增强层子码流需要与基本层子码流一起解码提供较高质量的视频内容。因此，为了更好地进行子码流抽取过程，还需要从所述跨层指示SEI消息中获取增强类型 (enhancement_type)参数。所述增强类型(enhancement_type)参数，用于指示待抽取码流中包含的增强层子码流类型。表1为跨层指示SEI消息的payload语法结构表。
表1跨层指示SEI消息的payload语法结构表
cross_layer_indication(payloadSize){ 描述
cli_enhancement_type u(3)
}  
其中，增强类型参数，即cli_enhancement_type用于指示待抽取码流中所包含的增强层子码流的类型，表2为增强层子码流的类型的取值和对应含义的对应表。
表2增强层子码流的类型的取值和对应含义的对应表
Figure PCTCN2022081593-appb-000001
当cli_enhancement_type取值等于0时，指示待抽取码流的增强层子码流中只包含独立增强层类型。当cli_enhancement_type取值等于1时，指示待抽取码流的增强层子码流中只包含依赖增强层类型。当cli_enhancement_type取值等于2时，指示待抽取码流的增强层子码流中同时包含依赖增强层和独立增强层两种类型。
步骤3.根据待抽取码流中包含的增强层子码流类型，获取该类型对应的子码流参数信息，并根据所述子码流参数信息确定需抽取的子码流标识。
所述子码流参数信息可以包括增强层数量(num_enhancement_layers)参数，即增强层子码流的数量，质量等级(quality_level_idx)参数以及各质量等级所对应子码流的层标识等。
所述增强层子码流的数量(num_enhancement_layers)，用于指示所述待抽取 码流中包含的增强层的数量，所述质量等级(quality_level_idx)参数，用于指示所述待抽取码流中的子码流可以提供的质量等级集合。
表3为本申请实施例提供的一种质量等级与对应集合关系的示意表，参见表3，当质量等级参数，即quality_level_idx等于0时，指示所述待抽取码流中的子码流只可以提供基础质量的视频。当quality_level_idx等于1时，指示所述待抽取码流中的子码流可以同时提供基础质量和中等质量的视频。当quality_level_idx等于2时，指示所述待抽取码流中的子码流可以同时提供基础质量、中等质量和高质量的视频。
表3本申请实施例提供的一种质量等级与对应集合关系的示意表
Figure PCTCN2022081593-appb-000002
确定待抽取码流中的子码流可以提供的质量等级集合后，用户侧根据实际情况(输入的指令，预先的配置，设备的能力等等)从所述质量等级集合中选择合适的质量等级，获取所述选择的质量等级对应的层标识列表，所述质量等级对应的层标识列表为需抽取的子码流的层标识。
如步骤2中所述，待抽取码流中包含的增强层子码流类型有3种，以下针对这三种不同类型分别进行说明所述质量等级集合中各质量等级对应的层标识列表以及确定需抽取的子码流标识的过程。
示例1：cli_enhancement_type等于0，待抽取码流中只包含独立增强层子码流。
此时，待抽取码流中的基础层子码流和增强层子码流都可以独立解码，所以所述质量等级对应的码流层标识列表中有且只有一个码流层标识，表4为本申请提供的一种质量等级与层标识的对应表。
表4本申请提供的一种质量等级与层标识的对应表
质量等级 与质量等级相对应的层标识列表
0(基本质量画面) 层0
1(中等质量画面) 层1
2(高质量画面) 层2
参见表4，当客户端选择基本质量画面时，对应的码流标识列表中只包含层0，所以层0就是需抽取的子码流层标识，而码流层标识为层0的子码流就是需抽取的子码流。
当客户端选择中等质量画面时，对应的码流标识列表中只包含层1，所以层1就是需抽取的子码流层标识，而码流层标识为层1的子码流就是需抽取的子码流。
当客户端选择高质量画面时，对应的码流标识列表中只包含层2，所以层0就是需抽取的子码流层标识，而码流层标识为层2的子码流就是需抽取的子码流。
示例2，cli_enhancement_type等于1时，待抽取码流中只包含依赖增强层子码流。
由于依赖增强层子码流不可以单独解码，所以依赖增强层子码流必须结合基础层子码流一起才能实现解码，所以所述质量等级(除基本质量以外)对应的码流标识列表中应至少包含两个码流层标识，基本质量对应码流标识列表中可以只包含一个码流层标识。
表5为本申请提供的又一种质量等级与层标识的对应表。
表5本申请提供的又一种质量等级与层标识的对应表
质量等级 与质量等级相对应的层标识列表
0(基本质量画面) 层0
1(中等质量画面) 层0，层1
2(高质量画面) 层0，层2或层0，层1，层2
参见表5，当用户选择基本质量画面时，对应的码流标识列表中只包含层0，所以层0就是需抽取的子码流层标识，而码流层标识为层0的子码流就是需抽 取的子码流。
当用户选择中等质量画面时，对应的码流标识列表中同时包含层0，层1，所以层0和层1就是需抽取的子码流层标识，而码流层标识为层0和层1的子码流就是需抽取的子码流。
当用户选择高质量画面时，对应的码流标识列表中同时包含层0，层2，所以层0和层2就是需抽取的子码流层标识，而码流层标识为层0和层2的子码流就是需抽取的子码流。
又或者，当用户选择高质量画面时，对应的码流标识列表中同时包含层0，层1和层2，所以层0，层1和层2就是需抽取的子码流层标识，而码流层标识为层0，层1和层2的子码流就是需抽取的子码流。
基本质量对应码流标识列表中也可以包含一个以上的码流层标识。表6为本申请提供的再一种质量等级与层标识的对应表。
表6本申请提供的再一种质量等级与层标识的对应表
质量等级 与质量等级相对应的层标识列表
0(基本质量画面) 层0，层1
1(中等质量画面) 层0，层1，层2或层0，层2
2(高质量画面) 层0，层1，层2，层3或其他可能的组合
参见表6，当用户选择基本质量画面时，对应的码流标识列表中同时包含层0，层1，所以层0和层1就是需抽取的子码流层标识，而码流层标识为层0和层1的子码流就是需抽取的子码流。
示例3，cli_enhancement_type等于2，待抽取码流中同时包含独立增强层子码流和依赖增强层子码流。
由于待抽取码流中同时包含独立增强层子码流和依赖增强层子码流，所述质量等级对应的码流标识列表中可以包含一个或一个以上的码流层标识。表7为本申请提供的另一种质量等级与层标识对应表。
表7本申请提供的另一种质量等级与层标识对应表
质量等级 与质量等级相对应的层标识列表
0(基本质量画面) 层0
1(中等质量画面) 层1
2(高质量画面) 层1，层2或层0，层2
参见表7，当用户选择中等质量画面时，对应的码流标识列表中只包含层1，所以层1就是需抽取的子码流层标识，而码流层标识为层1的子码流就是需抽取的子码流。
当用户选择高质量画面时，对应的码流标识列表中可以包含层1，层2，所以层1和层2就是需抽取的子码流层标识，而码流层标识为层1和层2的子码流就是需抽取的子码流。
所述质量等级对应的码流标识列表中包含的需抽取的子码流标识的数量应不大于前述增强层子码流数量(num_enhancement_layers)参数所指示的数量。
步骤4.根据所述需抽取的子码流标识，从所述待抽取码流中抽取需抽取的子码流。
根据子码流标识从完整的码流中抽取子码流属于标准中已经定义的相关技术，本领域技术人员可以根据标准协议内容实施，本方案中对抽取过程不进行详细描述。
因为参数集(Parameter Set)是视频编解码标准中一个重要的语法结构，可以包含编解码过程中所需使用的公共信息，如标志位(flag)、公共参数信息(如最大层数(max layers)等)。在实际应用本申请方案时，需要确保参数集中相关参数的值的设置与跨层指示SEI消息中的参数保持一致。例如，当跨层指示SEI消息中的cli_enhancement_type取值等于0时，所述跨层指示SEI消息关联的待抽取码流所关联的视频参数集中的vps_all_independent_layers_flag参数的取值应为1；当跨层指示SEI消息中的cli_enhancement_type取值不等于0时，所述跨层指示SEI消息关联的待抽取码流所关联的视频参数集中的vps_all_independent_layers_flag参数的取值应为0。再比如，视频参数集中的最大层数参数vps_max_layers_minus1的取值应与跨层指示SEI消息中的增强层子码流数量num_enhancement_layers的取值保持一致。
实施例二
本实施例给出一种跨层指示SEI消息的语法结构和语义信息的示例，可以对应实施例一中所描述的能力。表8为本申请提供的一种跨层指示SEI消息的语法结构表。
表8本申请提供的一种跨层指示SEI消息的语法结构表
Figure PCTCN2022081593-appb-000003
参见表8，增强类型参数，即cli_enhancement_type：用于指示SEI消息关联的待解码码流中的增强层子码流的类型，如实施例一中描述，当cli_enhancement_type＝0时，指示待解码码流中只包含独立增强层子码流。当cli_enhancement_type＝1时，指示待解码码流中只包含依赖增强层子码流。当cli_enhancement_type＝2时，指示待解码码流中同时包含独立增强层子码流和依赖增强层子码流。
增强层数量参数，即cli_num_enhancement_layers：指示SEI消息关联的待解码码流中的增强层子码流的数量。
层标识，即cli_layer_id[i]：指示第i个子码流的层(layer)标识。
质量等级参数，又称质量等级，即cli_quality_level_idx：指示所述待抽取码流中的子码流可以提供的质量等级(集合)，见实施例一中描述。
质量等级对应的层标识，即cli_extractable_layer_id[i]：指示质量等级为i时对应的可抽取子码流的层标识，此时SEI消息关联的待解码码流中的增强层类型应为独立增强型，即cli_enhancement_type＝0。
cli_extractable_layer_id[i]也可以表示为cli_extractable_layer_id[i][0]，指示质量等级为i时对应的可抽取子码流的层标识。
cli_extractable_layer_flag[i][j]指示质量等级为i时第j层子码流的可抽取标志位，当其值等于1时，表示第j层的子码流是质量等级为i时的可抽取子码流，当其值等于0时，表示第j层的子码流不是质量等级为i时的可抽取子码流。满足cli_extractable_layer_flag[i][j]等于1的所有j对应的子码流层标识组成质量等级为i时对应的可抽取子码流标识列表。
本实施例只是给出了一种可行的跨层指示SEI消息语法结构示例，所有符合实施例一中所描述的全部或部分方案的SEI消息都是在本申请方案保护范围内的。
实施例三
本实施例在实施例一的基础上，增加关键帧标志位，通过跨层指示SEI消息中的关键帧标志位指示待解码码流是否需要涉及跨层操作，这种特定类型的SEI消息(即实施例二中的SEI消息)本身不指示任何信息，当待解码码流中包含跨层指示SEI消息时，需要去解析关键帧标志位。跨层指示SEI消息中还可以包含取消标志位(cli_cancel_flag)和持久性标志位(cli_persistence_flag)。表9为本申请提供的又一种跨层指示SEI消息的语法结构表。
参见表9，取消标识指示信息，即cli_cancel_flag：取值等于1时，表示所述跨层指示SEI消息取消了任何先前的跨层指示SEI消息的持久性且不使用相关的SEI功能。反之，取值等于0时，表示紧随其后的是跨层指示信息如第一多层编码参数信息。
持久性标识信息，即cli_persistence_flag：用于指示当前跨层指示SEI消息的持久性。当持久性标志位(cli_persistence_flag)的取值等于0时，指示跨层指示SEI消息只对当前解码的(子)图像适用。当持久性标志位(cli_persistence_flag)的取值等于1时，指示所述跨层指示SEI消息不仅对当前解码的(子)图像适用，对后续在持续范围(peisistence_scope)内的解码(子)图像也持续适用，
例如，跨层指示SEI消息的持续范围(peisistence_scope)可以是一个访问单元(Access Unite，AU)，也可以是已编码视频序列(Coded Video Sequence，CVS)，也可以是未指定(unspecified)，本申请对此并不作限制。
关键帧标识信息，即cli_key_frame_flag：取值等于1时，指示待解码码流 为多层编码码流，需要根据增强层类型对待解码码流进行子码流抽取过程，因此所述待解码码流也可以表示为待抽取码流。反之，当其取值等于0时，指示待解码码流非多层编码码流，按照标准中原有的基础解码流程对所述待解码码流执行相应的操作。
cli_extractable_layer_flag[i][j]指示质量等级为i时第j层子码流的可抽取标志位，当其值等于1时，表示第j层的子码流是质量等级为i时的可抽取子码流，当其值等于0时，表示第j层的子码流不是质量等级为i时的可抽取子码流。满足cli_extractable_layer_flag[i][j]等于1的所有j对应的子码流层标识组成质量等级为i时对应的可抽取子码流标识列表。
与实施例二所不同的是，本实施例中统一采用cli_extractable_layer_flag[i][j]指示质量等级为i时第j层子码流的可抽取标志位。但是由于cli_enhancement_type＝＝0时，此时SEI消息关联的待解码码流中的增强层子码流类型应为独立增强型子码流，故每个质量等级下对应的可抽取子码流只有1个，所以只要有一个j使得cli_extractable_layer_flag[i][j]的值为1时，该j层的子码流就是等级为i时对应的可抽取子码流，无需再继续执行do循环。
表9本申请提供的又一种跨层指示SEI消息的语法结构表
Figure PCTCN2022081593-appb-000004
Figure PCTCN2022081593-appb-000005
实施例四
本实施例给出另一种跨层指示SEI消息的语法结构和语义信息的示例。利用H.266/VVC version1标准中已有的两种特殊SEI消息类型实现本申请方案。这两种特殊SEI消息类型分别为国际电信联盟电信标准分局(ITU Telecommunication Standardization Sector，ITU-T)T.35建议书注册的用户数据(User data registered by Recommendation ITU-T T.35)SEI消息(payloadType＝4)和用户数据未注册的(User data unregistered)SEI消息(payloadType＝5)，其语法结构和语义信息分别如表10和表11所示，表10为本申请提供的又一种SEI消息的语法结构表。表11为本申请提供的再一种SEI消息的语法结构表。
表10本申请提供的又一种SEI消息的语法结构表
Figure PCTCN2022081593-appb-000006
Figure PCTCN2022081593-appb-000007
表11本申请提供的再一种SEI消息的语法结构表
Figure PCTCN2022081593-appb-000008
与ITU-T T.35建议书注册的用户数据SEI消息类似，可以通过用户数据未注册的SEI消息中的语法元素user_data_payload_byte来实现本申请方案，例如，可以在语法元素user_data_payload_byte中包含实施例一中所提到的能力或者包含实施例二中示例的SEI消息中的全部或部分内容，从而达到实现本申请方案的目的。
除了前述提到的分别通过语法元素itu_t_t35_payload_byte和user_data_payload_byte可以实现利用H.266/VVC version1标准中已有的ITU-T T.35建议书注册的用户数据SEI消息和用户数据未注册的SEI消息来实现本申请方案以外，其他任何利用这两种SEI消息实现本申请方案的扩展也都是支持的。
实施例五
本实施例是一种视频的编码方法，应用于对视频进行编码处理的终端设备。所述装置的输入是视频所包含的图像，输出是图像码流或包含图像码流的传输 流或媒体文件。
本实施例中编码方法包括如下步骤：
步骤S1.读取图像，该图像是视频序列中的一幅完整图像，也可以是一幅完整图像中的一个子图像。
步骤S2.判断所述图像是否涉及多层编码，若是，则执行步骤S3，否则将执行步骤S5。
判断所述图像是否涉及多层编码的方法包括但不限于以下3种：1)所述读取的图像中包含需要进行多层编码的指示，例如指示所述图像为关键画面，需要对其进行多层编码；2)编码端分析图像特性，根据图像特性确定需要进行多层编码；3)所述读取的图像包含在前一指示多层编码图像对应的跨层指示SEI消息的持续范围(persistent_scope)内。
多层编码可以认为是将图像编码成多层码流。
步骤S3.获取第一多层编码参数信息。
所述第一多层编码参数信息包括增强类型参数，增强层数量参数，质量等级(quality_level_idx)参数以及质量等级所对应的码流层标识等。
ITU-T T.35建议书注册的用户数据SEI消息包含按照ITU-T T.35建议书的规定注册的用户数据。
可以通过该SEI消息中的语法元素itu_t_t35_payload_byte来实现本申请方案，例如，可以在语法元素itu_t_t35_payload_byte中包含实施例一中所提到的能力或者包含实施例二中示例的SEI消息中的全部或部分内容，从而达到实现本申请方案的目的。
步骤S4.对图像编码生成多层编码码流，执行步骤S8。
编码方法为相关技术可实现，本申请不作限制，生成的多层编码码流应满足第一多层编码参数信息。此外，本申请并不限制S3和S4之间的先后顺序。
步骤S5.判断所述图像是否是用于结束多层编码，若是，则执行步骤S6，否则执行步骤S7’。
判断所述图像是否是用于结束多层编码的方法包括但不限于以下3种：1)所述读取的图像包含需要结束多层编码的指示；2)编码端分析图像特性，根据图像特性确定需要结束多层编码；3)编码端分析图像特性，确定不需要进行多层编码，但所述读取的图像仍包含在前一指示多层编码图像对应的跨层指示SEI消息的持续范围(persistent_scope)内。
步骤S6.获取第二多层编码参数信息。
所述第二多层编码参数信息至少包括取消标志位(cancel_flag)，即取消标识指示信息。
步骤S7/S7’.对图像编码生成码流。
编码方法为相关技术可实现，本申请不作限制。此外，本申请并不限制S6和S7之间的先后顺序。
步骤S8.将携带多层编码参数信息的SEI消息写入码流。
将携带第一多层编码参数信息或第二多层编码参数信息的SEI消息写入码流。所述携带第一多层编码参数信息或第二多层编码参数信息的SEI消息可以是实施例二到四中的所描述的SEI消息示例中的任意一种。
多层编码参数信息包括第一多层编码参数信息和第二多层编码参数信息。第一多层编码参数信息添加至基于步骤S4生成的多层编码码流，第二多层编码参数信息添加至基于步骤S7生成的码流。
当所述SEI消息携带第一多层编码参数信息且所述携带第一多层编码参数信息的SEI消息中包含持续性范围(peisistence_scope)参数时，编码端还可以检查所述第一多层编码参数信息与前一携带多层编码参数信息的SEI消息中的第三多层编码参数信息是否完全相同，若第一多层编码参数信息与第三多层编码参数信息完全相同且携带第三多层编码参数信息的SEI消息的持续性范围(peisistence_scope)包括当前图像时，可以选择不用将携带第一多层编码参数信息的SEI消息写入码流。
步骤S9.输出图像码流或包含图像码流的传输流或媒体文件。
随着视频监控在城市安防中发挥的作用越来越大，需要传输和存储的视频数据也日益增多，如何能够最大效率地实现快速传输以及延长关键视频的存储时长就愈发重要。尤其是随着实时智能分析技术的不断发展和成熟，快速区分出视频流中的关键画面，利用高质量编码尽量保存关键画面中的细节，降低普通画面的质量是一种必然的趋势。而本申请方案正是为了实现上述目的而对编解码标准所做的一种优化。未来应用前景明确，市场空间较大。
在一个示例性实施方式中，图4为本申请实施例提供的一种码流处理装置的结构示意图，该装置可以集成在终端设备中，如图4所示，该装置包括：
第一获取模块31，设置为获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；第二获取模块32，设置为获取所述编码指示信息；解码模块33，设置为根据所述编码指示信息解码所述编码后码流。
本实施例提供的码流处理装置用于实现如图1所示实施例的码流处理方法，本实施例提供的码流处理装置实现原理和技术效果与图1所示实施例的码流处理方法类似，此处不再赘述。
在上述实施例的基础上，提出了上述实施例的变型实施例，为了使描述简要，在变型实施例中仅描述与上述实施例的不同之处。
在一个实施例中，所述编码指示信息包括SEI消息，所述码流类型对应的码流包括多层编码码流和终止码流，所述终止码流为终止编码为多层编码码流的码流。
在一个实施例中，所述SEI消息包括如下至少一项：
扩展后的用户数据SEI消息；扩展后的用户数据未注册的SEI消息；新增的SEI消息。
在一个实施例中，所述SEI消息，包括如下至少一项：
增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息；所述增强类型参数指示所述编码后码流中包含的增强层子码流的增强类型，所述子码流参数信息用于确定所需从所述编码后码流中抽取的子码流，所述关键帧标识信息指示所述编码后码流是否为多层编码码流，所述取消标识指示信息指示所对应SEI消息是否取消了前一SEI消息的持久性限制，所述持久性标识信息指示是否设置持久性限制。
在一个实施例中，子码流参数信息包括如下至少一项：增强层数量参数、质量等级、质量等级所对应子码流的层标识，所述质量等级所对应子码流的数量为一个或多个；其中，所述增强层数量参数指示所述编码后码流中包括的增强层子码流的数量，所述质量等级指示所述编码后码流中子码流可以提供的质量等级。
在一个实施例中，解码模块33设置为：
根据所述编码指示信息从所述编码后码流中获取子码流；解码所述子码流。
在一个实施例中，解码模块33根据所述编码指示信息从所述编码后码流中获取子码流，包括：
获取所述编码指示信息所包括的增强类型参数；获取对应所述增强类型参数的子码流参数信息；根据所述子码流参数信息和所需质量等级，确定需提取的子码流的层标识；根据所述子码流的层标识，从所述编码后码流中提取子码流。
在一个示例性的实施方式中，本申请提供了又一种码流处理装置，图5为 本申请实施例提供的一种码流处理装置的结构示意图，该装置集成在终端设备中。如图5所示，该装置包括：
生成模块41，设置为生成图像编码码流和所述图像编码码流对应的编码指示信息；添加模块42，设置为将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；发送模块43，设置为发送所述编码后码流。
本实施例提供的码流处理装置用于实现如图3所示实施例的码流处理方法，本实施例提供的码流处理装置实现原理和技术效果与图3所示实施例的码流处理方法类似，此处不再赘述。
在上述实施例的基础上，提出了上述实施例的变型实施例，为了使描述简要，在变型实施例中仅描述与上述实施例的不同之处。
在一个实施例中，在所述图像编码码流为多层编码码流的情况下，所述编码指示信息包括如下至少一项：增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息。
在一个实施例中，在所述图像编码码流在前一个编码指示信息的持续范围内，且所述图像编码码流对应的图像结束编码为多层编码码流的情况下，所述编码指示信息包括取消标识指示信息，所述图像编码码流为多层编码码流或终止码流。
在一个实施例中，所述编码指示信息用于生成多层编码码流，或所述编码指示信息在生成多层编码码流后获取。
在一个示例性实施方式中，本申请实施例提供了一种终端设备，图6为本申请实施例提供的一种终端设备的结构示意图。如图6所示，本申请提供的终端设备，包括一个或多个处理器51和存储装置52；该终端设备中的处理器51可以是一个或多个，图6中以一个处理器51为例；存储装置52用于存储一个或多个程序；所述一个或多个程序被所述一个或多个处理器51执行，使得所述一个或多个处理器51实现如本申请实施例中所述的码流处理方法。
终端设备还包括：通信装置53、输入装置54和输出装置55。
终端设备中的处理器51、存储装置52、通信装置53、输入装置54和输出装置55可以通过总线或其他方式连接，图6中以通过总线连接为例。
输入装置54可用于接收输入的数字或字符信息，以及产生与终端设备的用户设置以及功能控制有关的按键信号输入。输出装置55可包括显示屏等显示设备。
通信装置53可以包括接收器和发送器。通信装置53设置为根据处理器51 的控制进行信息收发通信。信息包括但不限于编码后码流
存储装置52作为一种计算机可读存储介质，可设置为存储软件程序、计算机可执行程序以及模块，如本申请实施例所述码流处理方法对应的程序指令/模块(例如，码流处理装置中的第一获取模块31、第二获取模块32和解码模块33，又如码流处理装置中的获取模块41、添加模块42和发送模块43)。存储装置52可包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端设备的使用所创建的数据等。此外，存储装置52可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储装置52可包括相对于处理器51远程设置的存储器，这些远程存储器可以通过网络连接至终端设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
本申请实施例还提供一种存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现本申请任一所述方法，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现本申请实施例中任一所述的码流处理方法。
码流处理方法包括：
获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；获取所述编码指示信息；根据所述编码指示信息解码所述编码后码流。
或者，码流处理方法，包括：
生成图像编码码流和所述图像编码码流对应的编码指示信息；将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；发送所述编码后码流。
本申请实施例的计算机存储介质，可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是，但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read Only Memory，ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式只读光盘存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指 令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于：电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、无线电频率(Radio Frequency，RF)等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络，包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)，连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
以上所述，仅为本申请的示例性实施例而已。
本领域内的技术人员应明白，术语终端设备涵盖任何适合类型的无线用户设备，例如移动电话、便携数据处理装置、便携网络浏览器或车载移动台。
一般来说，本申请的多种实施例可以在硬件或专用电路、软件、逻辑或其任何组合中实现。例如，一些方面可以被实现在硬件中，而其它方面可以被实现在可以被控制器、微处理器或其它计算装置执行的固件或软件中，尽管本申请不限于此。
本申请的实施例可以通过移动装置的数据处理器执行计算机程序指令来实现，例如在处理器实体中，或者通过硬件，或者通过软件和硬件的组合。计算机程序指令可以是汇编指令、指令集架构(Instruction Set Architecture，ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码。
本申请附图中的任何逻辑流程的框图可以表示程序步骤，或者可以表示相互连接的逻辑电路、模块和功能，或者可以表示程序步骤与逻辑电路、模块和 功能的组合。计算机程序可以存储在存储器上。存储器可以具有任何适合于本地技术环境的类型并且可以使用任何适合的数据存储技术实现，例如但不限于只读存储器(Read-Only Memory，ROM)、随机访问存储器(Random Access Memory，RAM)、光存储器装置和系统(数码多功能光碟(Digital Video Disc，DVD)或光盘(Compact Disk，CD))等。计算机可读介质可以包括非瞬时性存储介质。数据处理器可以是任何适合于本地技术环境的类型，例如但不限于通用计算机、专用计算机、微处理器、数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程逻辑器件(Field-Programmable Gate Array，FPGA)以及基于多核处理器架构的处理器。

Claims (15)

  1. 一种码流处理方法，包括：
    获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；
    获取所述编码指示信息；
    根据所述编码指示信息解码所述编码后码流。
  2. 根据权利要求1所述的方法，其中，所述编码指示信息包括补充增强信息SEI消息，所述码流类型对应的码流包括多层编码码流和终止码流，所述终止码流为终止编码为多层编码码流的码流。
  3. 根据权利要求2所述的方法，其中，所述SEI消息包括如下至少一项：
    扩展后的用户数据SEI消息；扩展后的用户数据未注册的SEI消息；新增的SEI消息。
  4. 根据权利要求2所述的方法，其中，所述SEI消息，包括如下至少一项：
    增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息；
    所述增强类型参数指示所述编码后码流中包含的增强层子码流的增强类型，所述子码流参数信息用于确定所需从所述编码后码流中抽取的子码流，所述关键帧标识信息指示所述编码后码流是否为多层编码码流，所述取消标识指示信息指示所对应SEI消息是否取消了前一SEI消息的持久性限制，所述持久性标识信息指示是否设置持久性限制。
  5. 根据权利要求4所述的方法，其中，所述子码流参数信息包括如下至少一项：增强层数量参数、质量等级、质量等级所对应子码流的层标识，所述质量等级所对应子码流的数量为至少一个；
    其中，所述增强层数量参数指示所述编码后码流中包括的增强层子码流的数量，所述质量等级指示所述编码后码流中子码流可以提供的质量等级。
  6. 根据权利要求1所述的方法，其中，所述根据所述编码指示信息解码所述编码后码流，包括：
    根据所述编码指示信息从所述编码后码流中获取子码流；
    解码所述子码流。
  7. 根据权利要求6所述的方法，其中，所述根据所述编码指示信息从所述编码后码流中获取子码流，包括：
    获取所述编码指示信息所包括的增强类型参数；
    获取对应所述增强类型参数的子码流参数信息；
    根据所述子码流参数信息和所需质量等级，确定需提取的子码流的层标识；
    根据所述子码流的层标识，从所述编码后码流中提取子码流。
  8. 一种码流处理方法，包括：
    生成图像编码码流和所述图像编码码流对应的编码指示信息；
    将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；
    发送所述编码后码流。
  9. 根据权利要求8所述的方法，其中，
    在所述图像编码码流为多层编码码流的情况下，所述编码指示信息包括如下至少一项：增强类型参数、子码流参数信息、关键帧标识信息、取消标识指示信息和持久性标识信息。
  10. 根据权利要求8所述的方法，其中，
    在所述图像编码码流在前一个编码指示信息的持续范围内，且所述图像编码码流对应的图像结束编码为多层编码码流的情况下，所述编码指示信息包括取消标识指示信息，所述图像编码码流为多层编码码流或终止码流。
  11. 根据权利要求8所述的方法，其中，所述编码指示信息用于生成多层编码码流，或所述编码指示信息在生成多层编码码流后获取。
  12. 一种码流处理装置，包括：
    第一获取模块，设置为获取编码后码流，所述编码后码流包括编码指示信息，所述编码指示信息指示所述编码后码流的码流类型；
    第二获取模块，设置为获取所述编码指示信息；
    解码模块，设置为根据所述编码指示信息解码所述编码后码流。
  13. 一种码流处理装置，包括：
    生成模块，设置为生成图像编码码流和所述图像编码码流对应的编码指示信息；
    添加模块，设置为将所述编码指示信息添加至所述图像编码码流中，得到编码后码流；
    发送模块，设置为发送所述编码后码流。
  14. 一种终端设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-11任一项所述的码流处理方法。
  15. 一种存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-11任一项所述的码流处理方法。
PCT/CN2022/081593 2021-04-20 2022-03-18 码流处理方法、装置、终端设备及存储介质 WO2022222656A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/555,912 US20240223794A1 (en) 2021-04-20 2022-03-18 Methods and apparatuses for processing bitstream, terminal device, and storage medium
EP22790765.6A EP4329306A1 (en) 2021-04-20 2022-03-18 Methods and apparatuses for processing code stream, terminal device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110426571.1 2021-04-20
CN202110426571.1A CN115225913A (zh) 2021-04-20 2021-04-20 一种码流处理方法、装置、终端设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022222656A1 true WO2022222656A1 (zh) 2022-10-27

Family

ID=83604705

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/081593 WO2022222656A1 (zh) 2021-04-20 2022-03-18 码流处理方法、装置、终端设备及存储介质

Country Status (4)

Country Link
US (1) US20240223794A1 (zh)
EP (1) EP4329306A1 (zh)
CN (1) CN115225913A (zh)
WO (1) WO2022222656A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115866254A (zh) * 2022-11-24 2023-03-28 亮风台（上海）信息科技有限公司 一种传输视频帧及摄像参数信息的方法与设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116828231B (zh) * 2023-08-02 2024-05-24 纽扣数字智能科技(深圳)集团有限公司 一种视频传输优化方法、系统、电子设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160212439A1 (en) * 2014-12-18 2016-07-21 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
CN110419223A (zh) * 2017-03-21 2019-11-05 高通股份有限公司 必需的及非必需的视频补充信息的信号发送
WO2021049580A1 (ja) * 2019-09-11 2021-03-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、復号装置、符号化方法、および復号方法
WO2021061392A1 (en) * 2019-09-24 2021-04-01 Futurewei Technologies, Inc. Error mitigation for sublayers in video coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160212439A1 (en) * 2014-12-18 2016-07-21 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
CN110419223A (zh) * 2017-03-21 2019-11-05 高通股份有限公司 必需的及非必需的视频补充信息的信号发送
WO2021049580A1 (ja) * 2019-09-11 2021-03-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、復号装置、符号化方法、および復号方法
WO2021061392A1 (en) * 2019-09-24 2021-04-01 Futurewei Technologies, Inc. Error mitigation for sublayers in video coding

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115866254A (zh) * 2022-11-24 2023-03-28 亮风台（上海）信息科技有限公司 一种传输视频帧及摄像参数信息的方法与设备

Also Published As

Publication number Publication date
EP4329306A1 (en) 2024-02-28
CN115225913A (zh) 2022-10-21
US20240223794A1 (en) 2024-07-04

Similar Documents

Publication Publication Date Title
WO2022222656A1 (zh) 码流处理方法、装置、终端设备及存储介质
US11076160B2 (en) Devices and methods for identifying a leading picture
US20230007316A1 (en) Apparatus for transmitting broadcast signal, apparatus for receiving broadcast signal, method for transmitting broadcast signal and method for receiving broadcast signal
EP3399752B1 (en) Image decoding method and decoding device
KR102332387B1 (ko) 수신 장치, 수신 방법, 송신 장치 및 송신 방법
CN110662114B (zh) 视频处理方法、装置、电子设备及存储介质
TW200822758A (en) Scalable video coding and decoding
US10270989B2 (en) Broadcasting signal transmission device, broadcasting signal reception device, broadcasting signal transmission method, and broadcasting signal reception method
US11956159B2 (en) Transmission device, transmission method, reception device, and reception method
WO2020109154A1 (en) Method, device, and computer program for encapsulating media data into a media file
US11336965B2 (en) Method and apparatus for processing video bitstream, network device, and readable storage medium
US10412422B2 (en) Apparatus for transmitting broadcasting signal, apparatus for receiving broadcasting signal, method for transmitting broadcasting signal, and method for receiving broadcasting signal
US20240080487A1 (en) Method, apparatus for processing media data, computer device and storage medium
US20230319374A1 (en) Method and device for creating/receiving media file containing layer information, and media file transfer method
CN116506412A (zh) 兼容h.264和h.265编码标准的视频实时加密传输方法及系统
WO2024079718A1 (en) Apparatus and method for integrating neural-network post-filter supplemental enhancement information with iso base media file format
CN118296178A (zh) 文件生成方法、装置、设备及可读存储介质
CN114040172A (zh) 一种基于视频监控系统的车辆卡口数据融合传输方法及系统
CN116235502A (zh) 生成/接收包括输出层集合信息的媒体文件的方法和设备及发送媒体文件的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22790765

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18555912

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022790765

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022790765

Country of ref document: EP

Effective date: 20231120