WO2014047938A1

WO2014047938A1 - Digital video code stream decoding method, splicing method and apparatus

Info

Publication number: WO2014047938A1
Application number: PCT/CN2012/082483
Authority: WO
Inventors: 张园园; 杨晓峰; 石腾
Original assignee: 华为技术有限公司
Priority date: 2012-09-29
Filing date: 2012-09-29
Publication date: 2014-04-03
Also published as: CN103959796A; CN103959796B

Abstract

Provided are a digital video code stream decoding method, a digital video code stream splicing method and apparatus. By means of a parameter set identification modification message of a first video containing parameter set identification (ID) modification information of the first video, a terminal decodes and outputs the first video according to the parameter set identification (ID) modification information. Not only can a correct parameter set be used when a terminal decodes a first video code stream, but also additional hardware resources do not need to be added to a splicer of an existing network or there is no need to replace the splicer of the existing network with a new splicer, which requires fewer operating costs than those required in the prior art.

Description

Method and device for splicing digital video code stream decoding method

TECHNICAL FIELD The present application relates to digital video splicing techniques, and more particularly to a decoding method, a splicing method and apparatus for a digital video stream. Background technique

The code stream splicing technology has a wide range of application scenarios. For example, operators can select personalized advertisements according to user preferences, and replace the universal advertisements in the program with personalized advertisements by stream stream splicing technology to improve the hit rate and viewing effect of advertisements, thereby improving user experience and operator revenue.

In video coding, a video content can be divided into one or more encoded video sequences (CVS: Coded Video Sequence); one CVS contains multiple pictures (Picture); one picture can be divided into one or more pictures (Slice One or more encoded pictures at the same time are called an access unit (AU: Access Unit). After the encoder encodes the video content, the output video stream is in the network abstraction layer unit (NALU:

Network Abstraction Layer Unit) is composed of units. NALU is divided into two categories: Video Coding Layer (VCL: Video Coding Layer) NALU and non-VCL NALU. One of the VCL NALUs is a coded slice, so it is often referred to as the code slice NALU; instead of VCL NALU is metadata and additional information. Non-VCL NALU can be further divided into: Parameter set NALU: NALU for encapsulating parameter sets, Supplemental Enhancement Information (SEI) NALU: NALU for encapsulating SEI messages. The parameter set NALU includes a parameter set and a parameter set identifier corresponding to the parameter set (ID: Identifier). If the parameter set also references other parameter sets, the parameter set further includes a parameter set corresponding to the parameter set of the bow I. ID.

The parameter set contains header information shared by multiple VCL NALUs, such as profile and level information, video color format information, video resolution information, and the like. Parameter sets can be divided into multiple types, and different types of parameter sets contain different types of shared header information. A video stream can contain one or more parameter sets of the same type. A parameter set contains a parameter set ID, and the value of the parameter set ID is unique in a parameter set of the same type of a code stream. Shared header information After being placed in the parameter set, the corresponding VCL NALU refers to the header information in the parameter set through the parameter set ID. The parameter set was introduced by the video coding standard H.264 and is widely used by subsequent video coding standards. The purpose of introducing a parameter set is for robust and efficient transmission of header information. After the parameter set is introduced, the header information shared by multiple VCL NALUs needs to be transmitted only once, thus improving the transmission efficiency.

Different video coding standards define different parameter sets. The parameter sets defined by the H.264 standard are: Sequence Parameter Set (SPS) and PPS. The Moving Picture Experts Group (MPEG: Moving Pictures Experts Group) is developing a parameter set defined by the HEVC (High Efficiency Video Coding) standard in addition to SPS and PPS, and also includes a Video Parameter Set (VPS: Video Parameter Set). Two parameter sets of APS: Adaptation Parameter Set, the parameter set defined by MPEG's 3DVC: Three-Dimensional Video Coding standard, in addition to SPS and PPS, includes depth range parameters. Set (DRPS: Depth Range Parameter Set). However, the parameter sets defined by different video coding standards can be divided into two categories: the parameter set directly referenced by VCL NALU in the video stream and the parameter set indirectly referenced by VCL NALU. If the parameter set is a parameter set directly referenced by VCL NALU, the VCL NALU contains the parameter set ID of the parameter set; if the parameter set is a parameter set indirectly referenced by VCL NALU, the parameter set ID of the parameter set is not included in the VCL NALU, and The parameter set ID of the parameter set is included in the directly referenced parameter set of the VCL NALU or in the indirectly referenced parameter set.

In the video splicing technology, the bitstream splicing technology receives a first video code stream (such as personal advertisement content, etc.) and a second video code stream (such as a program requested by a terminal, etc.) for the splicer. The splicer sends the first video code stream to the terminal during the splicing period, and sends the second video code stream to the terminal at other time periods in the process of outputting the second video code stream for the terminal. The second video and the first video are independently encoded using different encoders (second encoder, first encoder). Since the parameter set identifier (ID: Identifier) is entropy coded, the smaller the value of the parameter set ID is, the smaller the number of bits required to represent the parameter set ID is. Therefore, when the encoder assigns the parameter set ID, it is often allocated from 0 value. Therefore, the parameter sets ID assigned by the two encoders to the same type of parameter set may be the same. The second video code stream is the same as the parameter set ID of the same type of parameter set in the first video code stream, and the parameter set ID in the output video code stream cannot uniquely identify a parameter set. When the parameter set adopts the out-of-band transmission mode, it will cause the wrong parameter set to be used in decoding, resulting in decoding errors. For example, a first The picture parameter set (PPS: Picture Parameter Set) ID in the network abstraction layer unit of the video coding layer (VCL: Video Coding Layer) of the video is the same as the PPS ID of a certain PPS of the second video. The decoder may decode the first video VCL NALU using the PPS of the second video, resulting in a decoding error.

In order to solve the above problem, the prior art modifies the parameter set ID in the first video bitstream by the splicer to ensure the uniqueness of the parameter set ID in the output video bitstream. The terminal receives the output video stream, decodes each VCL NALU, and obtains a parameter set referenced by the VCL NALU from the parameter set obtained by the out-of-band method according to the parameter set ID, and uses these parameter sets to decode the VCL NALU.

As mentioned above, since the parameter set ID is entropy encoded, the prior art requires the splicer to have entropy decoding and entropy coding capabilities. However, splicers in existing networks usually do not have entropy decoding and entropy coding capabilities. Entropy decoding and entropy coding operations require more computing resources and storage resources, and parameter set IDs appear frequently in video streams, ie, splicing. The device requires frequent entropy decoding and entropy encoding operations. Therefore, in order to enable the splicer to have entropy decoding and entropy coding capabilities, it is necessary to add additional hardware resources to the splicer in the existing network, or use a new splicer to replace the splicer in the existing network, thereby giving the operator Adding a lot of additional operating costs. Summary of the invention

The invention provides a decoding and splicing method and device for a digital video code stream, which not only ensures that the correct parameter set is used when the terminal decodes the first video code stream, but also does not need to add extra on the splicer of the existing network. The hardware resources, or the replacement of splicers in existing networks with new splicers, are less expensive to operate than in the prior art.

A first aspect of the embodiments of the present invention provides a method for decoding a digital video code stream, including: receiving a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video includes a parameter set identifier ID modification information of the first video; acquiring a parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit according to the parameter set ID modification information of the first video; Decoding a parameter set ID of a parameter set of the VCL NALU to obtain a parameter set for decoding the VCL NALU; decoding the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded first video. With reference to the first implementation manner of the first aspect of the embodiment, the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original parameter corresponding to the parameter set type of the parameter set ID to be modified. An offset of the set ID; the first video code stream includes an original parameter set ID for decoding a parameter set of the VCL NALU; and the parameter set ID modification information acquisition according to the first video is used to decode the VCL The parameter set ID of the parameter set of the NALU is specifically obtained by acquiring, according to the parameter set type of the parameter set ID to be modified, a location corresponding to the parameter set type of the parameter set ID to be modified from the first video code stream. Deriving the original parameter set ID; obtaining the parameter set type of the parameter set ID to be modified according to the original parameter set ID and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified The corresponding parameter set for decoding the parameter set of the VCL NALU is combined with the second implementation manner of the first aspect of the embodiment of the present invention. The parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and is used for decoding the VCL. NALU parameter set a parameter set ID according to the parameter set ID modification information of the first video, and the parameter set ID of the parameter set for decoding the VCL NALU is specifically: according to the parameter set type of the parameter set ID to be modified The parameter set ID modification information acquires a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

In a third implementation manner of the first aspect of the embodiments of the present invention, the parameter set ID modification information includes an original parameter set ID of a parameter set type of the parameter set ID to be modified, and a corresponding to the original parameter set ID. Decoding the parameter set ID of the parameter set of the VCL NALU;

Determining, according to the parameter set ID modification information of the first video, the decoding used to decode the VCL

The parameter set ID of the parameter set of the NALU is specifically: acquiring a parameter set of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. ID.

In a fourth implementation manner of the first aspect of the embodiment, the first video parameter set ID modification information is a parameter set type of the parameter set ID to be modified, and the method further includes: sending a real-time transmission control protocol RTCP request to the splicer The RTCP request includes a parameter set type of the parameter set ID to be modified; receiving an RTCP response returned by the splicer, where the RTCP response carries an original parameter set corresponding to a parameter set type of a parameter set ID to be modified. The offset of the ID or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the original parameter set ID of the parameter set type of the parameter set ID to be modified And with the original parameter set ID Corresponding parameter set ID of the parameter set for decoding the VCL NALU; the parameter set ID of the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video is specifically: according to the The parameter set of the parameter set for decoding the VCL NALU corresponding to the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified carried in the RTCP response or the parameter set type of the parameter set ID to be modified ID, or from the parameter set ID modification information according to the original parameter set ID of the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID Obtaining a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

In conjunction with a first possibility of any implementation of the present invention, the method further includes: by means of an out-of-band method, before obtaining a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU Obtaining a modified parameter set of the first video, where the parameter set ID of the modified parameter set is a parameter set ID of a parameter set used for decoding the VCL NALU; or acquiring an original parameter set of the first video by using an outband manner; a parameter set ID for decoding a parameter set of the VCL NALU, the original parameter set of the first video carries the original parameter set ID; modifying the original parameter according to the parameter set ID of the parameter set for decoding the VCL NALU Set ID.

With reference to the second possibility of any implementation of the present invention, the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and the information acquisition is modified according to the parameter set ID of the first video. The parameter set ID of the parameter set for decoding the VCL NALU is specifically: obtaining a parameter set ID of a parameter set for directly decoding a VCL NALU directly referenced by the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, the method further comprising Obtaining, according to the obtained parameter set ID of the parameter set directly used by the VCL NALU for decoding the VCL NALU, the parameter set directly referenced by the VCL NALU in the first video code stream; obtaining the decoding from the directly referenced parameter set The parameter set ID of the parameter set indirectly referenced by the VCL NALU of the VCL NALU; or the parameter set type of the parameter set ID to be modified is the parameter set type directly referenced by the VCL NALU and the parameter set indirectly referenced by the VCL NALU The parameter set ID of the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video is specifically: respectively according to the parameter set of the first video The ID modification information acquires a parameter set ID of a directly referenced parameter set of the VCL NALU for decoding the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, and acquires a parameter set type corresponding to the VCL NALU indirect reference a parameter set ID of a parameter set for indirect reference of a VCL NALU for decoding a VCL NALU; The parameter set ID of the parameter set for decoding the VCL NALU acquires a parameter set for decoding the VCL NALU, specifically: a parameter set ID according to the obtained directly referenced parameter set and a parameter set ID of the indirectly referenced parameter set Obtaining a parameter set of a direct reference of the VCL NALU and a parameter set of an indirect reference of the VCL NALU;

Decoding the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded first video is specifically: a parameter set according to the obtained directly referenced VCL NALU and an indirect of the VCL NALU The referenced parameter set decodes the VCL NALU and outputs the decoded first video.

A second aspect of the embodiments of the present invention further provides a method for splicing a digital video code stream, including: receiving information of a first video code stream from an application server, where information of the first video code stream includes parameters of a first video stream And modifying the information according to the first video parameter set, and sending, by the first video parameter set, the first video code stream and the parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video carries the parameter of the first video And the identifier ID modification information is obtained, so that the terminal acquires a parameter set ID of a parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal is used to decode the VCL according to the The parameter set ID of the parameter set of the NALU acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.

With reference to the first implementation manner of the second aspect of the embodiment of the present invention, the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified, and the method further includes: receiving a real-time transmission control protocol RTCP request sent by the terminal The RTCP request includes the parameter set type of the parameter set ID to be modified; and returns an RTCP response to the terminal, where the RTCP response carries the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. Offset, or a parameter set ID carrying a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the original of the parameter set type carrying the parameter set ID to be modified A parameter set ID and a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the original parameter set ID.

With reference to the second implementation manner of the second aspect of the embodiment of the present invention, the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original parameter corresponding to the parameter set type of the parameter set ID to be modified. The offset of the set ID, or the parameter set type including the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU, or the original of the parameter set type including the parameter set ID to be modified a parameter set ID and a decoding corresponding to the original parameter set ID for decoding the VCL a parameter set ID of a parameter set of the NALU, the parameter set ID modification information further comprising a modification effective flag, the parameter set for decoding the VCL NALU being acquired according to the parameter set ID of the parameter set for decoding the VCL NALU The method further includes: sending a parameter set identifier modification message of the first video that is not valid when the second video code stream is sent to the terminal; receiving a second media description information acquisition request sent by the terminal; returning the second to the terminal The media description information acquisition response, the second media description information acquisition response carries a modified parameter set of the first video, and the parameter set ID of the modified parameter set of the first video is a parameter used for decoding the VCL NALU The parameter set ID of the set.

A third aspect of the embodiments of the present invention provides a method for splicing a digital video code stream, including: receiving media description information of a first video sent by a first content server, where media description information of the first video carries a first video The original parameter set and the parameter set of the first video and the original parameter set ID of the parameter set corresponding to each type; the media description information of the second video sent by the second content server, and the media description information of the second video The parameter set of the second video parameter set carrying the second video and the parameter set ID of the parameter set corresponding to each type; determining the parameter set ID of the first video of the same type and the parameter set of the second video The set ID is the same, and the parameter set identification modification information of the first video is determined; the information of the first video stream is sent to the splicer, where the information of the first video code stream includes parameter set identification modification information of the first video, where The parameter set modification identifier information of a video is carried in the parameter set identifier modification message of the first video, so that the splicer sets the parameters of the first video. Modify message to the terminal identification, and to modify the decoded message to the terminal according to the first set of parameters identifying the video and outputs the first video.

A fourth aspect of the embodiments of the present invention provides a decoding apparatus for a digital video code stream, including: a receiving module, configured to receive a first video code stream and a parameter set identifier modification message of the first video, where the parameter set of the first video The identifier modification message includes parameter set identifier ID modification information of the first video, and the first obtaining module is configured to obtain a video for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video received by the receiving module. a parameter set ID of a parameter set of the coding layer VCL NALU; a second acquisition module, configured to acquire a parameter for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU acquired by the first acquisition module And a decoding output module, configured to decode the VCL NALU according to the parameter set for decoding the VCL NALU acquired by the second acquiring module, and output the decoded first video.

With reference to the first implementation manner of the fourth aspect of the embodiment of the present invention, the parameter set ID modification information packet And a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified; the first video code stream includes the VCL for decoding The original parameter set ID of the parameter set of the NALU; the first acquiring module is specifically configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, from the first code stream Determining the original parameter set ID corresponding to the parameter set type of the modified parameter set ID; and offsetting the original parameter set ID corresponding to the original parameter set ID and the parameter set type of the parameter set ID to be modified Obtaining a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

With reference to the second implementation manner of the fourth aspect of the embodiment of the present invention, the parameter set ID modification information includes a parameter set type of a parameter set ID to be modified and a parameter set ID of a parameter set for decoding a VCL NALU, where the first The obtaining module is specifically configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, the use of the parameter set type corresponding to the parameter set ID to be modified from the parameter set ID modification information. The parameter set ID of the parameter set for decoding VCL NALU.

In a third implementation manner of the fourth aspect of the embodiments of the present invention, the parameter set ID modification information is an original parameter set ID of a parameter set type of the parameter set ID to be modified, and the original parameter set ID is used for decoding. a parameter set ID of a parameter set of the VCL NALU; the first obtaining module is configured to: obtain the information according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified received by the receiving module The parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID.

A fifth aspect of the embodiments of the present invention provides a splicing apparatus for a digital video code stream, including: a receiving module, configured to receive information of a first video code stream from an application server, where the first video code stream information includes a first video a parameter set modification information, the sending module, configured to send, according to the first video parameter set modification information received by the receiving module, a first video code stream and a parameter set identifier modification message of the first video, where the first The parameter set identifier modification message of the video carries the parameter set identifier ID modification information of the first video, so that the terminal acquires the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video. a parameter set ID, and the terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal decodes the VCL according to the parameter set for decoding the VCL NALU NALU and output the decoded first video.

A fifth aspect of the embodiments of the present invention provides a splicing apparatus for a digital video code stream, including: The first receiving module is configured to receive the media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the type of the parameter set of the first video and The original parameter set ID of the parameter set corresponding to each type; the second receiving module is configured to receive the media description information of the second video sent by the second content server, where the media description information of the second video carries the parameter of the second video a set, a type of the second video parameter set, and a parameter set ID of the parameter set corresponding to each type; a determining module, configured to determine an original parameter set ID of the first video received by the first receiving module of the same type, The second video parameter set ID received by the second receiving module is the same, and the parameter set identification modification information of the first video is determined; the sending module is configured to send information of the first video stream to the splicer, where the first video code stream is The information includes the parameter set modification information of the first video, and the parameter set modification information of the first video is carried in the parameter set identifier modification message of the first video, so as to facilitate Connector set of parameters identifying the first video modification message sent to the terminal, decode the message and to modify the parameter set according to the terminal identifier of the first video and outputs the first video.

It can be seen from the foregoing embodiment that the parameter set identifier ID modification information of the first video is modified by the parameter set identifier of the first video, so that the terminal decodes and outputs the first video according to the parameter set identifier ID modification information, thereby eliminating the need for a splicer pair. The parameter set ID in each VCL NALU of a video stream is modified, not only when the terminal decodes the first video stream, but also ensures that the correct parameter set is used, and there is no need to add extra on the splicer of the existing network. Hardware resources, or the replacement of splicers in existing networks with new splicers, requires less operational cost than in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. The drawings are some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without any inventive labor.

1 is a network system architecture diagram of a splicing of a typical digital video code stream according to an embodiment of the present invention;

2 is a schematic diagram of relationship between an output video code stream, a second video code stream, and a first video code stream according to an embodiment of the present invention;

3 is a schematic diagram of reference between a VCL NALU and a parameter set of a video code stream complying with the HEVC standard according to an embodiment of the present invention; 4 is a flowchart of a method for decoding a digital video code stream according to an embodiment of the present invention; FIG. 5 is a flowchart of a method for splicing a digital video code stream according to an embodiment of the present invention; A flowchart of a method for splicing a digital video stream; FIG. 7 is a flowchart of another method for splicing a digital video stream according to an embodiment of the present invention; FIG. 8 is a parameter set of a real-time transmission control protocol according to an embodiment of the present invention; Identification modification information request

( RTCP PSIC-R: Real-time Transport Control Protocol Parameter Set ID Change - Request)

FIG. 9 is a schematic structural diagram of feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure;

FIG. 10 is another feedback control information provided by an embodiment of the present invention (FCI: Feedback Control)

Schematic diagram of Information);

FIG. 11 is a schematic structural diagram of still another feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of still another feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure;

13 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention;

14 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention;

FIG. 15 is a schematic structural diagram of a digital video code stream decoding apparatus according to an embodiment of the present invention; FIG. 16 is a schematic structural diagram of a digital video code stream splicing apparatus according to an embodiment of the present invention; FIG. 18 is a schematic structural diagram of a decoding apparatus or a splicing apparatus for a digital video code stream according to an embodiment of the present invention. The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. The embodiments are part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present invention.

In addition, the term "and / or" in this article is merely an association describing the associated object, indicating There may be three relationships, for example, A and/or may represent: Alone, there are three cases of A and B alone. In addition, the character "/" in this article generally means that the contextual object is an "or" relationship. FIG. 1 is a schematic structural diagram of a code stream splicing system according to an embodiment of the present invention, which mainly includes a second encoder, a first encoder, a second content server, a first content server, a splicer, a router, and a terminal. The second encoder is connected to the second content server, and the second content server provides the video code stream output by the second encoder as a second video code stream to the splicer. The first encoder is coupled to the first content server, and the first content server provides the video stream output by the first encoder as a first video stream to the splicer. Splicer: The second video stream and the first video stream are received from the second content server and the first content server, respectively. During the splicing period, the first video code stream is sent to the terminal, and the second video code stream is sent to the terminal at other time periods. In the embodiment of the present invention, the video code stream (including the second video code stream and the first video code stream) sent by the splicer to the terminal is collectively referred to as an output video code stream. FIG. 2 is a schematic diagram showing a relationship between an output video code stream and a second video code stream and a first video code stream according to an embodiment of the present invention. During the splicing period, the output video stream is the first video stream; at other times, the output video stream is the second video stream. The starting point of the splicing period, that is, the point at which the first video stream is outputted, is referred to as a splicing point; the end point of the splicing period, that is, the point at which the end of the output of the first video stream is ended, is referred to as a return point. The splicer is usually located on a physical entity at the edge of the Internet Protocol (IP) network, such as a router or an edge media server. These physical entities are usually not as intelligent as encoders and decoders, such as entropy coding and/or entropy decoding, unlike encoders and decoders. The splicer and content server can be on the same physical entity or on different physical entities. The application server: respectively receiving media description information sent by the second content server and the first content server, and sending media description information for decoding the output video code stream to the terminal (the media description information includes parameters for decoding the received video stream) Set and parameter set ID). The router acts as a device for transmitting routes. The terminal receives the output video code stream sent by the splicer according to the received media description information sent by the application server, and decodes and displays the output video code stream. FIG. 3 is a schematic diagram of reference between a VCL NALU and a parameter set of a video code stream complying with the HEVC standard according to an embodiment of the present invention. As shown in FIG. 3, the video code stream conforms to the HEVC standard, and the VPS, the SPS, the PPS, and the APS respectively include the parameter set ID corresponding to the parameter set, that is, the VPS ID, the SPS ID, the PPS ID, and the APS ID, respectively. PPS and APS are the parameter sets directly referenced by VCL NALU. The parameters contained in PPS and APS can be directly referenced by all code fragments NALU (that is, VCL NALU) of the same picture or pictures. Each VCL NALU contains the PPS ID and APS ID. SPS, VPS is a parameter set indirectly referenced by VCL NALU. The parameters contained in SPS can be referenced by one or more PPSs. Therefore, the PPS also contains the SPS ID. The corresponding VCL NALU indirectly references the SPS through the PPS. The parameters contained in the VPS can be One or more SPS references, so the SPS also contains the VPS ID, and the corresponding VCL NALU indirectly references the SPS through the PPS, and indirectly references the VPS through the SPS. The value of the parameter set ID is an unsigned integer, and most parameter set IDs are entropy encoded, such as unsigned exponential Columbus encoding. The purpose of entropy coding is to improve coding efficiency. The characteristic of entropy coding is that the codeword obtained by coding is variable in length, and the smaller the value, the smaller the number of bits required to represent the value.

When the application server transmits the parameter set transmission to the terminal, it needs to ensure that the decoder located in the terminal receives the parameter set referenced by the VCL NALU before receiving the VCL NALU, including the directly referenced parameter set and the indirectly referenced parameter set, so the parameter Sets are usually transmitted in an out-of-band manner. For example, the parameter set is used as a kind of media description information, which is sent to the terminal before the session is established or during the session establishment process. If the parameter set has changed during the session, the parameter set is sent to the terminal during the session change or through the notification message in the session. There are various encapsulation formats for parameter sets, including but not limited to the parameters of the Session Description Protocol (SDP), and the elements of the Extensible Markup Language (XML). The sending of the parameter set can be based on a variety of protocol methods, including but not limited to Session Initiation Protocol (SIP), Invite, Re-Invite, SIP, and SIP notification. Info), Real Time Streaming Protocol (RTSP) Description (Describe), RTSP Notification (Notify), RTSP Play Notification (Play-Notify), RTSP Set Parameter (Set Parameter), RTSP Get Parameter (Get Parameter) Hypertext Transfer Protocol (HTTP) acquires one of the protocol methods (Get) and HTTP (Post).

FIG. 4 is a schematic diagram of a method for decoding a digital video code stream according to an embodiment of the present invention, including the following steps Step:

Step 401: Receive a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier of the first video includes a parameter set identifier ID modification information of the first video. Specifically, the terminal receives the splicing The first video code stream sent by the device and the parameter set identifier modification message of the first video, the parameter set ID modification information including a parameter set type of the parameter set ID to be modified and a parameter set with the parameter set ID to be modified An offset of the original parameter set ID corresponding to the type; the first video code stream includes an original parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit.

In another optional manner, the terminal receives the first video code stream sent by the splicer and the parameter set identification modification message of the first video, where the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified, and The parameter set ID of the parameter set for decoding VCL NALU.

In another optional manner, the terminal receives the first video code stream sent by the splicer and the parameter set identifier modification message of the first video, where the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified. The original parameter set ID, the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID.

The parameter set identification modification message of the first video is an SEI message generated by a splicer and/or in a manner of generating a supplemental enhanced SEI message, specifically a parameter set identifier modification supplementary enhanced SB message, and an adaptive parameter set identifier modification The SEI message, the picture parameter set identifier modifies the SEI message. In addition, the SEI message may also be another type of SEI message, which is not limited herein. The parameter set identification modification message of the first video is sent to the terminal together with the first video code stream.

Step 402: Acquire a parameter set ID for decoding a parameter set of the VCL NALU according to the parameter set ID modification information of the first video (that is, a network abstraction layer unit VCL for decoding a video coding layer in the first video code stream. The parameter set ID of the parameter set of the NALU, that is, the modified parameter set ID. Unless otherwise stated in the present application, the modified parameter set ID refers to the video coding layer used for decoding the first video code stream. The parameter set ID of the parameter set of the network abstraction layer unit VCL NALU, that is, the parameter set ID of the parameter set used to decode the VCL NALU);

Specifically, the terminal acquires an original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified from the first video code stream according to the parameter set type of the parameter set ID to be modified; according to the original parameter The set ID and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified acquire the VCL corresponding to the parameter set type of the parameter set ID to be modified. The parameter set ID of the NALU's parameter set.

As another alternative manner, when the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and a parameter set ID of a parameter set for decoding the VCL NALU, the terminal directly according to the parameter set to be modified. The parameter set type of the ID obtains, from the parameter set ID modification information, a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

In another optional manner, when the parameter set ID modification information includes the original parameter set ID of the parameter set type of the parameter set ID to be modified, and the decoding corresponding to the original parameter set ID, the VCL NALU is decoded. Obtaining, according to the parameter set type of the parameter set ID to be modified, the terminal, according to the parameter set type of the parameter set ID to be modified, the decoding corresponding to the parameter set type of the parameter set ID to be modified in the first video code stream for decoding The parameter set ID of the parameter set of the VCL NALU; the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID is obtained according to the original parameter set ID. For the implementation, if the parameter set type of the parameter set ID to be modified includes at least one original parameter set ID, the parameter set type of the parameter set ID to be modified needs to be respectively corresponding to each original parameter set ID. The parameter set ID of the parameter set used to decode the VCL NALU.

Further, when the parameter set ID modification information includes the original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter set of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID The ID of the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID is obtained by the terminal according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.

Step 403: Acquire a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.

Specifically, the terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.

Step 404: Decode the VCL according to the parameter set used to decode the VCL NALU.

NALU and output the decoded first video.

Specifically, the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.

As another optional manner, the first video parameter set ID modification information may also be a parameter set type of the parameter set ID to be modified, and the terminal sends a real-time transmission control protocol RTCP request to the splicer. The RTCP request includes the parameter set type of the parameter set ID to be modified; the terminal receives the RTCP response returned by the splicer, and the RTCP response carries the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. Offset, or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the original parameter set ID of the parameter set type of the parameter set ID to be modified And a parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID; the acquiring the parameter of the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video The set ID is specifically: according to the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified carried in the RTCP response or the parameter set type corresponding to the parameter set ID to be modified, for decoding The parameter set ID of the parameter set of the VCL NALU, or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID according to the original parameter set ID of the parameter set type of the parameter set ID to be modified, Parameter set ID Change information acquiring the ID to be modified parameter set parameter set corresponding to the type of parameter sets a parameter set for decoding VCL NALU's ID.

In another optional manner, the parameter set ID modification information further includes a modification effective flag, and before the receiving the first video code stream and the parameter set identification modification message of the first video, the method further includes: the receiving is not effective. Determining, by the parameter set of the first video, a modification message; determining that the parameter set identifier modification message of the first video that is not in effect includes flag information indicating that the parameter set ID modification information is not valid; and sending a second media description information acquisition request to the application server; And receiving, by the application server, a second media description information acquisition response, where the second media description information acquisition response carries the modified parameter set of the first video, where the parameter set ID of the parameter set in the first video modification is The parameter set ID of the parameter set used to decode the VCL NALU.

In another optional manner, before the acquiring the parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, the method further includes: acquiring the first video by using an outband manner. The modified parameter set, the parameter set ID of the modified parameter set is a parameter set ID of a parameter set for decoding the VCL NALU; or, the original parameter set of the first video is obtained by an out-of-band method and used for decoding a parameter set ID of a parameter set of the VCL NALU, the original parameter set of the first video carries the original parameter set ID; and the original parameter set ID is modified according to the parameter set ID of the parameter set for decoding the VCL NALU.

As another optional manner, the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and the information acquisition is modified according to the parameter set ID of the first video. The parameter set ID of the parameter set for decoding the VCL NALU is specifically: obtaining a parameter set ID of a parameter set for directly decoding a VCL NALU directly referenced by the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, the method further comprising Obtaining, according to the obtained parameter set ID of the parameter set directly used by the VCL NALU for decoding the VCL NALU, a parameter set directly referenced by the VCL NALU in the first video code stream; obtaining the decoding from the directly referenced parameter set The parameter set ID of the parameter set indirectly referenced by VCL NALU of VCL NALU; or

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU and a parameter set type indirectly referenced by the VCL NALU, and the parameter information is modified according to the parameter set ID of the first video. The parameter set ID of the parameter set for decoding the VCL NALU is specifically: acquiring the VCL NALU for decoding the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU according to the parameter set ID modification information of the first video, respectively. a parameter set ID of a directly referenced parameter set, and a parameter set ID of a parameter set for decoding a VCL NALU indirect reference of the VCL NALU corresponding to the parameter set type indirectly referenced by the VCL NALU; Obtaining the parameter set ID for decoding the VCL NALU in the parameter set ID of the parameter set of the decoded VCL NALU is specifically: acquiring the parameter set ID according to the obtained directly referenced parameter set and the parameter set ID of the indirectly referenced parameter set a directly referenced parameter set of the VCL NALU and a parameter set of the indirect reference of the VCL NALU;

It can be seen from the foregoing embodiment that the parameter set identifier ID modification information of the first video is modified by the parameter set identifier of the first video, so that the terminal decodes and outputs the first video according to the parameter set identifier ID modification information instead of directly according to the first video. The original parameter set ID in the code stream obtains a parameter set of the VCL NALU that decodes the first video code, thereby eliminating the need for a splicer for each VCL of the first video stream.

The parameter set ID in the NALU is modified, not only when the terminal decodes the first video stream, but also ensures that the correct parameter set is used, and there is no need to add additional hardware resources to the splicer of the existing network, or use a new one. The splicer replaces the splicer in the existing network with less operational cost than is required in the prior art.

FIG. 5 is a flowchart of a method for splicing a code stream according to an embodiment of the present invention, including the following steps: Step 501: The second content server receives the second video code stream and the second video parameter set.

Before performing this step, the second encoder encodes the second video content, and the parameter set ID can be obtained by entropy coding. And transmitting the outputted second video code stream and the acquired parameter set of the second video to the second content server.

Step 502: The first content server receives the first video code stream and the first video parameter set.

Before performing this step, the first encoder encodes the first video content, and the parameter set ID may be acquired by entropy coding, and the output first video code stream and the acquired parameter set ID are included as the original parameter set ID. The original parameters of the first video are sent to the first content server.

It should be noted that, in the embodiment of the present invention, there is no temporal sequence between step 501 and step 502.

Step 503: The terminal sends a first media description information acquisition request message to the application server, where the first media description information acquisition request message includes a content identifier of the second content.

Step 504: The application server acquires media description information of the second content according to the content identifier of the second content. Including but not limited to the following information: Parameter set of the second video The type of the second video parameter set and the parameter set ID of the parameter set corresponding to each type.

An optional manner for the application server to obtain the media description information of the second content is: after the application server receives the media description information acquisition request message sent by the terminal, determining, according to the content identifier of the second content, that the content server where the second content is located is the second The content server, and sending a media description information acquisition request message to the second content server. The media description information acquisition request message in step 503 is sent during the session establishment process, that is, a session establishment request message, specifically, a SIP Invite request message. Correspondingly, the media description information acquisition request message sent by the application server to the second content server is also a session establishment request message.

The second content server receives the media description information acquisition request message, determines the second video code stream, the parameter set NALU of the second video according to the content identifier of the second content, and allocates the second video code stream identification information for the second video code stream. Send a media description information acquisition response message to the application server. Specifically, the media description information acquisition response message is a session establishment response message corresponding to the foregoing session establishment request message, such as a SIP 200 response message. The media description information acquisition response message includes a parameter set of the second video and second video code stream identification information. After receiving the media description information acquisition response message, the application server obtains and stores the parameter set of the second video carried in the media description information acquisition response message. Specifically, the SDP is included in the media description information acquisition response message, and the parameter set NALU of the second video is included in the SDP. Of course, the information in the parameter set NALU can be represented in another manner, such as using an SDP parameter or an XML element. The embodiment of the present invention is not limited herein. The SDP may further include a source IP address, a port, and a synchronization source identifier (S SRC Synchronization Source) of a real time transport protocol (RTP: Real Time Transport) stream carrying the second video stream, where the RTP stream carrying the second video stream is included. The source IP address, port, and SSRC are the second video stream identification information. The specific method of including the parameter set NALU in the SDP is the prior art, and the embodiments of the present invention are not described in detail herein.

As an alternative manner for the application server to obtain the media description information of the second content, the media description acquisition request message in step 503 may also be used as an initialization media description information acquisition request message (such as an RTSP Describe request message, before the session is established). HTTP Get request message, etc.) is sent. Correspondingly, the media description information acquisition request message sent by the application server to the second content server is an initialization media description information acquisition request message, and the second content server receives the initialization media description information acquisition request message and sends the second video to the application server. The media description information acquisition response message of the media description information is an initialization media description information acquisition response message. The initial media description information acquisition response message may be an RTSP 200 response message, an HTTP 200 response message, or the like.

A further optional way for the application server to obtain the media description information of the second content is that the media description acquisition request message in step 503 is sent before the session is established, and before the step 504, the second content server is the second video code stream. The second video code stream identification information is allocated, and the content identifier of the second content and the media description information of the second content are sent to the application server. The media description information of the second content includes a parameter set of the second video and second video code stream identification information. After receiving the media description acquisition request message sent by the terminal, the application server obtains the parameter set of the second video and the second video code stream identification information from the received media description information according to the content identifier of the second content.

In addition, the application server may obtain the media description information of the second content from the second content server in another manner, which is not described in detail herein.

Step 505: The application server acquires media description information of the first content. Including but not limited to the following information: the original parameter set of the first video and the acquisition information of the first video.

An optional manner for the application server to obtain the media description information of the first content is that the application server sends the first video media to the first content server according to the configured address of the first content server. Describe the information acquisition request message.

The first content server receives the first video media description information acquisition request message sent by the application server, and sends the media description information of the first video to the application server, where the media description information of the first video includes at least the original parameter set of the first video and The information of the video stream, the type of the parameter set of the first video, and the original parameter set ID of the parameter set corresponding to each type. The information of the first video stream may be used to obtain the first video from the first content server. The RTSP uniform resource locator (URL: Uniform Resource Locator) of the code stream.

As an optional manner, in the embodiment of the present invention, the terminal may further include the identifier of the user in the request message for sending the media description information sent to the application server in step 503, and the media sent by the application server to the first content server. The obtaining request message of the description information carries the identifier of the user, so that the first content server can determine the first content for the user according to the identifier of the user, and determine the parameter set of the first video and the obtained information of the first video code stream. Sent to the application server.

Step 506: The application server modifies the original parameter set ID of the first video, where the original parameter set ID of the first video is the parameter set ID included in the original parameter set of the first video in step 505.

Specifically, after receiving the original parameter set of the first video, the application server determines, for each type of parameter set ID, whether the original parameter set ID of the first video of the type is the parameter set ID of the second video of the type. The same, if the same, you need to modify the original parameter set ID of the first video of this type. This process is described by taking the same original parameter set ID of the first video of the type and the parameter set ID of the second video of the type as an example.

Then, the application server determines the offset of the original parameter set ID of the first video of the type, and adds the value of the original parameter set ID of the first video of the type plus the offset of the original parameter set ID as the first of the type. The parameter set ID of the video for decoding the parameter set of the VCL NALU; or the application server directly assigns a new value to the original parameter set ID of the first video of the type, the new value of the allocation is the same as all the same type The parameter set ID of the two videos is different, and the original parameter set IDs of all other first videos of the same type are also different, and the assigned new value is used as the parameter set ID of the parameter set for decoding the VCL NALU (ie, Decoding the parameter set ID of the parameter set of the VCL NALU in the first video bitstream).

Specifically, the application server determines whether the original parameter set ID of the first video of the type is the same as the parameter set ID of the second video of the type, and the application server is the first type of the type. The original parameter set ID of the video is compared to the parameter set ID of all second videos of the type, if the original parameter set ID of the first video of the type is the same as the parameter set ID of one of the second videos of the type. The application server determines that the original parameter set ID of the first video of the type is the same as the parameter set ID of the second video of the type.

Specifically, an optional manner for the application server to determine an offset of the original parameter set ID of the first video of the certain type is: the application server determines a maximum value of all parameter IDs of the second video of the type, all the types The minimum value in the original parameter set ID of the first video, the difference between the maximum value of the parameter set IDs of all the second videos and the minimum value of the original parameter set IDs of all the first videos is calculated, and the application server compares the difference The sum of the value and a positive integer is the offset of the original parameter set ID of the first video of that type. Specifically, as an alternative, the positive integer is 1. The offset can be applied to the original parameter set ID of the first video of the type, and the original parameter set ID of the first video of the different types can be set to different offsets, which is not limited herein.

Step 507: The application server sends a first media description information acquisition response to the terminal, where the parameter set of the second video and the parameter set of the first video are included. Specifically, the SDP is included in the first media description information acquisition response message, and the parameter set of the second video is included in the SDP (may be a set of parameter encapsulation parameters of the NALU, and may also be other forms, and the implementation of the present invention includes no special It is indicated that NALU is only used as an optional example and does not limit the encapsulation form of the parameter set), the modified parameter set of the first video. The parameter set ID of the modified parameter set of the first video is the parameter set ID of the parameter set of the first video for decoding the VCL NALU, specifically, the value of the original parameter set ID of the first video plus the original parameter set. The offset of the ID, or the value assigned by the application server directly to the parameter set ID of the parameter set of the first video used to decode the VCL NALU.

Steps 501-507 In the method for the terminal to acquire the second video parameter set and the first video parameter set, the terminal is a parameter set obtained from outside the video code stream, and thus may be referred to as an out-of-band mode (or called an external mode). Steps 501-507: The method for the terminal to acquire the second video parameter set and the first video parameter set is only an optional manner in which the terminal acquires the second video parameter set and the first video parameter set in an outband manner, and the terminal may also adopt another The method obtains the second video parameter set and the first video parameter set in an out-of-band manner, and the embodiment of the present invention is no longer detailed.

Step 508: The application server sends the second video code stream information and the first video code stream information to the splicer. Specifically, the second video code stream information includes the second video code stream identification information, where the first video code stream information includes: the first video code stream acquisition information and the first video parameter set ID modification information (eg, The parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or the parameter set type of the parameter set ID to be modified and the application server to be modified The parameter set ID of the parameter set for decoding the VCL NALU assigned by the parameter set ID, or the original parameter set ID of the parameter set type of the parameter set ID to be modified, and the parameter set for decoding the VCL NALU corresponding to the original parameter set ID The parameter set ID (that is, the parameter set ID of the parameter set for decoding the VCL NALU allocated by the application server for the parameter set ID to be modified), and the parameter set ID of the parameter set for decoding the VCL NALU is The terminal is configured to decode a parameter set ID of a parameter set of the VCL NALU in the first video code stream after receiving the first video code stream. The parameter set ID modification information of the first video is the parameter set ID modification information of the parameter set type directly referenced by the VCL NALU of the first video, that is, the parameter set type of the parameter set ID of the first video to be modified can only be The parameter set type directly referenced by VCL NALU.

Step 508 is an example in which the parameter set modification information of the first video includes the parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. The acquiring information of the first video code stream may be an RTSP URL for acquiring the first video code stream to the first content server, where the second video code stream identification information may be an RTP stream for carrying the second video code stream. The combination of the source IP address, the port, and the SSRC; the parameter set type of the parameter set ID of the first video to be modified may be a parameter set type represented by a three-byte text form, for example, vps,,, "sps,,," pps ,,, "aps" respectively represent the parameter set types as VPS, SPS, PPS, and APS. The parameter set ID offset can be a 1-byte unsigned integer. The second video stream and the first video stream information can be as shown in Table 1 (ie, the video configuration information table):

Table 1

The source IP address in the second video code stream identification information is 219.133.0.1, the port is 49170, and the SSRC is 1234. The information about the first video stream corresponding to the second video stream is rtsp: 〃 example. Com/fizzle/foo; The application server determines the VPS, SPS, PPS in the first video, The ID of the APS needs to be modified, and the offset of the original parameter set ID of each type of the first video is 1, that is, when the parameter set ID of the first video parameter set of the corresponding type is modified, all the original parameter set IDs are The value is incremented by 1 to the value of the parameter set ID of the corresponding parameter set for decoding the VCL NALU. However, the parameter set type directly referenced by VCL NALU only contains PPS and APS. Therefore, Table 1 only contains 2 parameter set ID modification information, that is, parameter set ID modification information of PPS and parameter set ID modification information of APS. It should be noted that the offset of the original parameter set ID in the above example is 1, for the sake of simplification of the example, and does not mean that the parameter set ID offsets of different types of parameter sets are the same.

It is worth noting that steps 507 and 508 have no prioritization in time.

Step 509: The second content server sends a second video code stream to the splicer.

Specifically, if the splicer is located at the edge router, the destination IP address of the RTP stream carrying the second video stream, the port is a multicast IP address, and the port, and the RTP stream carrying the second video stream is sent to the splicing based on the network routing policy. Before step 509, the second content server acquires a multicast IP address and port allocated by the network management system for the second video code stream.

Or, if the splicer is located at the edge server, the destination IP address of the RTP stream carrying the second video stream, the port is the IP address of the splicer, and the port. Before step 509, the second content server obtains the IP address and port of the splicer. The session establishment request message sent by the application server to the second content server in step 504 is taken as an example for description. Specifically, the second content server obtains the IP address and port of the splicer from the session establishment request message sent by the application server to the second content server in step 504. In this way, how the application server obtains the IP address and port of the splicer and how to establish the IP address of the splicer, the IP address of the port and the terminal, and the correspondence between the ports are well-known technologies, and will not be described again.

Step 510: The splicer sends the second video code stream as an output video code stream to the terminal.

If the splicer receives the destination IP address of the second video stream, the port is the IP address of the splicer, and the port. The splicer changes the destination IP address and port 中 in all IP packets carrying the second video stream to the IP address and port of the corresponding terminal.

Step 511: The terminal acquires VCL NALU from a parameter set obtained in an out-of-band manner (the out-of-band manner is from the second video stream) according to the parameter set ID in the VCL NALU of the received second video code stream. The referenced parameter set (ie, the parameter set of the VCL NALU used to decode the second video stream) is used to decode and output the second video using the obtained parameter set.

Step 512: The splicer according to the second video code stream identification information from the video configuration information table, that is, Table 1 The obtaining information of the first video code stream is found, and the first video code stream is obtained according to the obtaining information of the first video code stream.

Specifically, before performing this step, the splicer determines whether splicing preparation is required. If splicing preparation is required, the splicer acquires the first video code stream from the first content server.

An optional manner for determining whether the splicing preparation is required is: the media description information of the second content obtained by the application server in step 504, and the second video code stream information sent by the application server to the splicer in step 508 further includes a splicing point. The time information is determined by the splicer according to the pre-configured preparation threshold (the interval between the time point for starting the splicing preparation and the splicing point), whether the splicing preparation is required, that is, whether the first video code stream is obtained from the first content server.

Specifically, the splicer sets the time for starting the splicing preparation to be the time information of the splicing point minus the preparation threshold. If the current time has reached the time for starting the splicing preparation, the splicer needs to perform the splicing preparation. In this embodiment, the format of the time information may be hh:mm:ss.fff, where hh represents hours, mm represents minutes, ss represents seconds, and fff represents milliseconds. Prepare the threshold in milliseconds. For example, the time information of the splicing point is 01:01:01.001 for 1 hour, 1 minute, 1 second, and 1 millisecond. The preparation threshold is 500, which means 500 milliseconds. The time to start the stitching preparation is 01:01:00.499 for 1 hour, 1 minute, and 499 milliseconds. If the current time is equal to or later than 01:01:00.499, the splicing preparation is required to acquire the first video code stream from the first content server.

Another alternative method for determining whether the splicing preparation is required is that the second content server sets the time for starting the splicing preparation. If the current time has reached the time for starting the splicing preparation, the RTP belongs to the RTP stream that carries the second video stream. The packet header is inserted into the repare tag to instruct the splicer to prepare for splicing. The splicer checks whether the RTP header corresponding to the second video stream contains a prepare tag. If yes, the splicing preparation is required to obtain the first video stream from the first content server.

Step 513: The splicer generates a parameter set identifier ID modification message of the first video, where the first video parameter set ID modification information is included. Specifically, the parameter set identifier modification message may be a Parameter Set ID Change Info SEI message, or a PPS ID Change Info SEI message and/or an APS identifier modification information SEI. The APS ID Change Info SEI message, where the parameter set identification modification information SEI message may carry one or more modification information of the parameter set type of the parameter set ID to be modified, and the PPS identification modification information SEI message includes only the parameter to be modified. The parameter set type of the set ID is the repair of the PPS The information modification, the APS identification modification information SEI message includes only the modification information of the parameter set type of the parameter set ID to be modified as APS.

Specifically, before performing this step, the splicer determines whether the video configuration information table (ie, Table 1) includes the first video parameter set ID modification information corresponding to the second video code stream identification information, such as the parameter set type of the ID to be modified. And a parameter set ID offset corresponding to the parameter set type of the modified ID.

If the video configuration information table includes the first video parameter set ID modification information corresponding to the second video code stream identification information, the splicer generates a parameter set identifier modification message of the first video (specifically, the splicer adopts a method for generating an SEI message) The SEI message is generated, such as the parameter set identifier modification supplementary enhancement information SEI message, and the adaptive parameter set identifier modification SEI message or the picture parameter set identifier modification SEI message may also be another type of SEI message, which is not limited herein. ). The parameter set type of the parameter set ID to be modified is the parameter set ID modification information of the first video of the parameter set type directly referenced by the VCL NALU.

This is because, in step 507, the parameter set ID of the modified parameter set of the first video sent by the application server to the terminal in the outband manner is the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal obtains the VCL NALU directly. After the referenced parameter set, the parameter set ID of the parameter set indirectly referenced by the VCL NALU in the parameter set directly referenced by the VCL NALU is the parameter set ID of the parameter set used to decode the VCL NALU. Therefore, the parameter set ID modification information of the parameter set type indirectly referenced by the VCL NALU by the application server is not sent to the splicer, and is sent to the terminal by the splicer through the parameter set ID modification message. In this step, the parameter set modification information of the first video includes the parameter set type of the parameter set ID of the video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.

The method for generating the parameter set identifier modification information SEI message as the parameter set identifier modification message may be:

The splicer obtains the first video parameter set ID modification information from the video configuration information table, and includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. At this time, the first video parameter set type of the parameter set ID to be modified only contains the parameter set type directly referenced by the VCL NALU. The splicer generates the parameter set identification modification information SEI message according to the obtained first video parameter set ID modification information. In this step, the first encoder is an encoder that complies with the HEVC standard, that is, the parameter set type directly referenced by the VCL NALU includes PPS and APS as an example. Specifically, when the first video parameter set type of the ID to be modified includes the PPS and/or the APS, the splicer generates the parameter set identifier modification information SEI message. The parameter set identification modification information SEI message includes two flags (flag): a picture parameter set identification change flag (pps_id_change_flag), and an adaptive parameter set identification change flag (aps_id_change_flag), which are respectively used to indicate the parameter set ID of the corresponding type parameter set ( That is, whether the PPS ID and APS ID need to be modified. If the parameter set ID of the corresponding type parameter set needs to be falsified, the parameters ps id change flag and aps id change flag have a value of 1. Certainly, other manners, such as the character, the T, the string URE, and the like, may be used to indicate that the parameter set ID of the corresponding type parameter set needs to be modified, which is not limited herein. In addition, the parameter set identifier modification information SEI message further includes an offset of the original parameter set ID of the parameter set type of the parameter set ID to be modified. If the ID of the corresponding type parameter set needs to be modified, the values of the parameter ps id change flag and the aps id change flag are equal to 1. The format of the parameter set identification modification information SEI message is as shown in Table 2:

As shown in Table 2, the parameter aps id change flag has a value of 1, indicating that the APS ID needs to be modified. The value of the parameter ps id change flag is 1, indicating that the PPS ID needs to be modified. The descriptor ( Descirptor ) u(l) indicates that the parameters aps id change flag and ps id change flag occupy one bit. The parameter aps_id_diff (adaptive parameter set identifier offset) carries the offset of the original parameter set ID corresponding to the APS, and the parameter ps_id_diff (picture parameter set identifier offset) carries the offset of the original parameter set ID corresponding to the PPS. u(5) indicates an unsigned integer represented by 5 bits, and u(8) indicates an unsigned integer represented by 8 bits.

The PPS identification modification information SEI message and the APS identification modification information SEI message may be separately generated as the parameter set identification modification message for the change of the PPS ID and the change of the APS ID. Specifically, the PPS identifier modification information SEI message carries the offset of the original parameter set ID corresponding to the PPS, and the message format is as follows: Table 3 shows:

Table 3

The parameter pps_id_diff carries the offset of the original parameter set ID corresponding to the PPS.

The APS identification modification information SEI message carries the offset of the corresponding original parameter set ID, and its message format: ^ Table 4:

Table 4

The parameter aps_id_diff carries the offset of the original parameter set ID corresponding to the APS.

Generally, if the parameter set type of the first video of the parameter set ID to be modified includes the PPS and does not include the APS, the splicer only generates the PPS identification modification information SEI message; if the first video parameter set type of the ID needs to be modified, APS, excluding PPS, the splicer only generates the APS identification modification information SEI message; if the first video parameter set type of the ID needs to be modified to include both PPS and APS, the splicer can separately generate the PPS identification modification information SEI message and the APS identification modification. Information SEI message.

Of course, the first video parameter set type that needs to be modified ID includes both PPS and APS, and the splicing can generate a parameter set identifier modification information SEI message, and the parameter set identification information SEI message includes PPS and APS modification information.

Different SEI messages are distinguished by the PayloadType in the message, so the SEI message needs to be assigned a unique PayloadType value. Step 514: The splicer sends a parameter set identifier modification message (such as a parameter set identifier modification information SEI message) of the first video to the terminal, and the first video code stream, that is, the splicer modifies the parameter set identification message of the first video with the first The video stream is sent to the terminal together.

Before performing the step 514, the splicer further determines whether the splicing point is reached, and sends the parameter set identification modification message and the first video code stream to the terminal when the splicing point is reached. Specifically, an optional manner of determining whether the splicing point is reached is: the splicer determines whether the current time of outputting the second video code stream has reached the starting point of the splicing point period, and if the starting point of the splicing period has been reached, the splicing point is considered to be reached. . Another alternative way of determining whether the splicing point is reached is that the second content server inserts a slice tag into the header of the RTP packet corresponding to the second video stream before the splicing point and specifies a distance splicing point in the slice tag. At the time, the splicer checks whether the RTP header corresponding to the second video stream contains a slice tag and the time at which the splicing point is specified is 0. If it is included, it indicates that the splicing point is reached.

To ensure random access, the parameter set identification modification message is sent at least with the random access point (RAP: Random Access Point) and applied to all pictures between the associated RAP and the next RAP. The RAP is a picture in the middle of the code stream that only uses intra prediction, and the decoder can start the decoding process from the picture. When the parameter set identification modification message is sent with the RAP, the splicer inserts the NALU encapsulated with the parameter set identification modification message into the associated RAP AU separator NALU, before the VCL NALU in the first video code stream.

Step 515: After receiving the parameter set identifier modification message (such as the parameter set identifier modification information SEI message) of the first video and the first video code stream, the terminal modifies the message according to the parameter set identifier of the first video (such as the parameter set identifier modification information SEI). Message) Determine the parameter set ID of the parameter set referenced by the VCL NALU in the first video stream (ie, the parameter set ID of the parameter set used to decode the VCL NALU). The parameter set whose parameter set ID is the determined parameter set ID is obtained from the parameter set obtained by the out-of-band method according to the determined parameter set ID, that is, the parameter set of the VCL NALU used for decoding the first video code stream.

In this step, the parameter set identifier modification message of the first video is used as the parameter set identifier modification information SEI message, and the parameter set type directly referenced by the VCL NALU includes PPS and APS as an example for description. Modifying information according to the parameter set identifier The SEI message determines the parameter set ID of the parameter set referenced by the VCL NALU and the method of the parameter set may be:

The terminal first identifies the modification information according to the parameter set identifier. The SEI message determines the parameter set ID of the parameter set directly referenced by the VCL NALU, and then obtains the parameter set directly referenced by the VCL NALU from the parameter set obtained by the out-of-band method according to the parameter set ID.

Specifically, the terminal confirms that the parameter set identification information SEI message carries the APS ID change flag, such as the parameter aps id change flag value in the message is 1, or other identifier. Obtain the APS ID from the received VCL NALU of the first video stream, that is, the original parameter set ID of the adaptive parameter set referenced by the VCL NALU (ie, the original aps id of the aps id), and according to the APS ID. Original The value and the parameter set identification modification information SEI message offset from the original parameter set ID corresponding to the APS calculates a new value (ie, the value of the parameter set ID used to decode the parameter set of the VCL NALU). Specifically, the original value + the offset of the original parameter set ID corresponding to the APS, that is, new_aps_id = original aps id + aps id diff, takes a new value, and uses the new aps id as a reference_aps_id (reference adaptive The value of the parameter set identifier is obtained, and then the APS including the aps_id which is the same as the reference_aps_id (that is, the same as the new_aps_id) is obtained from the parameter set obtained by the outband method, that is, the parameter set for decoding the VCL NALU in the first video bitstream. .

In addition, the parameter set for decoding the VCL NALU in the first video bitstream from the parameter set obtained by the outband mode according to the value of the reference adaptive parameter set identifier is specifically:

The terminal obtains the parameter set NALU from the media description information obtained in step 507. according to

The nal_unit_type in the NALU header determines the type of the parameter set in the NALU. Different types of parameter sets have different formats, and the parameter set type must be determined before the parameter set can be parsed according to the corresponding format. If the type of the parameter set is APS, get the aps id in it. If the asp_id is equal to the reference aps id, that is, the parameter set corresponding to the asp_id is the APS for decoding the VCL NALU in the first video bitstream; otherwise, the next parameter set NALU is processed until the aps_id is equal to the reference aps id. APS.

Further, the terminal confirms the parameter set identification modification information, and the SEI message carries the PPS ID change flag, and obtains the offset of the original parameter set ID corresponding to the PPS from the parameter set identifier modification information SEI message. The terminal obtains the original parameter set ID of the PPS referenced by the VCL NALU from the VCL NALU of the received first video stream, and calculates a new value according to the original parameter set ID and the offset of the original parameter set ID corresponding to the PPS. And taking the new value as the value of the modified PPS ID (reference picture parameter set identifier), and obtaining the PPS of the modified PPS ID obtained from the parameter set NALU in step 507.

The terminal then obtains the parameter set indirectly referenced by VCL NALU, such as SPS and VPS, according to the parameter set ID indirectly referenced by VCL NALU contained in the parameter set directly referenced by VCL NALU.

Specifically, the SPS including the same sps id as the sps id in the PPS is obtained from the parameter set NALU obtained in step 507 according to the sps_id in the PPS; and the parameter set NALU is obtained from the step 507 according to the vps_id in the SPS. The VPS of the same vps id as the vps id in the SPS.

Both the parameter set directly referenced by the VCL NALU and the parameter set indirectly referenced by the VCL NALU serve as a parameter set for decoding the VCL NALU of the first video stream. As an alternative, if the terminal does not generate the parameter set identification modification information SEI message in step 513, but generates the PPS identification modification information SEI message and the APS identification modification information SEI message respectively, then in step 515, The terminal determines a parameter set ID of the PPS referenced by the VCL NALU according to the PPS identifier modification information SEI message, and determines a parameter set ID of the APS referenced by the VCL NALU according to the APS identifier modification information SEI message, and according to the determined parameter set ID of the PPS And the parameter set ID of the APS obtains the parameter set ID of the PPS and the PPS and APS of the parameter set ID of the APS from the parameter set obtained by the outband mode, and acquires the SPS according to the sps id in the PPS; according to the SPS The vps_id in the middle gets the VPS.

The specific acquisition method is the same as the method for obtaining the SEI message according to the parameter set identification modification information. The embodiment of the present invention is not described in detail herein.

Step 516: Using a parameter set obtained from a parameter set obtained by an out-of-band method (for example, using the first encoder as an encoder conforming to the HEVC standard, including APS, PPS, SPS, and VPS) to the first video stream. The VCL NALU in the decoding is performed.

It can be seen from the above embodiment that when the parameter set ID of the parameter set directly referenced by the VCL NALU of the first video of the certain type is the same as the parameter set ID of the parameter set of the same type of the second video, the splicer only needs to generate the parameter set identifier. The modification message, such as (parameter set identification modification information SEI message, or PPS identification modification information SEI message and/or APS identification modification information SEI message). The parameter set identifier modification information carries the parameter set ID modification information of the first video, and the terminal determines the parameter set ID of the parameter set used for decoding the VCL NALU according to the parameter set identifier modification message, without directly modifying the first video stream by using the splicer. The parameter set ID in . Since the operation of generating the parameter set identification modification message is relatively simple, the splicer does not need entropy decoding and entropy coding capability and the operation frequency is low, so only a small amount of computing resources and storage resources are needed. This requires only a simple upgrade of the splicer in the existing network, without the need to add additional hardware resources to the splicer in the existing network, or to replace the splicer in the existing network with a new splicer. Therefore, operators need less operating costs than the prior art.

In the process corresponding to FIG. 5, the application server modifies the first video parameter set parameter set ID in step 506, and then transmits the first video parameter set including the modified parameter set ID to the terminal in step 507. However, as an alternative, the application server may not modify the parameter set ID of the first video parameter set, and the media description information acquisition response sent to the terminal in step 507 indicates that the first video parameter set is used for The parameter set ID of the parameter set that decodes the VCL NALU. That is, the media description information acquisition response sent by the application server to the terminal includes a parameter set of the second video, an original parameter set of the first video, and an original parameter set of the first video for decoding the VCL.

The parameter set ID of the NALU parameter set, where the original parameter set of the first video is the application server in the step

The parameter set of the first video obtained in 504, thereby reducing the operation of the application server.

The media description information sent to the terminal in step 507 is taken as an SDP format as an example. Specifically, the application server obtains a set of sub-parameters of the parameter sprop-parameter-sets (flow characteristic parameter set) in the SDP in the media description information in the media description information: <parameter set id>: <parameter set data > to carry a parameter set NALUc where: parameter set type is a three-byte text representing the type of the parameter set. The parameter set id is a 1-byte unsigned integer representing the value of the parameter set ID. The value of this field can be different from the value of the original parameter set ID in the parameter set NALU. If the value of the parameter set id is different from the value of the original parameter set ID in the parameter set NALU, the value of the parameter set id field is used as the parameter set ID of the parameter set used to decode the VCL NALU. The original parameter set

The ID is the parameter set ID in the parameter set NALU sent by the first content server to the application server in step 502, that is, the parameter set ID of the first video that has not been modified. When a set of sub-parameters is used to carry the parameter set NALU of the first video whose parameter set ID needs to be modified, the parameter set id is the parameter set ID of the parameter set that needs to be used to decode the VCL NALU.

The parameter set data is the Base64 (basic 64 character) encoded parameter set NALU. When a set of sub-parameters is used to carry a parameter set NALU of the first video whose parameter set ID needs to be modified, the carried parameter set NALU is a parameter set NALU sent by the first content server to the application server.

The parameter sprop-parameter-sets can contain multiple sets of sub-parameters to carry multiple parameter sets NALU. If parameter-sets contains multiple sets of sub-parameters to carry multiple parameter sets NALU, each set of sub-parameters is separated by the character ",".

An example in which the second encoder and the first encoder are both encoders conforming to the HEVC standard will be described. In the above manner, in step 507, the SDP example of the parameter set NALU sent by the application server to the terminal is as follows:

Specifically, it is assumed that the second video parameter set NALU received by the application server includes a VPS NALU, an SPS NALU, a PPS NALU, and an APS NALU, corresponding parameter sets. The VPS ID, SPS ID, PPS ID, and APS ID in the NALU are both 0. The first video parameter set NALU received by the application server also includes a VSP NALU, an SPS NALU, a PPS NALU and an APS NALU, and the VPS ID, SPS ID, PPS ID, and APS ID in the corresponding parameter set NALU are also 0. .

Therefore, all types of original parameter set IDs of the first video need to be modified because they are the same as the same type of parameter set ID of the second video, and the application server determines that the offset of the parameter set ID corresponding to each type of parameter set is 1, That is, the new values of the VPS ID, the SPS ID, the PPS ID, and the APS ID of the first video VSP, SPS, PPS, and APS are all the offsets of the original parameter set ID + parameter set ID of all types of the first video, that is, 0+1. =1. Therefore, the SDP carries eight parameter sets NALU, and the first to fourth sets of sub-parameters carry the parameter set NALU of the second video, and the corresponding parameter set types are VPS, SPS, PPS, APS; the parameter set of the second video parameter set The ID does not need to be modified, so the corresponding sub-parameter arameter set id in the SDP is 0, and the parameter set ID in the parameter set NALU in the corresponding parameter set data is 0; the fifth to eighth sub-parameters carry the first video. NALU, the parameter set ID of the first video parameter set needs to be modified and the new value is all 1, so the corresponding sub-parameter parameter set id in the SDP is 1, but the parameter in the parameter set NALU in the corresponding parameter set data The set ID is 0.

m=video 49170 RTP/AVP 98

a=rtpmap:98 H264/90000

a=fmtp:98 sprop-parameter-sets=<vps><0><parameter set data#l> ,

<sps><0><parameter set data#2> ,

<pps><0><parameter set data#3> ,

<vps>< 1 xparameter set data#5> ,

<sps>< 1 xparameter set data#6> ,

<pps>< 1 xparameter set data#7> ,

The parameter set modification information of the first video includes the parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified as an example. Be explained.

Since the parameter set ID in the original parameter set NALU of the first video sent by the application server to the terminal in step 507 is the original parameter set ID of the first video, after the terminal acquires the parameter set directly referenced by the VCL NALU, the VCL NALU is directly The parameter set ID of the parameter set indirectly referenced by the VCL NALU in the referenced parameter set is the original parameter set ID. Since the parameter set ID is based on the parameter set ID indicated in the media description response message, the terminal cannot pass the original parameter set ID. The parameter set obtained in the parameter set acquired in the outband manner obtains the correct parameter set. Therefore, in step 508, the parameter set ID modification information of the first video further includes the parameter set ID modification information indirectly referenced by the VCL NALU. Correspondingly, in step 513, the parameter set identification modification message also needs to include the parameter set ID modification information of the VCL NALU indirect reference. The first encoder is an encoder that complies with the HEVC standard, that is, the parameter set type indirectly referenced by VCL NALU includes SPS and VPS as an example.

The parameter set identification modification information SEI message also needs to include vps_id_change_flag (video parameter set identification change flag), sps_id_change_flag (sequence parameter set identification change identifier), if the parameter set type of the parameter set ID to be modified includes vps, the splicer sets vps_id_change_flag to 1 And passing the corresponding parameter set ID offset by vps id diff (video parameter set identifier offset); if the parameter set type of the parameter set ID to be modified includes sps, the splicer sets sps_id_change_flag to 1, and identifies by the parameter sequence parameter set The offset (sps_id_diff ) passes the corresponding parameter set ID offset. That is, in this way, the format of the parameter set identification modification information SEI message is as shown in Table 5:

Table 5 The description of the parameters of the fifth table is the same as that of the second embodiment. The embodiment of the present invention is not described in detail herein. After the terminal acquires the SDP carrying the second video parameter set NALU and the original parameter set NALU of the first video, Each parameter set NALU is stored in a parameter set NALU record table, and the format of the parameter set NALU record table is as shown in Table 6. Corresponding to each sub-parameter of the parameter sprop-parameter-sets in the SDP:

Parameter set type parameter set id parameter set data where, as described above, arameter set type represents the parameter set type, arameter set id represents the parameter set ID of the parameter set used to decode the VCL NALU, and the parameter set data is the Base64 encoded modified parameter. Specifically, if the SDP of the original parameter set NALU carrying the second video parameter set NALU and the first video acquired by the terminal is the SDP example of the foregoing carrying parameter set NALU, the parameter set NALU record table is as follows. Table 7 shows

Table 7

In step 515, after receiving the parameter set identification modification information SEI message, the terminal obtains the reference aps id and the reference_pps_id according to the parameter set identifier tamper information SEI message, and takes the reference aps id and the reference_pps_id from the parameters learned by the sufficiency method. The centralized capture contains the aps id as the reference aps id APS, including the 々ps id as the reference_pps_id PPS. Then, the sps_id included in the PPS is obtained, that is, the original_sps_id, and the reference information is modified according to the parameter set identifier SEI message to obtain the reference_sps_id (reference sequence parameter set identifier value), and the included sps id is obtained from the parameter set obtained by the outband method according to the reference aps id. SPS for reference sps id. The vps id in the SPS is taken as the original_vps_id, and the reference information is modified according to the parameter set identifier SEI message to obtain the reference_vps_id (reference video parameter set identifier value), and the included parameter is obtained from the parameter set obtained by the outband method according to the reference vps id. Vps id VPS for reference vps id.

Specifically, the reference parameter set is obtained by using a reference aps id , a reference_pps_id, a reference sps id, or a reference vps id, and the parameter set type is obtained from the parameter set NALU record table and the arameter set id is equal to the reference aps id , reference_pps_id, Reference sps id or reference vps id

The NALU record extracts the corresponding Base64 encoded parameter set NALU from the parameter set data of the record, and then decodes the VCL NALU.

As another alternative, when the first video includes only one VPS, one SPS, one PPS, and one APS, in order to simplify the terminal operation, after the splicer performs step 512, when step 513 is performed, that is, when the splicer confirms a certain When the parameter ID of the type needs to be modified, the splicer may directly include the new value of the corresponding parameter set ID in the Parameter Set ID Change Info SEI message instead of the corresponding parameter set ID. Offset. Alternatively, the splicer may directly include the modified PPS in the PPS identification modification information SEI message or the APS identification modification information SEI message.

ID or modified APS ID. Correspondingly, in step 515, the terminal directly obtains the corresponding parameter set ID from the parameter set identification modification message of the first video to determine the parameter set ID of the parameter set used to decode the VCL NALU. As another alternative, the parameter set ID modification information of the first video in step 508 is a parameter set type of the parameter set ID to be modified and a parameter set ID of a parameter set for decoding the VCL NALU, where The parameter set ID of the parameter set of the VCL NALU is the value of the modified parameter set ID assigned by the application server for the parameter set ID to be modified. In step 507, a parameter set including a first video in which the parameter set ID is modified in the SDP is adopted.

NALU is taken as an example for illustration. As an optional manner, in step 514, the splicer is sent to the terminal.

Pps id change flag u(l) if (aps id change flag)

New aps id u(5) if (pps id change flag)

New_pps_id u(8)

}

The parameter APS identifier change identifier (aps_id_change_flag) has a value of 1, indicating that the APS ID needs to be modified, and the parameter PPS identifier change identifier (pps_id_change_flag) has a value of 1, indicating that the PPS ID needs to be modified. Descirptor:u(l) indicates that the parameters aps_id_change_flag and pps_id_change_flag occupy one bit. The parameter new_aps_id carries the modified APS ID, and the parameter new_aps_id carries the modified PPS ID. Descirptor: u(5) indicates an unsigned integer incremented by a 5-bit table, and u(8) indicates an unsigned integer represented by 8 bits.

Of course, if the splicer sends the PPS identification modification information SEI message and/or the APS identification modification information SEI message to the terminal, the message format is similar to that of the table 8, which is not described in detail in the embodiment of the present invention.

In step 507, the original parameter set NALU and the original parameter set including the first video in the SDP are adopted.

The parameter set ID of the parameter set for decoding the VCL NALU corresponding to the NALU is taken as an example for description. When the first splicer performs step 513, the parameter set identifies the modification information. The format of the SEI message is as shown in Table 9:

Table 9

The functions of the parameters in Table 9 are similar to those in Table 8. The embodiments of the present invention are not described in detail herein. As another implementation manner of the embodiment of the present invention, the first video may also be determined in the middle of the session process. With this solution, the address of the first content server needs to be pre-configured on the splicer. When the splicer determines that splicing preparation is required, the first video code stream is requested from the first content server. With such a scheme, the terminal only has a parameter set of the second video in the media description information obtained before the session is initiated or the session is established, and the terminal needs to re-acquire the media set including the parameter set of the second video and the parameter set of the first video in the middle of the session. Description. Referring to FIG. 6, another flowchart of a code stream splicing method provided by an embodiment of the present invention includes the following steps:

The steps 601-605 are the same as the steps 501-504, and the embodiments of the present invention are not described in detail herein. Step 606: The application server sends a first media description information acquisition response message to the terminal, where the media description information includes a parameter set of the second video.

The steps 607-609 are the same as the steps 509-511, and the embodiments of the present invention are not described in detail herein.

Step 610: The splicer sends a first video acquisition request to the first content server according to the configured address of the first content server, where the first video acquisition request further includes a user identifier and second video code stream identification information.

Specifically, before performing this step, the splicer determines whether splicing preparation is required. If splicing preparation is required, the splicer sends a first video acquisition request to the first content server.

For the method of determining whether the splicer needs to be spliced, refer to the description of step 512, and the embodiment of the present invention will not be described in detail herein.

Step 611: The first content server determines the first video code stream, and sends the first video code stream to the splicer.

As an alternative manner of determining the first video code stream, the first video acquisition request sent by the splicer in step 608 received by the first content server includes the identifier of the terminal user, the first content. The server acquires the user preference of the terminal user according to the identifier of the terminal user, selects the first video according to the acquired user preference, and determines the first video code stream.

As another alternative manner of determining the first video code stream, the first content server may further acquire an insertion policy of the second content according to the identification information of the second video stream, select the first content according to the insertion strategy, and determine the first video. Code stream. When the method is used, the first video acquisition request in step 608 needs to include the identification information of the second video stream. Of course, the first content server can also select the first content based only on the stored business policy. In addition, the first content server may determine the first video bitstream in another manner, which is not described in detail herein. Step 612: The first content server sends the media description information of the first video to the application server, where the media description information of the first video includes an original parameter set of the first video, a type of the parameter set of the first video, and a type corresponding to each type. The original parameter set ID of the parameter set.

Step 613: The application server modifies the original parameter set ID of the first video.

Before the step 613 is performed, the application server determines, according to the identifier of the terminal user or the second video code stream identification information, the description information of the second video corresponding to the received media description information of the first video. The specific method of the application server to modify the original parameter set ID of the first video is the same as the step 506, and the embodiment of the present invention will not be described in detail herein.

Step 614: The application server sends the second video code stream and the first video code stream information to the splicer. Specifically, the second video code stream information is the second video code stream identification information, and the first video code stream information is the first video parameter set ID modification information. Step 614 is the same as step 508, and the embodiment of the present invention will not be described in detail herein.

Step 615: The splicer generates a parameter set identifier modification message (such as a parameter set identifier modification information SEI message) that is not valid according to the first video parameter set ID modification information, and the parameter set identifies the modification message except the parameter of the first video. In addition to the ID modification information, the flag information indicating that the parameter set ID modification information is not valid is further included. In the process, the parameter set identifier modification message includes only the parameter set ID modification information corresponding to the parameter set type directly referenced by the VCL NALU, and the parameter set identifier modification message is the parameter set identifier modification information SEI message as an example. Specifically, in the parameter set identifier modification information SEI message, the modification information validation flag (change_info_enable_flag) is included in the example, and the value of the flag is set to 0. Of course, other methods, such as taking the value, F, The character string, the TALSE, and other identifiers are not valid, and the parameter set ID modification information carried in the parameter set ID modification information SEI message is not valid.

The parameter set modification type of the parameter set of the first video includes the parameter set type of the parameter set ID of the first video and the parameter set ID offset corresponding to the parameter set type of the parameter set ID to be modified, and the first encoder is The encoder according to the HEVC standard is explained. Further, the format of the parameter Set ID Change Info SEI message carrying the modification information valid flag is shown in the port table 10:

Table 10

Among them, in this step, the splicer sets change_info_enable_flag to 0. Indicates that the parameter set ID modification information of the message does not take effect temporarily. For other parameters of the message, as described in Table 2, the embodiments of the present invention are not described in detail herein.

Step 616: The splicer sends, to the terminal, a parameter set identifier modification message (ie, a parameter set identifier modification message that is not valid) and a second video code stream, where the first video that indicates that the parameter set ID modification information is not valid is sent.

Specifically, the splicer sends, according to the second video code stream sending step 615, a parameter set identifier modification message that includes flag information indicating that the parameter set ID modification information is not valid, such as including the indication parameter set. The parameter set ID modification information SEI message of the flag information in which the ID modification information does not take effect. Similarly, in order to ensure that the terminal receives the SEI message, the NALU encapsulating the parameter set identifier tampering message may be inserted in consecutive AUs.

In order to ensure smooth transmission, the splicer usually has a buffer. After receiving the video stream, the splicer stores the video stream in a buffer. In order to reduce the overhead caused by the parameter set identifier modification message including the flag information indicating that the parameter set ID modification information is not valid, the splicer may wait for the second video stream before the splicing point to be buffered, and determine the current to be sent. Whether the second video picture is the last RAP before the splicing point, and if so, the parameter set identification modification message including the flag information indicating that the parameter set ID modification information is not valid is sent with the second video RAP. Similarly, in order to ensure that the terminal receives the SEI message, the NALU encapsulating the parameter set identifier tampering message may be inserted into multiple consecutive AUs after the RAP AU.

If the parameter set identification modification message is a PPS identification modification information SEI message and/or an APS identification modification information SEI message, the splicer may similarly generate PPS identification modification information including flag information indicating that the parameter set ID modification information is not valid. The SEI message and/or the APS identifier containing the flag information indicating that the parameter set ID change information is not valid tamper with the message SEI message.

Step 617: The terminal receives the parameter set identifier modification message of the first video that includes the flag information indicating that the parameter set ID modification information is not valid, and confirms that the flag information indicating that the parameter set ID modification information is invalid is not included, and sends the second media to the application server. Describe the information acquisition request (such as a session change request).

Specifically, the terminal determines that the change info enable flag in the parameter set identifier modification message is 0, that is, the confirmation parameter set identifier modification message includes the flag information indicating that the parameter set ID modification information is invalid.

The terminal sends a media description information acquisition request to the application server, such as a SIP Re-Invite request,

RTSP Get Parameter request.

Step 618: The application server sends a second media description information acquisition response (such as a session change response) to the terminal, where the media description information acquisition response includes a second video parameter set NALU, a first view. Frequency parameter set NALU. The parameter set ID in the first video parameter set NALU is the parameter set ID of the modified first video, specifically the original value of the parameter set ID of the first video plus the offset of the parameter set ID, or the application server. The value assigned directly to the parameter set ID of the modified first video.

Step 619: The splicer generates a parameter set identifier modification message (such as a parameter set ID modification information SEI message, or a PPS identifier modification information SEI message and/or an APS identifier tamper information SEI, including flag information indicating that the parameter set ID modification information is valid. The message is that change_info_enable_flag is set to 1.

Step 620: The splicer sends, to the terminal, a parameter set identifier modification message and a first video code stream that include flag information indicating that the parameter set ID modification information is valid.

Before performing step 620, the splicer also determines if the splice point is reached. The splicer sends a parameter set identification modification message and a first video code stream to the terminal when the splicing point is reached. The specific manner in which the splicer judges whether or not the splice point is reached is the same as the same description in the flow described in FIG.

In order to ensure random access, the parameter set identification modification message (parameter set ID modification information SEI message, or PPS identification modification information SEI message and/or APS identification modification information SEI message) is transmitted at least with the random access point and applied to the associated RAP All pictures between the next RAP and the next RAP. The RAP is a picture in the middle of the code stream that only uses intra prediction, and the decoder can start the decoding process from the picture. When the parameter set identifier modification message is sent with the RAP, the splicer inserts the NALU encapsulated with the parameter set identifier modification message into the AU separator NALU of the associated RAP, before all VCL NALUs.

621. After receiving, by the terminal, a parameter set identifier modification message (such as a parameter set ID modification information SEI message, or a PPS identity modification information SEI message and/or an APS identity modification information SEI message) that includes flag information indicating that the parameter set ID modification information is valid, The acknowledgment includes the flag information indicating that the parameter set ID modification information is valid (for example, the change_info_enable_flag in the parameter set identifier modification message is 1), and the modification is determined according to the parameter set identifier including the flag information indicating that the parameter set ID modification information is valid. The parameter set ID of the parameter set for decoding VCL NALU. Obtaining the parameter set ID from the parameter set obtained by the out-of-band method according to the determined parameter set ID and determining the parameter set ID The same parameter set.

The method for determining the parameter set ID of the VCL NALU parameter set in the video code stream according to the parameter set identification modification message may refer to the description of step 515, which is not described in detail herein. As an optional manner, after performing step 614, the application server may not actively perform the step 615 to 618, and the application server actively sends the media description information to the terminal (not shown in the flowchart). The media description information includes a parameter set of the second video and a modified modified parameter set of the first video. The modified parameter set ID of the first video is the parameter set ID of the modified first video (ie, the parameter set ID of the parameter set used to decode the VCL NALU). Specifically, the application server may send a session change request message including the media description information, such as a SIP Re-Invite request message or a SIP Update request message, to the terminal, or send a notification message including the media description information to the terminal, such as an RTSP Play-Notify request. Message, RTSP Set Parameter request message, RTSP Notify request message or SIP Info request message. The terminal receives the media description information sent by the application server, and obtains the second video parameter set NALU and the first video parameter set NALU. It is also possible to perform steps 513-516 without performing steps 619-622.

As an alternative, the parameter set identifier modification message in the process corresponding to FIG. 6 further includes parameter set ID modification information corresponding to the parameter set type indirectly referenced by the VCL NALU of the first video, and correspondingly, the media in step 618 The description information acquisition response message or the media description information that the application server actively sends to the terminal includes the original parameter set of the first video and the parameter set ID of the parameter set for decoding the VCL NALU.

As another optional manner of the present invention, the parameter set identifies the modification message (such as parameter set ID modification)

The SEI message contains the parameter set type of the parameter set ID to be modified, and does not carry other parameter set ID modification information, such as carrying the modified value or the parameter set ID to modify the offset, that is, carrying one or more * * * _id_change_flag, without carrying * * * - id - diff parameter or new_* * *_id parameter. After receiving the parameter set identifier modification message, the terminal determines the type of the first video parameter set that needs to be modified according to the parameter set identifier modification message, and requests the splicer to send the first video parameter set type corresponding to the ID to be modified by the RTCP message. *_id_diff or new_***_id, the splicer will use the RTCP message to terminate the terminal The requested parameter set ID modification information is sent to the terminal. That is, in order to guarantee random access, only ***_id_change_flag is circulated at the RAP interval, and ***_id_diff or new_***_id need only be sent once. In this embodiment, "***" is a parameter set type. FIG. 7 is a flowchart of another method for splicing a code stream according to an embodiment of the present invention, including the following steps:

Steps 701-712 are the same as steps 501-512, and embodiments of the present invention are not described in detail herein.

Step 713: The splicer generates a parameter set identifier modification message (such as a parameter set ID modification information SEI message). The parameter set identifies a parameter set type including only the parameter set ID to be modified in the modification message, and does not carry other parameter set ID modification information, such as a parameter set ID or a parameter set ID that does not carry a parameter set for decoding the VCL NALU. Offset.

Specifically, if the media description information sent by the application server to the terminal in step 707 carries the modified parameter set of the first video (that is, the parameter set ID in the modified parameter set is used to decode the parameter set of the VCL NALU. Parameter set ID), then the parameter set type of the parameter set ID to be modified in this step is only the parameter set type directly referenced by VCL NALU. The first encoder is an encoder that complies with the HEVC coding standard as an example. The parameter set type of the parameter set ID to be modified is APS and/or PPS, that is, the parameter set ID modification information carries aps_id_change_flag and pps id change flag, and if the splicer receives the ID of the parameter set ID of the first video to be modified, The first video parameter set type includes APS, the splicer sets aps_id_change_flag to 1, otherwise it is set to 0; if the splicer receives the first video parameter set ID modification information, the first video parameter set type of the ID to be modified includes the PPS The splicer sets pps_id_change_flag to 1, otherwise it is set to 0.

As another alternative, first, if the media description information sent by the application server to the terminal carries the original first video parameter set NALU in step 709, the parameter set type of the parameter set ID to be modified in this step is further A parameter set type that contains an indirect reference to the VCL NALU. The first encoder is an encoder that complies with the HEVC coding standard as an example. The parameter set identifier modification message must include the vps id change flag, the sps id change flag, the aps id change flag, and the pps id change flag. The specific format of the parameter set ID modification information SEI message is as shown in Table 11. Parameter set id change info (PayloadSize) { Descriptor vps id change flag u(l) sps id change flag u(l) aps id change flag u(l) pps id change flag u(l)

}

Table 11

The setting of each parameter in Table 11 is the same as that in Table 9. The embodiment of the present invention is not described in detail herein. Step 714: The splicer sends a parameter set identifier modification message (such as a Parameter Set ID Change Info SEI message) and the first video code stream to the terminal. The parameter set identifies a parameter set type including only the modified parameter set ID in the modification message, and does not carry other parameter set ID modification information, such as a parameter set ID or a original parameter set that does not carry a parameter set for decoding the VCL NALU. The offset of the ID.

Step 715: The terminal determines, according to the parameter set identifier modification message, a parameter set type that needs to be modified. Step 716: The terminal sends an RTCP request to the splicer, where the parameter set type determined in step 715 that needs to modify the parameter set ID is included. The above RTCP request may be an RTCP parameter set identifier tamper information request Parameter Set ID Change - Request message, referred to as RTCP PSIC-R message. The RTCP PSIC-R message is an RTCP feedback message, as shown in Figure 8.

Where: PT (Payload Type): RTCP packet type

FMT (Feedback Message Type): feedback message type, unique under the package type

SSRC of packet sender: SSRC of the sender of the RTCP feedback message

SSRC of media source: SSRC of the RTP stream corresponding to the RTCP feedback message.

FCI (Feedback Control Information): The content of the RTCP feedback message. Different types of RTCP feedback messages contain different contents.

The PT value of the RTCP PSIC-R message is 206, which means it is Payload Specific Feedback Message; the FMT value is 4, which means it is a PSIC message; the SSRC of packet sender is the SSRC of the terminal; and the SSRC of media source is obtained from the SDP. RTP for carrying the output video stream The SSRC of the stream, that is, the SSRC in the second video stream identification information. (For the terminal, the second video stream and the first video stream are not distinguished, and the SSRC seen by the terminal is one). The format of the FCI is shown in Figure 9. among them

SFMT (Sub Feedback Message Type): The type of PSIC message. Set here to 1, representing the PSIC-R message;

Number of Parameter Set Type: The number of parameter set types whose ID needs to be modified;

Parameter Set Type: The parameter set type identifier of the ID to be modified. There are a total of Number of Parameter Set Type Parameter Set Type. The Parameter Set Type can take many forms. For example, the first encoder is an encoder that complies with the HEVC standard. The value of the Parameter Set Type can be a positive integer, and the values 1, 2, 3, and 4 represent VPS, SPS, PPS, and APS, respectively.

Step 717: After receiving the RTCP request (such as the RTCP PSIC-R message), the splicer sends the parameter set ID modification information of the first video to the terminal by using an RTCP response (such as an RTCP PSIC-I message). The RTCP PSIC-I message is also an RTCP PSIC message whose SFMT is set to 2, including Number of Parameter Set ID Change Info, indicating the parameter set identifier contained in the RTCP PSIC-I message. Modify the number of messages.

If the parameter set ID modification information includes the first video parameter set type of the ID to be modified and the offset of the corresponding original parameter set ID, the format of the FCI is as shown in FIG. A parameter set identification modification information includes a parameter set type (Parameter Set Type) and an original parameter set ID offset (Parameter Set Diff).

If the parameter set ID modification information includes the first video parameter set type of the ID to be modified and the corresponding parameter set ID mapping relationship, the FCI format is as shown in FIG. A parameter set identification modification information includes a Parameter Set Type, an Original Parameter Set ID, and a New Parameter Set ID. If the parameter set ID modification information includes a first video parameter set type of the ID to be modified and a corresponding new value of the parameter set ID, the FCI format is as follows. Figure 12 shows. A parameter set identification modification information includes a Parameter Set Type and a New Parameter Set ID.

Step 720: The terminal receives the RTCP PSIC-I message, for each first video VCL NALU, The terminal determines the value of the parameter set ID of the VCL NALU reference parameter set according to the RTCP PSIC-I message, and obtains all referenced parameter sets, and decodes the corresponding VCL NALU by using the obtained parameter set.

As another alternative manner of the embodiment of the present invention, the first video parameter set ID modification information may also be sent by the splicer to the terminal by using the transport layer signaling. The splicer generates an RTCP PSCI-I message as described above for the RTCP PSCI-I message. After reaching the splicing point, the splicer sends a first video bitstream and an RTCP PSCI-I message to the terminal, and the terminal receives the first video bitstream and the RTCP PSCI-I message. For each first video VCL NALU, the terminal is based on the RTCP PSIC- The I message determines the value of the parameter set ID of the VCL NALU reference parameter set, and obtains all referenced parameter sets, and the terminal uses these parameter sets to decode the corresponding VCL NALU.

In addition, in order to ensure that the terminal obtains the first video parameter set ID modification information before decoding the first video code stream, the splicer may also determine the sequence number of the first RTP packet used to encapsulate the first video code stream, and then Send an RTCP PSCI-I message. At this time, the RTCP PSCI-I message includes the first sequence number (first sequence number) in addition to all the information of the RTCP PSCI-I message in step 717, and the first sequence number is used to encapsulate the first video stream. The serial number of the RTP package. In order to ensure that the terminal receives the RTCP PSCI-I message, the splicer can continuously send multiple identical RTCP PSCI-I messages.

In the above embodiment, the parameter set identification modification message (such as: Parameter Set ID Change Info SEI Message) is generated by the splicer. In actual applications, the parameter set identification modification message (such as: Parameter Set ID Change Info SEI Message) can also be generated by the application server and sent to the splicer. For example, when the application server sends the parameter set identifier modification information to the splicer, the parameter set ID modification information of the first video is directly sent in the form of a Parameter Set ID Change Info SEI Message. The splicer is only responsible for the transmission of the Parameter Set ID Change Info SEI Message.

Since the application server has entropy coding and entropy decoding capability, when the application server generates the Parameter Set ID Change Info SEI Message, the ***_id_diff or the new_***_id in the Parameter Set ID Change Info SEI Message may be entropy encoded, for example Unsigned index Columbus code. FIG. 13 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention, including the following steps: Step 1301: Receive information of a first video code stream from an application server, where information of the first video code stream includes first The parameter set modification information of the video; specifically, the splicer receives information of the first video code stream from the application server, where the first video code stream information includes parameter set modification information of the first video. Further, the splicer can also receive information of the second video stream from the application server. Step 1302: Send a first video code stream and a parameter set identifier modification message of the first video to the terminal according to the first video parameter set modification information, where the parameter set identifier modification message of the first video carries a parameter set of the first video Identifying ID modification information, so that the terminal acquires a parameter set ID of a parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal is used to decode the VCL NALU according to the The parameter set ID of the parameter set acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video. As an alternative, prior to step 1302, the splicer also pauses outputting the second video stream during the output of the second video stream. As an alternative,

The parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified, and the splicer further receives a real-time transmission control protocol RTCP request sent by the terminal receiving terminal, where the RTCP request includes the parameter set ID to be modified. a parameter set type; returning an RTCP response to the terminal, where the RTCP response carries an original parameter corresponding to the parameter set type of the parameter set ID to be modified An offset of the set ID, or a parameter set ID carrying a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set type carrying the parameter set ID to be modified The original parameter set ID and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU. or

The parameter set ID modification information is an offset of a parameter set type of the parameter set ID to be modified and an original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set ID to be modified. a parameter set type and a parameter set ID of a parameter set for decoding a VCL NALU, or the original parameter set ID of a parameter set type including a parameter set ID to be modified, and a decoding corresponding to the original parameter set ID for decoding a parameter set ID of a parameter set of the VCL NALU, the parameter set ID modification information further comprising a modification validation flag, the parameter for decoding the VCL NALU being obtained according to the parameter set ID of the parameter set for decoding the VCL NALU Before the set, the receiver further sends a parameter set identifier modification message of the first video that is not valid when the second video code stream is sent to the terminal; the second media description information acquisition request sent by the terminal is received; and the second media is returned to the terminal. a description information acquisition response, the second media description information acquisition response, the modified parameter set of the first video, the parameter set ID in the first video parameter set NALU Parameter set parameter set for decoding VCL NALU's ID.

As an optional manner, the parameter set identification modification message of the first video is: a parameter set identifier modification supplementary enhancement information SEI message, an adaptive parameter set identifier modification SEI message, or a picture parameter set identifier modification SEI message, and the splicer further Generating a parameter set identification modification message of the first video according to the first video parameter set modification information.

It can be seen that, after receiving the parameter set modification information of the first video sent by the application server, the splicer does not modify the parameter set ID referenced by the VCL NALU in the first video code stream according to the parameter set modification information. Rather, the parameter set identifier modification message carrying the parameter set modification information is sent to the terminal, and the terminal obtains the parameter of the VCL NALU in the first video code stream according to the parameter set identifier modification message before decoding the VCL NALU in the first video stream. Set to decode the parameter set. Therefore, there is no need for the splicer to have the ability to modify the entropy coding and entropy decoding of the parameter set ID of the VCL NALU reference in the first video stream, thereby eliminating the need to add extra hard on the splicer of the existing network. Component resources, or use a new splicer to replace the splicer in the existing network, and ensure that the terminal decoding is correct.

FIG. 14 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention, including the following steps:

Step 1401: Receive media description information of the first video sent by the first content server, where the media description information of the first video carries the type of the parameter set of the first video of the original parameter set of the first video and the parameter corresponding to each type The original parameter set ID of the set; Step 1402: Receive media description information of the second video sent by the second content server, where the media description information of the second video carries the parameter set of the second video parameter set of the second video and a parameter set ID of a parameter set corresponding to each type;

Step 1403: Determine that the parameter set ID of the first video of the same type is the same as the parameter set ID of the parameter set of the second video, and determine the parameter set identification modification information of the first video. Step 1404: Send the first to the splicer The information of the video stream, the information of the first video code stream includes the parameter set identifier modification information of the first video, and the parameter set modification identifier information of the first video is carried in the parameter set identifier modification message of the first video, so that And sending, by the splicer, the parameter set identification modification message of the first video to the terminal, and in order for the terminal to modify the message according to the parameter set identifier of the first video to decode and output the first video. As an optional manner, the application server further modifies the original parameter set ID of the original parameter set of the first video to the video coding layer for decoding the network abstraction layer unit according to the determined parameter set identification modification information of the first video. The parameter set ID of the parameter set of the VCL NALU; the modified parameter set carrying the first video is sent to the terminal, and the parameter set ID in the modified parameter set is the parameter set ID of the parameter set used to decode the VCL NALU;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an original parameter of the first video corresponding to the parameter set type of the parameter set ID to be modified. The offset of the set ID, or the parameter set type including the parameter set ID to be modified, and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the first The parameter set identification modification message of the video includes a parameter set ID of the parameter set type of the parameter set ID to be modified, the original parameter set ID, and a parameter set for decoding the VCL NALU corresponding to the original parameter set ID. ;

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU.

In another optional manner, the application server further needs to send, to the terminal, an original parameter set of the first video and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU, where the first video is The original parameter set carries the original parameter set ID;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or The parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID of the parameter set type of the parameter set ID to be modified, and a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the original parameter set ID;

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU. It can be seen from the foregoing embodiment that the application server determines that the original parameter set ID of the first video parameter set of the same type is the same as the parameter set ID of the second video parameter set, and determines the offset of the original parameter set ID of the first video of the type. Or a parameter set ID for decoding a parameter set of the VCL NALU, and directly sending a parameter set identification modification message of the first video to the splicer, so that the splicer identifies the parameter set of the first video to modify the message, thereby eliminating the need for a splicer The ability to modify the entropy encoding and entropy decoding of the parameter set ID referenced by the VCL NALU in the first video stream, thereby eliminating the need to add additional hardware resources to the splicer of the existing network, or using a new splicer pair There is a splicer in the network to replace, and can ensure that the terminal decoding is correct. 15A is a schematic structural diagram of a decoding apparatus for a digital video code stream according to an embodiment of the present invention, including a receiving module 1501, a first obtaining module 1502, a second obtaining module 1503, a decoding input module 1504, a determining module 1505, and a sending module 1506. Specifically, the receiving module 1501 is configured to receive a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier modification message of the first video includes the parameter set identifier ID modification information of the first video; a first obtaining module 1502, configured to acquire, according to the parameter set ID modification information of the first video received by the receiving module 1501, a parameter set ID of a parameter set for decoding a video encoding layer VCL NALU of the network abstraction layer unit; The obtaining module 1503 is configured to acquire, according to the parameter set ID of the parameter set for decoding the VCL NALU acquired by the first acquiring module 1502, a parameter set for decoding the VCL NALU, and the decoding output module 1504, according to the second The parameter set for decoding the VCL NALU acquired by the obtaining module 1503 decodes the VCL NALU and outputs the decoded first video.

As an optional manner, the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified; The first video code stream includes an original parameter set ID for decoding a parameter set of the VCL NALU; the first obtaining module 1502 is specifically configured to: according to the parameter set to be modified received by the receiving module 1501 Obtaining, by the parameter set type of the ID, the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified from the first code stream; according to the original parameter set ID and the parameter to be modified The offset of the original parameter set ID corresponding to the parameter set type of the set ID is obtained by acquiring the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

As an alternative, the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and a parameter set ID of the parameter set for decoding the VCL NALU, and the first obtaining module 1502 is specifically configured to: Obtaining, according to the parameter set type of the parameter set ID to be modified received by the receiving module 1501, from the parameter set ID modification information, the type corresponding to the parameter set type of the parameter set ID to be modified, for decoding the VCL NALU The parameter set ID of the parameter set. As another optional manner, the parameter set ID modification information is an original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU. The first acquiring module is configured to: obtain, according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified received by the receiving module, the original parameter set ID Decoding the parameter set ID of the parameter set of the VCL NALU; or the first obtaining module is specifically configured to obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, from the first video code stream The original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified; acquiring the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID according to the original parameter set ID. In another optional manner, the first video parameter set ID modification information is a parameter set to be modified.

The parameter set type of the ID, the device further includes: a sending module 1506, configured to send a real-time transport control protocol RTCP request to the splicer, where the RTCP request includes a parameter set type of the parameter set ID to be modified; 1501 is further configured to: receive an RTCP response returned by the splicer, where the RTCP response carries an offset of an original parameter set ID corresponding to a parameter set type of a parameter set ID to be modified, or a parameter set ID to be modified a parameter set ID of a parameter set for decoding a VCL NALU corresponding to a parameter set type, or the original parameter set ID of a parameter set type of a parameter set ID to be modified, and a corresponding to the original parameter set ID Decoding a parameter set ID of a parameter set of the VCL NALU; the first obtaining module 1502 is specifically configured to: according to the parameter set type of the parameter set ID to be modified carried in the RTCP response received by the receiving module 1501 The offset of the original parameter set ID or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or according to The original parameter set ID of the parameter set type of the modified parameter set ID and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID are obtained from the parameter set ID modification information and are to be modified. The parameter set ID of the parameter set type of the parameter set ID corresponding to the parameter set used to decode the VCL NALU.

Optionally, the parameter set ID modification information further includes a modification effective flag. Receiving module The method 1501 is further configured to: before receiving the parameter set identifier modification message of the first video code stream and the first video, receive a parameter set identifier modification message of the first video that is not valid; the apparatus further includes: the determining module 1505 is further configured to: Determining, by the receiving module, that the parameter set identifier modification message of the inactive first video includes the flag information indicating that the parameter set ID modification information is not valid; the sending module 1506 is configured to send the second media description information to the application server. The receiving module 1501 is further configured to: receive a second media description information acquisition response returned by the application server, where the second media description information acquisition response carries the second video parameter set, and the modified parameter set of the first video The parameter set ID of the parameter set in the first video modification is the parameter set ID of the parameter set for decoding the VCL NALU.

Optionally, the receiving module 1501 is further configured to pass the band before the second obtaining module 1503 obtains a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU. Obtaining a modified parameter set of the first video, the parameter set ID of the modified parameter set is a parameter set ID of a parameter set for decoding the VCL NALU; or the receiving module 1501 is in the second acquiring The module 1503 is further configured to obtain an original parameter set of the first video by using an out-of-band manner and to decode the VCL, before acquiring the parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU. The parameter set ID of the parameter set of the NALU; the original parameter set ID is modified according to the modified parameter set ID.

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU; the first obtaining module 1502 is specifically configured to: acquire a parameter set directly received by the receiving module and the VCL NALU The parameter set ID of the parameter set directly referenced by the VCL NALU for decoding the VCL NALU, and is obtained by the second obtaining module 1503 according to the parameter set ID of the parameter set directly referenced by the VCL NALU for decoding the VCL NALU. After the parameter set directly referenced by the VCL NALU in the first video bitstream, the first obtaining module 1502 further obtains, from the directly referenced parameter set, a parameter set of a parameter set for invoking a VCL NALU indirect reference of the VCL NALU. ID; or

The parameter set type of the parameter set ID to be modified is the parameter set type directly referenced by the VCL NALU and the parameter set type indirectly referenced by the VCL NALU. The first obtaining module 1502 is specifically configured to: respectively The parameter set ID modification information of a video acquires a parameter set ID of a parameter set of a direct reference for decoding a VCL NALU of the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, and obtains an indirect reference with the VCL NALU Reference a parameter set ID corresponding to the parameter set of the VCL NALU corresponding to the number set type;

The second obtaining module 1503 is specifically configured to: obtain a directly referenced parameter of the VCL NALU according to the parameter set ID of the directly referenced parameter set acquired by the first obtaining module 1502 and the parameter set ID of the indirectly referenced parameter set. And a parameter set of the indirect reference of the VCL NALU; the decoding output module 1504 is specifically configured to: according to the directly referenced parameter set of the VCL NALU acquired by the second obtaining module 1502 and the indirect of the VCL NALU The referenced parameter set decodes the VCL NALU and outputs the decoded first video. It should be noted that the device in FIG. 15 may be the terminal in the foregoing embodiment, and may perform any step performed by the terminal in the foregoing embodiment, and the embodiment of the present invention is not described in detail herein.

FIG. 16 is a splicing device for a digital video code stream according to an embodiment of the present invention, including: a receiving module

1601. A module 1602 and a sending module 1603 are generated.

Specifically, the receiving module 1601 is configured to receive, by the application server, information about the first video code stream, where the first video code stream information includes parameter set modification information of the first video.

The sending module 1603 is configured to send, according to the first video parameter set modification information received by the receiving module 1601, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier of the first video The modification message carries the parameter set identifier ID modification information of the first video, so that the terminal acquires the parameter set ID of the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and The terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoding. After the first video.

As an alternative, when the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified, the receiving module 1601 is further configured to receive a real-time transmission control protocol RTCP request sent by the terminal, where The RTCP request includes a parameter set type of the parameter set ID to be modified; the sending module 1603 is further configured to return an RTCP response to the terminal, where the RTCP response carries a parameter set type corresponding to the parameter set ID to be modified. The offset of the original parameter set ID, or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the parameter set type carrying the parameter set ID to be modified The original parameter set ID and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU. Optionally, the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified, and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or includes a to-be-modified The parameter set type of the parameter set ID and the parameter set ID of the parameter set for decoding the VCL NALU, or the original parameter set ID of the parameter set type including the parameter set ID to be modified and corresponding to the original parameter set ID When the parameter set ID of the parameter set of the VCL NALU is decoded, the parameter set ID modification information further includes a modification effective flag, and the sending module 1603 sends the first video code stream and the parameter set identifier modification of the first video to the terminal. Before the message is further used to send the parameter set identifier modification message of the first video that is not valid when the second video code stream is sent to the terminal; the receiving module 1601 is further configured to receive the second media description sent by the terminal. And the sending module 1603 is further configured to: return a second media description information acquisition response to the terminal, where the second media description information acquisition response carries the first video The modified parameter set, the parameter set ID of the modified parameter set of the first video is a parameter set ID of a parameter set for decoding the VCL NALU.

Optionally, the device advancement comprises a generating module 1602, configured to generate a parameter set identifier modification message of the first video according to the first video parameter set modification information.

It should be noted that the generating module 1602 is further configured to generate any message sent to the sending module 1603 according to the information received by the receiving module 1601. The embodiments of the present invention are no longer detailed herein. For details, refer to the description of each step of the foregoing method embodiment. In addition, the apparatus described in FIG. 16 may be a splicer in the foregoing implementation, and any of the steps performed by the splicer in the foregoing embodiment may be performed, and the embodiment of the present invention is not described in detail herein.

FIG. 17 is a splicing apparatus for a digital video code stream according to an embodiment of the present invention, including a first receiving module 1701, a second receiving module 1702, a determining module 1703, and a sending module 1704.

Specifically, the first receiving module 1701 is configured to receive the media description information of the first video that is sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the parameter set of the first video. Type and original parameter set ID of the parameter set corresponding to each type;

The second receiving module 1702 is configured to receive media description information of the second video sent by the second content server, where the media description information of the second video carries a parameter set of the second video, a type of the second video parameter set, and The parameter set ID of the parameter set corresponding to each type;

a determining module 1703, configured to determine an original parameter set ID of the first video received by the first receiving module 1701 of the same type, and a second video parameter set ID received by the second receiving module 1702 Same as, and determining parameter set identification modification information of the first video;

The sending module 1704 is configured to send information about the first video stream to the splicer, where the information of the first video code stream includes parameter set modification information of the first video, and the parameter set modification information of the first video is carried in the first The parameter set identifier of the video is modified in the message, so that the splicer sends the parameter set identification modification message of the first video to the terminal, and the terminal is configured to modify the message according to the parameter set identifier of the first video to decode and output the first message. video.

As an optional manner, the apparatus further includes: a modifying module 1705, configured to determine, according to the determining module 1703, the parameter set identifier modification information to modify an original parameter set ID of an original parameter set of the first video to be used for decoding The parameter set ID of the parameter set of the video coding layer VCL NALU of the network abstraction layer unit;

The sending module is further configured to: send the modified parameter set of the first video to the terminal, where the parameter set ID in the modified parameter set is the parameter set ID of the parameter set used to decode the VCL NALU;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of an original parameter set ID of the first video corresponding to the parameter set type of the parameter set ID to be modified, Or a parameter set type including the parameter set ID to be modified and a parameter set ID for decoding a parameter set of the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identifier modification of the first video The message includes a parameter set ID of the parameter set type of the parameter set ID to be modified corresponding to the original parameter set ID for decoding the parameter set of the VCL NALU;

In another optional manner, the sending module is further configured to: send, to the terminal, an original parameter set of the first video and a parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID, where The original parameter set ID of the first video carries the original parameter set ID;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or The parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID in a parameter set type of the parameter set ID to be modified, and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU; The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU.

The device shown in FIG. 17 may be an application server, and may perform any of the steps or functions performed by the application server in the foregoing embodiment. The embodiments of the present invention are not described in detail herein.

FIG. 18 is a schematic structural diagram of a decoding apparatus or a splicing apparatus for a digital video stream according to an embodiment of the present invention, including a processor 1801, an input device 1802, an output device 1803, a memory 1804, and a communication interface 1805. The computer system may specifically be a processor based computer such as a general purpose personal computer (PC), a portable device such as a tablet computer, or a smart phone. Computer systems include buses, processors, memories, communication interfaces, input devices, and output devices. The bus can include a path to transfer information between various components of the computer. The processor 1801 can be a general purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. The computer system also includes one or more memories 1805, which may be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or Other types of dynamic storage devices that store information and instructions may also be disk storage. These memories are connected to the processor via a bus.

Input device 1801 can include a means for receiving data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, and the like. The output device can include a means to allow output of information to the user, including display screens, printers, speakers, and the like. The computer system also includes a communication interface 1804 that uses devices such as any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), and the like.

A memory 1805, such as a RAM, holds an operating system and a program for executing the inventive scheme. The operating system is a program that controls the running of other programs and manages system resources. The program code for carrying out the inventive arrangement is stored in a memory and is controlled by the processor 1801 for execution.

Specifically, the memory 1805 can store instructions to cause the processor 1801 to perform the following processes:

And receiving a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier of the first video includes a parameter set identifier ID modification information of the first video, and a parameter set according to the first video ID modification information acquires a video coding layer for decoding a network abstraction layer unit a parameter set ID of a parameter set of the VCL NALU; acquiring a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU; according to the parameter set decoding unit for decoding the VCL NALU The VCL NALU is described and the decoded first video is output. It is to be noted that the processor 1805 can also be used to perform the steps performed by the terminal or the decoding device in FIG. 1 to FIG. 15 in the method embodiment, which is not described in detail herein.

As another optional manner of the embodiment of the present invention, the processor 1801 may further perform the following process: receiving information of the first video code stream from the application server, where the information of the first video code stream includes parameter set modification of the first video Transmitting, according to the first video parameter set modification information, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video carries a parameter set identifier of the first video The ID modifies the information, so that the terminal acquires the parameter set ID of the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal according to the method for decoding the VCL NALU The parameter set ID of the parameter set acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video. It should be noted that the processor 1805 can also be used to perform the steps performed by the splicer in FIG. 1 to FIG. 16 in the method embodiment, which is not described in detail herein.

As another optional manner, the processor may further perform the following process: receiving media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the first The type of the parameter set of the video and the original parameter set ID of the parameter set corresponding to each type; the media description information of the second video sent by the second content server, where the media description information of the second video carries the second video a parameter set second video parameter set type and a parameter set ID of a parameter set corresponding to each type; determining a parameter set ID of the first video of the same type is the same as a parameter set ID of a parameter set of the second video, and determining The parameter set of the first video identifies the modification information; the information of the first video stream is sent to the splicer, the information of the first video code stream includes parameter set identification modification information of the first video, and the parameter set of the first video is modified. The identifier information is carried in the parameter set identifier modification message of the first video, so that the splicer sends the parameter set identifier modification message of the first video to the terminal. And, in order for the terminal to modify the message according to the parameter set identification of the first video, and output the first video. It is to be noted that the processor 1805 can also be used to perform the steps performed by the application server in FIG. 1 to FIG. 17 in the embodiment of the method. The embodiments of the present invention are not described in detail herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise. The components displayed for the unit may or may not be physical units, ie may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The above software functional unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute the method of the various embodiments of the present application. Part of the steps. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program code. .

Finally, it should be noted that the above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application is described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that: The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently substituted; and the modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

Claim

A decoding method for a digital video bitstream, which is characterized in that it comprises

Receiving a parameter set identification modification message of the first video code stream and the first video, where the parameter set identifier of the first video includes the parameter set identification ID modification information of the first video;

Obtaining, according to the parameter set ID modification information of the first video, a parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit;

Acquiring a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU;

The VCL NALU is decoded according to the parameter set for decoding the VCL NALU, and the decoded first video is output.

The method according to claim 1, wherein the parameter set ID modification information comprises a parameter set type of the parameter set ID to be modified and an original corresponding to the parameter set type of the parameter set ID to be modified. An offset of the parameter set ID; the first video code stream includes an original parameter set ID for decoding a parameter set of the VCL NALU;

The parameter set ID obtained by acquiring the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video is specifically:

Obtaining, according to the parameter set type of the parameter set ID to be modified, the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified from the first video code stream;

Obtaining, according to the original parameter set ID and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, a decoding VCL corresponding to the parameter set type of the parameter set ID to be modified The parameter set ID of the NALU's parameter set.

3. The method according to claim 1, wherein the parameter set ID modification information comprises a parameter set type of a parameter set ID to be modified and a parameter set ID of a parameter set for decoding a VCL NALU;

The parameter set ID of the NALU parameter set is specifically:

Obtaining, from the parameter set ID modification information, a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, according to the parameter set type of the parameter set ID to be modified. .

4. The method of claim 1 wherein the parameter set ID modification information comprises An original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU;

The parameter set ID of the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video is specifically:

Obtaining, according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, the parameter set ID of the parameter set corresponding to the original parameter set ID for decoding the VCL NALU.

The method of claim 1, wherein the first video parameter set ID modification information is a parameter set type of a parameter set ID to be modified, the method further comprising:

Sending a real-time transmission control protocol RTCP request to the splicer, where the RTCP request includes the parameter set type of the parameter set ID to be modified;

Receiving an RTCP response returned by the splicer, where the RTCP response carries an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified or a parameter set type corresponding to the parameter set ID to be modified. a parameter set ID for decoding a parameter set of the VCL NALU, or the original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter set corresponding to the original parameter set ID for decoding the VCL NALU Parameter set ID;

And a parameter set for decoding the VCL NALU corresponding to the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified carried in the RTCP response or the parameter set type of the parameter set ID to be modified The parameter set ID, or the parameter set ID according to the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID, from the parameter set ID Obtaining, in the modification information, a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.

The method according to any one of claims 2 to 4, wherein the parameter set ID modification information further comprises a modification effective flag, and the parameter set identification modification message of the first video code stream and the first video is received. Previously, the method further included:

Receiving a parameter set identification modification message of the first video that does not take effect;

Determining that the parameter set identifier modification message of the first video that is not in effect includes flag information indicating that the parameter set ID modification information is not valid;

Sending a second media description information acquisition request to the application server; And receiving, by the application server, a second media description information acquisition response, where the second media description information acquisition response carries the modified parameter set of the first video, where the parameter set ID of the parameter set in the first video modification is The parameter set ID of the parameter set used to decode the VCL NALU.

The method according to any one of claims 1 to 5, wherein the method before acquiring a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU Further includes:

Obtaining, by an out-of-band manner, a modified parameter set of the first video, where the parameter set ID of the modified parameter set is a parameter set ID of a parameter set used for decoding the VCL NALU; or

Obtaining, by an outband manner, an original parameter set of the first video and a parameter set ID of a parameter set for decoding the VCL NALU, where the original parameter set of the first video carries the original parameter set ID; The parameter set ID of the parameter set of the NALU modifies the original parameter set ID.

The method according to any one of claims 1 to 7, wherein the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, according to the first video. The parameter set ID modification information acquires the parameter set ID of the parameter set for decoding the VCL NALU, specifically: acquiring parameters of the parameter set directly used by the VCL NALU for decoding the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU Set the ID, the method further comprising: obtaining, according to the obtained parameter set ID of the parameter set directly referenced by the VCL NALU for decoding the VCL NALU, a parameter set directly referenced by the VCL NALU in the first video code stream; from the direct reference The parameter set ID of the parameter set used to decode the VCL NALU indirect reference of the VCL NALU; or

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU and a parameter set type indirectly referenced by the VCL NALU, and the parameter information is modified according to the parameter set ID of the first video. The parameter set ID of the parameter set for decoding the VCL NALU is specifically: acquiring the VCL NALU for decoding the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU according to the parameter set ID modification information of the first video, respectively. a parameter set ID of a directly referenced parameter set, and a parameter set ID of a parameter set for decoding a VCL NALU indirect reference of the VCL NALU corresponding to the parameter set type indirectly referenced by the VCL NALU; Obtaining the parameter set ID for decoding the VCL NALU in the parameter set ID of the parameter set of the decoded VCL NALU is specifically: acquiring the parameter set ID according to the obtained directly referenced parameter set and the parameter set ID of the indirectly referenced parameter set Parameter set and direct reference of VCL NALU a parameter set of an indirect reference to the VCL NALU;

The method according to any one of claims 1-8, wherein the parameter set identification modification message of the first video is: supplementary enhancement information SEI message.

10. A method of splicing a digital video stream, the special feature of which comprises:

Receiving, by the application server, information of the first video code stream, where the information of the first video code stream includes parameter set modification information of the first video;

And sending, by the first video parameter set modification information, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video carries the parameter set identifier ID of the first video. Information, so that the terminal acquires a parameter set ID of a parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal according to the parameter set for decoding the VCL NALU The parameter set ID acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.

1 1. The method of claim 10, wherein the method further comprises: receiving information of the second video code stream from the application server, and suspending outputting the second during the outputting of the second video code stream. Video stream.

The method of claim 10 or 11, wherein the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified, the method further comprising:

Receiving a real-time transmission control protocol RTCP request sent by the terminal, where the RTCP request includes a parameter set type of the parameter set ID to be modified;

Returning an RTCP response to the terminal, where the RTCP response carries the parameter set to be modified

The offset of the original parameter set ID corresponding to the parameter set type of the ID, or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or carried and to be modified The original parameter set ID of the parameter set type of the parameter set ID and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID.

13. The method of claim 10 or 11, wherein the parameter set ID is modified The information includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set type including the parameter set ID to be modified. And a parameter set ID of a parameter set for decoding the VCL NALU, or the original parameter set ID of the parameter set type including the parameter set ID to be modified, and a parameter for decoding the VCL NALU corresponding to the original parameter set ID a parameter set ID of the set, the parameter set ID modification information further comprising a modification effective flag, the obtaining, before acquiring the parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU The method further includes:

And transmitting, by the terminal, the parameter set identifier modification message of the first video that is not effective when the second video code stream is sent;

Receiving, by the terminal, a second media description information acquisition request;

Returning, to the terminal, a second media description information acquisition response, where the second media description information acquisition response carries the modified parameter set of the first video, where the parameter set ID of the modified parameter set of the first video is used for The parameter set ID of the parameter set that decodes the VCL NALU.

The method according to any one of claims 9-13, wherein the parameter set identification modification message of the first video is: a parameter set identifier modification supplementary enhancement information SEI message, an adaptive parameter set identifier modification SEI The message or picture parameter set identifier modifies the SEI message, the method further includes: generating a parameter set identifier modification of the first video according to the first video parameter set modification information

15. A method of splicing a digital video code stream, comprising:

Receiving the media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the parameter set of the first video and the parameter set corresponding to each type Original parameter set ID;

Receiving the media description information of the second video sent by the second content server, where the media description information of the second video carries the parameter set of the second video, the type of the second video parameter set, and the parameter set of the parameter set corresponding to each type. ID;

Determining that the parameter set ID of the first video of the same type is the same as the parameter set ID of the parameter set of the second video, and determining parameter set identification modification information of the first video;

Sending information of the first video stream to the splicer, where the information of the first video code stream includes the first view The parameter set identifier of the frequency identifies the modification information, and the parameter set modification identifier information of the first video is carried in the parameter set identifier modification message of the first video, so that the splicer sends the parameter set identifier modification message of the first video to the splicer a terminal, and in order for the terminal to modify the message according to the parameter set identification modification of the first video and output the first video.

The method according to claim 15, wherein the method further comprises: modifying the original parameter set ID of the original parameter set of the first video to be used according to the determined parameter set identification modification information of the first video. a parameter set ID of a parameter set of a video coding layer VCL NALU for decoding a network abstraction layer unit;

Transmitting, to the terminal, the modified parameter set of the first video, where the parameter set ID in the modified parameter set is the parameter set ID of the parameter set used for decoding the VCL NALU;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of an original parameter set ID of the first video corresponding to the parameter set type of the parameter set ID to be modified, Or a parameter set type including the parameter set ID to be modified and a parameter set ID for decoding a parameter set of the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identifier modification of the first video The message includes a parameter set ID of the parameter set type of the parameter set ID to be modified, the original parameter set ID, and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU;

17. The method according to claim 15, wherein the method further comprises: transmitting, to the terminal, an original parameter set of the first video and a parameter of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID. Set the ID, the original parameter set of the first video carries the original parameter set ID;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or The parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID of the parameter set type of the parameter set ID to be modified, and a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the original parameter set ID; The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU.

18. A decoding apparatus for a digital video stream, comprising:

a receiving module, configured to receive a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier modification message of the first video includes a parameter set identifier ID modification of the first video

'harm,,

a first obtaining module, configured to acquire, according to the parameter set ID modification information of the first video received by the receiving module, a parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit;

a second acquiring module, configured to acquire, according to the parameter set ID of the parameter set for decoding the VCL NALU acquired by the first acquiring module, a parameter set used for decoding the VCL NALU;

And a decoding output module, configured to decode the VCL NALU according to the parameter set for decoding the VCL NALU acquired by the second acquiring module, and output the decoded first video.

The device according to claim 18, wherein the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original corresponding to the parameter set type of the parameter set ID to be modified. An offset of the parameter set ID; the first video code stream includes an original parameter set ID for decoding a parameter set of the VCL NALU;

The first acquiring module is configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, a parameter set from the first code stream and the parameter set ID to be modified. The original parameter set ID corresponding to the type;

Obtaining, according to the original parameter set ID and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, obtaining the VCL corresponding to the parameter set type of the parameter set ID to be modified The parameter set ID of the NALU's parameter set.

20. The apparatus according to claim 18, wherein the parameter set ID modification information comprises a parameter set type of a parameter set ID to be modified and a parameter set ID of a parameter set for decoding a VCL NALU, the An obtaining module is configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, the parameter set type corresponding to the parameter set ID to be modified from the parameter set ID modification information. The parameter set ID of the parameter set used to decode the VCL NALU.

The device according to claim 18, wherein the parameter set ID modification information is an original parameter set ID of a parameter set type of the parameter set ID to be modified, and corresponds to the original parameter set ID. Parameter set ID of the parameter set used to decode the VCL NALU;

The first acquiring module is configured to: obtain the original parameter set ID corresponding to the original parameter set ID according to the parameter set type of the parameter set ID to be modified received by the receiving module, and use the original parameter set ID to decode the VCL. The parameter set ID of the NALU's parameter set.

The device according to claim 18, wherein the first video parameter set ID modification information is a parameter set type of a parameter set ID to be modified, and the device further includes:

a sending module, configured to send a real-time transmission control protocol RTCP request to the splicer, where the RTCP request includes a parameter set type of the parameter set ID to be modified;

The receiving module is further configured to: receive an RTCP response returned by the splicer, where the RTCP response carries an offset of an original parameter set ID corresponding to a parameter set type of a parameter set ID to be modified, or is to be modified a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID, or the original parameter set ID of the parameter set type of the parameter set ID to be modified, and the original parameter set ID a parameter set ID for decoding a parameter set of the VCL NALU; the first obtaining module is specifically configured to: correspond to a parameter set type of the parameter set ID to be modified carried in the RTCP response received by the receiving module The offset of the original parameter set ID or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the original according to the parameter set type of the parameter set ID to be modified a parameter set ID and a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the original parameter set ID, and obtaining a parameter set from the parameter set ID modification information and the parameter set ID to be modified Type for the corresponding parameter set parameter set decoding VCL NALU's ID.

The device according to any one of claims 19-21, wherein the parameter set ID modification information further comprises a modification effective flag.

The receiving module is further configured to: before receiving the parameter set identification modification message of the first video code stream and the first video, receiving a parameter set identifier modification message of the first video that is not valid;

The device further includes: determining, by the determining module, that the parameter set identifier modification message of the inactive first video received by the receiving module includes flag information indicating that the parameter set ID modification information is not valid;

a sending module, configured to send a second media description information acquisition request to the application server; the receiving module is further configured to: receive a second media description information acquisition response returned by the application server, where the second media description information acquisition response carries Two video parameter sets, the first video The modified parameter set, the parameter set ID of the first video modified parameter set is the parameter set ID of the parameter set for decoding the VCL NALU.

24. Apparatus according to any of claims 18-23 wherein:

The receiving module is further configured to acquire the first video by using an outband manner before the second obtaining module acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU. a modified parameter set, the parameter set ID of the modified parameter set is a parameter set ID of a parameter set for decoding the VCL NALU; or

The receiving module is further configured to acquire the first video by using an outband manner before the second obtaining module acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU. The original parameter set and the parameter set ID of the parameter set for decoding the VCL NALU; the original parameter set ID is modified according to the parameter set ID of the parameter set for decoding the VCL NALU.

25. Apparatus according to any of claims 18-24 wherein:

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU; the first obtaining module is specifically configured to: acquire a parameter set type that is directly referenced by the receiving module and the VCL NALU Corresponding parameter set ID of a parameter set directly used by the VCL NALU for decoding the VCL NALU, and obtained by the second obtaining module according to the parameter set ID of the parameter set directly referenced by the obtained VCL NALU for decoding the VCL NALU After the parameter set directly referenced by the VCL NALU in the first video bitstream, the first obtaining module further obtains, from the directly referenced parameter set, a parameter set ID of a parameter set for invoking the VCL NALU indirect reference of the VCL NALU; or

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU and a parameter set type indirectly referenced by the VCL NALU, and the first obtaining module is specifically configured to: according to the first The parameter set ID modification information of a video acquires a parameter set ID of a parameter set of a direct reference for decoding a VCL NALU of the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, and obtains an indirect reference with the VCL NALU The parameter set ID of the parameter set corresponding to the parameter set type used to decode the VCL NALU;

The second obtaining module is configured to: obtain a parameter set directly referenced by the VCL NALU according to a parameter set ID of the directly referenced parameter set acquired by the first obtaining module, and a parameter set ID of the indirectly referenced parameter set; a parameter set of the indirect reference of the VCL NALU; The decoding output module is specifically configured to: decode the VCL NALU according to a directly referenced parameter set of the VCL NALU acquired by the second obtaining module, and an indirectly referenced parameter set of the VCL NALU, and output the decoded The first video.

26. A splicing apparatus for a digital video code stream, comprising:

a receiving module, configured to receive information of a first video code stream from an application server, where the first video code stream information includes parameter set modification information of the first video;

a sending module, configured to send, according to the first video parameter set modification information received by the receiving module, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier of the first video is modified The message carries the parameter set identifier ID modification information of the first video, so that the terminal acquires the parameter set ID of the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal Acquiring a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, and decoding, by the terminal, the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded The first video.

27. The apparatus according to claim 26, wherein the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified.

The receiving module is further configured to receive a real-time transmission control protocol RTCP request sent by the terminal, where the RTCP request includes a parameter set type of the parameter set ID to be modified;

The sending module is further configured to: return an RTCP response to the terminal, where the RTCP response carries an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or carries a parameter set to be modified. a parameter set ID of a parameter set for decoding a VCL NALU corresponding to a parameter set type of the ID, or the original parameter set ID carrying a parameter set type of the parameter set ID to be modified and a corresponding to the original parameter set ID The parameter set ID of the parameter set used to decode the VCL NALU.

The apparatus according to claim 26, wherein the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original corresponding to the parameter set type of the parameter set ID to be modified. The offset of the parameter set ID, or the parameter set type including the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU, or the parameter set type including the parameter set ID to be modified The original parameter set ID and the decoding corresponding to the original parameter set ID are used for decoding The parameter set ID of the parameter set of the VCL NALU, the parameter set ID modification information further includes a modification effective flag, and the sending module further uses the first video code stream and the parameter set identification modification message of the first video before transmitting to the terminal And sending, by the terminal, the parameter set identifier modification message of the first video that is not effective when the second video code stream is sent;

The receiving module is further configured to receive a second media description information acquisition request sent by the terminal, where the sending module is further configured to return a second media description information acquisition response to the terminal, where the second media description information acquisition response carries the first The modified parameter set of the video, the parameter set ID of the modified parameter set of the first video is a parameter set ID of a parameter set for decoding the VCL NALU.

29. Apparatus according to any of claims 26-28, wherein the apparatus further comprises:

And a generating module, configured to generate a parameter set identifier modification message of the first video according to the first video parameter set modification information.

30. A splicing device for a digital video stream, characterized in that:

The first receiving module is configured to receive the media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the type of the parameter set of the first video and The original parameter set ID of the parameter set corresponding to each type;

a second receiving module, configured to receive media description information of the second video sent by the second content server, where the media description information of the second video carries a parameter set of the second video, a type of the second video parameter set, and each The parameter set ID of the parameter set corresponding to the type;

a determining module, configured to determine that an original parameter set ID of the first video received by the first receiving module of the same type is the same as a second video parameter set ID received by the second receiving module, and determine a parameter set of the first video Identify the modification information;

a sending module, configured to send information of the first video stream to the splicer, where the information of the first video code stream includes parameter set modification information of the first video, and the parameter set modification information of the first video is carried in the first video The parameter set identifier is modified in the message, so that the splicer sends the parameter set identification modification message of the first video to the terminal, and the terminal is configured to modify the message according to the parameter set identifier of the first video to decode and output the first video. .

The device of claim 30, wherein the device further comprises: a modifying module, configured to determine, according to the determining module, that the parameter set identification modification information is to be first The original parameter set ID of the original parameter set of the video is modified to a parameter set ID of a parameter set for decoding the video coding layer VCL NALU of the network abstraction layer unit;

32. Apparatus according to claim 31 wherein:

The sending module is further configured to: send, to the terminal, an original parameter set of the first video and a parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID, where the original parameter of the first video is concentrated Carrying the original parameter set ID;

The parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or The parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID in a parameter set type of the parameter set ID to be modified, and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU;

The parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU.