WO2014047938A1 - Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil - Google Patents

Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil Download PDF

Info

Publication number
WO2014047938A1
WO2014047938A1 PCT/CN2012/082483 CN2012082483W WO2014047938A1 WO 2014047938 A1 WO2014047938 A1 WO 2014047938A1 CN 2012082483 W CN2012082483 W CN 2012082483W WO 2014047938 A1 WO2014047938 A1 WO 2014047938A1
Authority
WO
WIPO (PCT)
Prior art keywords
parameter set
video
modified
decoding
type
Prior art date
Application number
PCT/CN2012/082483
Other languages
English (en)
Chinese (zh)
Inventor
张园园
杨晓峰
石腾
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2012/082483 priority Critical patent/WO2014047938A1/fr
Priority to CN201280002009.5A priority patent/CN103959796B/zh
Publication of WO2014047938A1 publication Critical patent/WO2014047938A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement

Definitions

  • the present application relates to digital video splicing techniques, and more particularly to a decoding method, a splicing method and apparatus for a digital video stream. Background technique
  • the code stream splicing technology has a wide range of application scenarios. For example, operators can select personalized advertisements according to user preferences, and replace the universal advertisements in the program with personalized advertisements by stream stream splicing technology to improve the hit rate and viewing effect of advertisements, thereby improving user experience and operator revenue.
  • a video content can be divided into one or more encoded video sequences (CVS: Coded Video Sequence); one CVS contains multiple pictures (Picture); one picture can be divided into one or more pictures (Slice One or more encoded pictures at the same time are called an access unit (AU: Access Unit).
  • AU Access Unit
  • NALU network abstraction layer unit
  • Network Abstraction Layer Unit is composed of units.
  • NALU is divided into two categories: Video Coding Layer (VCL: Video Coding Layer) NALU and non-VCL NALU.
  • VCL Video Coding Layer
  • Non-VCL NALU can be further divided into: Parameter set NALU: NALU for encapsulating parameter sets, Supplemental Enhancement Information (SEI)
  • SEI Supplemental Enhancement Information
  • NALU NALU for encapsulating SEI messages.
  • the parameter set NALU includes a parameter set and a parameter set identifier corresponding to the parameter set (ID: Identifier). If the parameter set also references other parameter sets, the parameter set further includes a parameter set corresponding to the parameter set of the bow I. ID.
  • the parameter set contains header information shared by multiple VCL NALUs, such as profile and level information, video color format information, video resolution information, and the like. Parameter sets can be divided into multiple types, and different types of parameter sets contain different types of shared header information.
  • a video stream can contain one or more parameter sets of the same type.
  • a parameter set contains a parameter set ID, and the value of the parameter set ID is unique in a parameter set of the same type of a code stream.
  • Shared header information After being placed in the parameter set, the corresponding VCL NALU refers to the header information in the parameter set through the parameter set ID.
  • the parameter set was introduced by the video coding standard H.264 and is widely used by subsequent video coding standards. The purpose of introducing a parameter set is for robust and efficient transmission of header information. After the parameter set is introduced, the header information shared by multiple VCL NALUs needs to be transmitted only once, thus improving the transmission efficiency.
  • the parameter sets defined by the H.264 standard are: Sequence Parameter Set (SPS) and PPS.
  • SPS Sequence Parameter Set
  • MPEG Moving Pictures Experts Group
  • MPEG Moving Pictures Experts Group
  • VPS Video Parameter Set
  • Two parameter sets of APS Adaptation Parameter Set
  • the parameter set defined by MPEG's 3DVC Three-Dimensional Video Coding standard, in addition to SPS and PPS, includes depth range parameters.
  • Set DRPS: Depth Range Parameter Set).
  • the parameter sets defined by different video coding standards can be divided into two categories: the parameter set directly referenced by VCL NALU in the video stream and the parameter set indirectly referenced by VCL NALU. If the parameter set is a parameter set directly referenced by VCL NALU, the VCL NALU contains the parameter set ID of the parameter set; if the parameter set is a parameter set indirectly referenced by VCL NALU, the parameter set ID of the parameter set is not included in the VCL NALU, and The parameter set ID of the parameter set is included in the directly referenced parameter set of the VCL NALU or in the indirectly referenced parameter set.
  • the bitstream splicing technology receives a first video code stream (such as personal advertisement content, etc.) and a second video code stream (such as a program requested by a terminal, etc.) for the splicer.
  • the splicer sends the first video code stream to the terminal during the splicing period, and sends the second video code stream to the terminal at other time periods in the process of outputting the second video code stream for the terminal.
  • the second video and the first video are independently encoded using different encoders (second encoder, first encoder). Since the parameter set identifier (ID: Identifier) is entropy coded, the smaller the value of the parameter set ID is, the smaller the number of bits required to represent the parameter set ID is.
  • ID parameter set identifier
  • the encoder assigns the parameter set ID, it is often allocated from 0 value. Therefore, the parameter sets ID assigned by the two encoders to the same type of parameter set may be the same.
  • the second video code stream is the same as the parameter set ID of the same type of parameter set in the first video code stream, and the parameter set ID in the output video code stream cannot uniquely identify a parameter set.
  • the parameter set adopts the out-of-band transmission mode it will cause the wrong parameter set to be used in decoding, resulting in decoding errors.
  • a first The picture parameter set (PPS: Picture Parameter Set) ID in the network abstraction layer unit of the video coding layer (VCL: Video Coding Layer) of the video is the same as the PPS ID of a certain PPS of the second video.
  • the decoder may decode the first video VCL NALU using the PPS of the second video, resulting in a decoding error.
  • the prior art modifies the parameter set ID in the first video bitstream by the splicer to ensure the uniqueness of the parameter set ID in the output video bitstream.
  • the terminal receives the output video stream, decodes each VCL NALU, and obtains a parameter set referenced by the VCL NALU from the parameter set obtained by the out-of-band method according to the parameter set ID, and uses these parameter sets to decode the VCL NALU.
  • the prior art requires the splicer to have entropy decoding and entropy coding capabilities.
  • splicers in existing networks usually do not have entropy decoding and entropy coding capabilities.
  • Entropy decoding and entropy coding operations require more computing resources and storage resources, and parameter set IDs appear frequently in video streams, ie, splicing.
  • the device requires frequent entropy decoding and entropy encoding operations.
  • the invention provides a decoding and splicing method and device for a digital video code stream, which not only ensures that the correct parameter set is used when the terminal decodes the first video code stream, but also does not need to add extra on the splicer of the existing network.
  • the hardware resources, or the replacement of splicers in existing networks with new splicers, are less expensive to operate than in the prior art.
  • a first aspect of the embodiments of the present invention provides a method for decoding a digital video code stream, including: receiving a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video includes a parameter set identifier ID modification information of the first video; acquiring a parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit according to the parameter set ID modification information of the first video; Decoding a parameter set ID of a parameter set of the VCL NALU to obtain a parameter set for decoding the VCL NALU; decoding the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded first video.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original parameter corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set ID of the parameter set of the NALU is specifically obtained by acquiring, according to the parameter set type of the parameter set ID to be modified, a location corresponding to the parameter set type of the parameter set ID to be modified from the first video code stream.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and is used for decoding the VCL.
  • the parameter set ID modification information acquires a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set ID modification information includes an original parameter set ID of a parameter set type of the parameter set ID to be modified, and a corresponding to the original parameter set ID. Decoding the parameter set ID of the parameter set of the VCL NALU;
  • the parameter set ID of the parameter set of the NALU is specifically: acquiring a parameter set of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified. ID.
  • the first video parameter set ID modification information is a parameter set type of the parameter set ID to be modified
  • the method further includes: sending a real-time transmission control protocol RTCP request to the splicer
  • the RTCP request includes a parameter set type of the parameter set ID to be modified; receiving an RTCP response returned by the splicer, where the RTCP response carries an original parameter set corresponding to a parameter set type of a parameter set ID to be modified.
  • the parameter set ID of the parameter set for decoding the VCL NALU according to the parameter set ID modification information of the first video is specifically: according to the The parameter set of the parameter set for decoding the VCL NALU corresponding to the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified carried in the RTCP response or the parameter set type of the parameter set ID to be modified ID, or from the parameter set ID modification information according to the original parameter set ID of the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID Obtaining a parameter set ID of a parameter set for decoding the VCL NALU
  • the method further includes: by means of an out-of-band method, before obtaining a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU Obtaining a modified parameter set of the first video, where the parameter set ID of the modified parameter set is a parameter set ID of a parameter set used for decoding the VCL NALU; or acquiring an original parameter set of the first video by using an outband manner; a parameter set ID for decoding a parameter set of the VCL NALU, the original parameter set of the first video carries the original parameter set ID; modifying the original parameter according to the parameter set ID of the parameter set for decoding the VCL NALU Set ID.
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and the information acquisition is modified according to the parameter set ID of the first video.
  • the parameter set ID of the parameter set for decoding the VCL NALU is specifically: obtaining a parameter set ID of a parameter set for directly decoding a VCL NALU directly referenced by the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, the method further comprising Obtaining, according to the obtained parameter set ID of the parameter set directly used by the VCL NALU for decoding the VCL NALU, the parameter set directly referenced by the VCL NALU in the first video code stream; obtaining the decoding from the directly referenced parameter set
  • the parameter set ID of the parameter set indirectly referenced by the VCL NALU of the VCL NALU; or the parameter set type of the parameter set ID to be modified is the parameter set type directly referenced by the VCL NALU and the parameter set indirectly referenced by the VCL
  • Decoding the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded first video is specifically: a parameter set according to the obtained directly referenced VCL NALU and an indirect of the VCL NALU
  • the referenced parameter set decodes the VCL NALU and outputs the decoded first video.
  • a second aspect of the embodiments of the present invention further provides a method for splicing a digital video code stream, including: receiving information of a first video code stream from an application server, where information of the first video code stream includes parameters of a first video stream And modifying the information according to the first video parameter set, and sending, by the first video parameter set, the first video code stream and the parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video carries the parameter of the first video And the identifier ID modification information is obtained, so that the terminal acquires a parameter set ID of a parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal is used to decode the VCL according to the
  • the parameter set ID of the parameter set of the NALU acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the de
  • the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified
  • the method further includes: receiving a real-time transmission control protocol RTCP request sent by the terminal
  • the RTCP request includes the parameter set type of the parameter set ID to be modified; and returns an RTCP response to the terminal, where the RTCP response carries the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an original parameter corresponding to the parameter set type of the parameter set ID to be modified.
  • the method further includes: sending a parameter set identifier modification message of the first video that is not valid when the second video code stream is sent to the terminal; receiving a second media description information acquisition request sent by the terminal; returning the second to the terminal.
  • a third aspect of the embodiments of the present invention provides a method for splicing a digital video code stream, including: receiving media description information of a first video sent by a first content server, where media description information of the first video carries a first video The original parameter set and the parameter set of the first video and the original parameter set ID of the parameter set corresponding to each type; the media description information of the second video sent by the second content server, and the media description information of the second video The parameter set of the second video parameter set carrying the second video and the parameter set ID of the parameter set corresponding to each type; determining the parameter set ID of the first video of the same type and the parameter set of the second video The set ID is the same, and the parameter set identification modification information of the first video is determined; the information of the first video stream is sent to the splicer, where the information of the first video code stream includes parameter set identification modification information of the first video, where The parameter set modification identifier information of a video is carried in the parameter set identifier modification message of the first video, so that the splic
  • a fourth aspect of the embodiments of the present invention provides a decoding apparatus for a digital video code stream, including: a receiving module, configured to receive a first video code stream and a parameter set identifier modification message of the first video, where the parameter set of the first video
  • the identifier modification message includes parameter set identifier ID modification information of the first video
  • the first obtaining module is configured to obtain a video for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video received by the receiving module.
  • a parameter set ID of a parameter set of the coding layer VCL NALU configured to acquire a parameter for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU acquired by the first acquisition module
  • a decoding output module configured to decode the VCL NALU according to the parameter set for decoding the VCL NALU acquired by the second acquiring module, and output the decoded first video.
  • the parameter set ID modification information packet And a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified;
  • the first video code stream includes the VCL for decoding The original parameter set ID of the parameter set of the NALU;
  • the first acquiring module is specifically configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, from the first code stream Determining the original parameter set ID corresponding to the parameter set type of the modified parameter set ID; and offsetting the original parameter set ID corresponding to the original parameter set ID and the parameter set type of the parameter set ID to be modified Obtaining a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set ID modification information includes a parameter set type of a parameter set ID to be modified and a parameter set ID of a parameter set for decoding a VCL NALU, where the first
  • the obtaining module is specifically configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, the use of the parameter set type corresponding to the parameter set ID to be modified from the parameter set ID modification information.
  • the parameter set ID of the parameter set for decoding VCL NALU is specifically configured to: obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, the use of the parameter set type corresponding to the parameter set ID to be modified from the parameter set ID modification information.
  • the parameter set ID modification information is an original parameter set ID of a parameter set type of the parameter set ID to be modified, and the original parameter set ID is used for decoding.
  • a parameter set ID of a parameter set of the VCL NALU; the first obtaining module is configured to: obtain the information according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified received by the receiving module The parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID.
  • a fifth aspect of the embodiments of the present invention provides a splicing apparatus for a digital video code stream, including: a receiving module, configured to receive information of a first video code stream from an application server, where the first video code stream information includes a first video a parameter set modification information, the sending module, configured to send, according to the first video parameter set modification information received by the receiving module, a first video code stream and a parameter set identifier modification message of the first video, where the first The parameter set identifier modification message of the video carries the parameter set identifier ID modification information of the first video, so that the terminal acquires the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video.
  • the terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal decodes the VCL according to the parameter set for decoding the VCL NALU NALU and output the decoded first video.
  • a fifth aspect of the embodiments of the present invention provides a splicing apparatus for a digital video code stream, including:
  • the first receiving module is configured to receive the media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the type of the parameter set of the first video and The original parameter set ID of the parameter set corresponding to each type;
  • the second receiving module is configured to receive the media description information of the second video sent by the second content server, where the media description information of the second video carries the parameter of the second video a set, a type of the second video parameter set, and a parameter set ID of the parameter set corresponding to each type;
  • a determining module configured to determine an original parameter set ID of the first video received by the first receiving module of the same type, The second video parameter set ID received by the second receiving module is the same, and the parameter set identification modification information of the first video is determined;
  • the sending module is configured to send information of the first video stream to the splicer,
  • the parameter set identifier ID modification information of the first video is modified by the parameter set identifier of the first video, so that the terminal decodes and outputs the first video according to the parameter set identifier ID modification information, thereby eliminating the need for a splicer pair.
  • the parameter set ID in each VCL NALU of a video stream is modified, not only when the terminal decodes the first video stream, but also ensures that the correct parameter set is used, and there is no need to add extra on the splicer of the existing network.
  • Hardware resources, or the replacement of splicers in existing networks with new splicers, requires less operational cost than in the prior art.
  • FIG. 1 is a network system architecture diagram of a splicing of a typical digital video code stream according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of relationship between an output video code stream, a second video code stream, and a first video code stream according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of reference between a VCL NALU and a parameter set of a video code stream complying with the HEVC standard according to an embodiment of the present invention
  • 4 is a flowchart of a method for decoding a digital video code stream according to an embodiment of the present invention
  • FIG. 5 is a flowchart of a method for splicing a digital video code stream according to an embodiment of the present invention
  • FIG. 7 is a flowchart of another method for splicing a digital video stream according to an embodiment of the present invention
  • FIG. 8 is a parameter set of a real-time transmission control protocol according to an embodiment of the present invention
  • RTCP PSIC-R Real-time Transport Control Protocol Parameter Set ID Change - Request
  • FIG. 9 is a schematic structural diagram of feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure.
  • FIG. 10 is another feedback control information provided by an embodiment of the present invention (FCI: Feedback Control)
  • FIG. 11 is a schematic structural diagram of still another feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure
  • FIG. 12 is a schematic structural diagram of still another feedback control information (FCI: Feedback Control Information) according to an embodiment of the present disclosure
  • FIG. 13 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention
  • FIG. 14 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention
  • FIG. 15 is a schematic structural diagram of a digital video code stream decoding apparatus according to an embodiment of the present invention
  • FIG. 16 is a schematic structural diagram of a digital video code stream splicing apparatus according to an embodiment of the present invention
  • FIG. 18 is a schematic structural diagram of a decoding apparatus or a splicing apparatus for a digital video code stream according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of a code stream splicing system according to an embodiment of the present invention, which mainly includes a second encoder, a first encoder, a second content server, a first content server, a splicer, a router, and a terminal.
  • the second encoder is connected to the second content server, and the second content server provides the video code stream output by the second encoder as a second video code stream to the splicer.
  • the first encoder is coupled to the first content server, and the first content server provides the video stream output by the first encoder as a first video stream to the splicer.
  • Splicer The second video stream and the first video stream are received from the second content server and the first content server, respectively. During the splicing period, the first video code stream is sent to the terminal, and the second video code stream is sent to the terminal at other time periods.
  • the video code stream (including the second video code stream and the first video code stream) sent by the splicer to the terminal is collectively referred to as an output video code stream.
  • FIG. 2 is a schematic diagram showing a relationship between an output video code stream and a second video code stream and a first video code stream according to an embodiment of the present invention. During the splicing period, the output video stream is the first video stream; at other times, the output video stream is the second video stream.
  • the starting point of the splicing period that is, the point at which the first video stream is outputted, is referred to as a splicing point;
  • the end point of the splicing period that is, the point at which the end of the output of the first video stream is ended, is referred to as a return point.
  • the splicer is usually located on a physical entity at the edge of the Internet Protocol (IP) network, such as a router or an edge media server.
  • IP Internet Protocol
  • These physical entities are usually not as intelligent as encoders and decoders, such as entropy coding and/or entropy decoding, unlike encoders and decoders.
  • the splicer and content server can be on the same physical entity or on different physical entities.
  • the application server respectively receiving media description information sent by the second content server and the first content server, and sending media description information for decoding the output video code stream to the terminal (the media description information includes parameters for decoding the received video stream) Set and parameter set ID).
  • the router acts as a device for transmitting routes.
  • the terminal receives the output video code stream sent by the splicer according to the received media description information sent by the application server, and decodes and displays the output video code stream.
  • FIG. 3 is a schematic diagram of reference between a VCL NALU and a parameter set of a video code stream complying with the HEVC standard according to an embodiment of the present invention. As shown in FIG.
  • the video code stream conforms to the HEVC standard
  • the VPS, the SPS, the PPS, and the APS respectively include the parameter set ID corresponding to the parameter set, that is, the VPS ID, the SPS ID, the PPS ID, and the APS ID, respectively.
  • PPS and APS are the parameter sets directly referenced by VCL NALU.
  • the parameters contained in PPS and APS can be directly referenced by all code fragments NALU (that is, VCL NALU) of the same picture or pictures.
  • Each VCL NALU contains the PPS ID and APS ID.
  • SPS, VPS is a parameter set indirectly referenced by VCL NALU.
  • the parameters contained in SPS can be referenced by one or more PPSs. Therefore, the PPS also contains the SPS ID.
  • the corresponding VCL NALU indirectly references the SPS through the PPS.
  • the parameters contained in the VPS can be One or more SPS references, so the SPS also contains the VPS ID, and the corresponding VCL NALU indirectly references the SPS through the PPS, and indirectly references the VPS through the SPS.
  • the value of the parameter set ID is an unsigned integer, and most parameter set IDs are entropy encoded, such as unsigned exponential Columbus encoding.
  • the purpose of entropy coding is to improve coding efficiency.
  • the characteristic of entropy coding is that the codeword obtained by coding is variable in length, and the smaller the value, the smaller the number of bits required to represent the value.
  • the application server transmits the parameter set transmission to the terminal, it needs to ensure that the decoder located in the terminal receives the parameter set referenced by the VCL NALU before receiving the VCL NALU, including the directly referenced parameter set and the indirectly referenced parameter set, so the parameter Sets are usually transmitted in an out-of-band manner.
  • the parameter set is used as a kind of media description information, which is sent to the terminal before the session is established or during the session establishment process. If the parameter set has changed during the session, the parameter set is sent to the terminal during the session change or through the notification message in the session.
  • There are various encapsulation formats for parameter sets including but not limited to the parameters of the Session Description Protocol (SDP), and the elements of the Extensible Markup Language (XML).
  • SDP Session Description Protocol
  • XML Extensible Markup Language
  • the sending of the parameter set can be based on a variety of protocol methods, including but not limited to Session Initiation Protocol (SIP), Invite, Re-Invite, SIP, and SIP notification. Info), Real Time Streaming Protocol (RTSP) Description (Describe), RTSP Notification (Notify), RTSP Play Notification (Play-Notify), RTSP Set Parameter (Set Parameter), RTSP Get Parameter (Get Parameter) Hypertext Transfer Protocol (HTTP) acquires one of the protocol methods (Get) and HTTP (Post).
  • SIP Session Initiation Protocol
  • Invite Invite
  • Re-Invite Session Initiation Protocol
  • SIP Session Initiation Protocol
  • SIP Session Initiation Protocol
  • RTSP Real Time Streaming Protocol
  • RTSP Notification Notify
  • RTSP Play Notification Play-Notify
  • HTTP Hypertext Transfer Protocol
  • FIG. 4 is a schematic diagram of a method for decoding a digital video code stream according to an embodiment of the present invention, including the following steps Step:
  • Step 401 Receive a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier of the first video includes a parameter set identifier ID modification information of the first video.
  • the terminal receives the splicing The first video code stream sent by the device and the parameter set identifier modification message of the first video, the parameter set ID modification information including a parameter set type of the parameter set ID to be modified and a parameter set with the parameter set ID to be modified An offset of the original parameter set ID corresponding to the type; the first video code stream includes an original parameter set ID of a parameter set for decoding a video coding layer VCL NALU of the network abstraction layer unit.
  • the terminal receives the first video code stream sent by the splicer and the parameter set identification modification message of the first video, where the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified, and The parameter set ID of the parameter set for decoding VCL NALU.
  • the terminal receives the first video code stream sent by the splicer and the parameter set identifier modification message of the first video, where the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified.
  • the parameter set identification modification message of the first video is an SEI message generated by a splicer and/or in a manner of generating a supplemental enhanced SEI message, specifically a parameter set identifier modification supplementary enhanced SB message, and an adaptive parameter set identifier modification
  • the SEI message, the picture parameter set identifier modifies the SEI message.
  • the SEI message may also be another type of SEI message, which is not limited herein.
  • the parameter set identification modification message of the first video is sent to the terminal together with the first video code stream.
  • Step 402 Acquire a parameter set ID for decoding a parameter set of the VCL NALU according to the parameter set ID modification information of the first video (that is, a network abstraction layer unit VCL for decoding a video coding layer in the first video code stream.
  • the parameter set ID of the parameter set of the NALU that is, the modified parameter set ID.
  • the modified parameter set ID refers to the video coding layer used for decoding the first video code stream.
  • the parameter set ID of the parameter set of the network abstraction layer unit VCL NALU that is, the parameter set ID of the parameter set used to decode the VCL NALU);
  • the terminal acquires an original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified from the first video code stream according to the parameter set type of the parameter set ID to be modified; according to the original parameter
  • the set ID and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified acquire the VCL corresponding to the parameter set type of the parameter set ID to be modified.
  • the terminal directly according to the parameter set to be modified.
  • the parameter set type of the ID obtains, from the parameter set ID modification information, a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.
  • the VCL NALU is decoded.
  • the terminal according to the parameter set type of the parameter set ID to be modified, the decoding corresponding to the parameter set type of the parameter set ID to be modified in the first video code stream for decoding
  • the parameter set ID of the parameter set of the VCL NALU; the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID is obtained according to the original parameter set ID.
  • the parameter set type of the parameter set ID to be modified includes at least one original parameter set ID
  • the parameter set type of the parameter set ID to be modified needs to be respectively corresponding to each original parameter set ID.
  • the parameter set ID modification information includes the original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter set of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID
  • the ID of the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID is obtained by the terminal according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • Step 403 Acquire a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.
  • the terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.
  • Step 404 Decode the VCL according to the parameter set used to decode the VCL NALU.
  • NALU and output the decoded first video.
  • the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.
  • the first video parameter set ID modification information may also be a parameter set type of the parameter set ID to be modified
  • the terminal sends a real-time transmission control protocol RTCP request to the splicer.
  • the RTCP request includes the parameter set type of the parameter set ID to be modified; the terminal receives the RTCP response returned by the splicer, and the RTCP response carries the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • the set ID is specifically: according to the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified carried in the RTCP response or the parameter set type corresponding to the parameter set ID to be modified, for decoding
  • the parameter set ID modification information further includes a modification effective flag
  • the method further includes: the receiving is not effective. Determining, by the parameter set of the first video, a modification message; determining that the parameter set identifier modification message of the first video that is not in effect includes flag information indicating that the parameter set ID modification information is not valid; and sending a second media description information acquisition request to the application server; And receiving, by the application server, a second media description information acquisition response, where the second media description information acquisition response carries the modified parameter set of the first video, where the parameter set ID of the parameter set in the first video modification is The parameter set ID of the parameter set used to decode the VCL NALU.
  • the method before the acquiring the parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, the method further includes: acquiring the first video by using an outband manner.
  • the modified parameter set, the parameter set ID of the modified parameter set is a parameter set ID of a parameter set for decoding the VCL NALU; or, the original parameter set of the first video is obtained by an out-of-band method and used for decoding a parameter set ID of a parameter set of the VCL NALU, the original parameter set of the first video carries the original parameter set ID; and the original parameter set ID is modified according to the parameter set ID of the parameter set for decoding the VCL NALU.
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and the information acquisition is modified according to the parameter set ID of the first video.
  • the parameter set ID of the parameter set for decoding the VCL NALU is specifically: obtaining a parameter set ID of a parameter set for directly decoding a VCL NALU directly referenced by the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, the method further comprising Obtaining, according to the obtained parameter set ID of the parameter set directly used by the VCL NALU for decoding the VCL NALU, a parameter set directly referenced by the VCL NALU in the first video code stream; obtaining the decoding from the directly referenced parameter set The parameter set ID of the parameter set indirectly referenced by VCL NALU of VCL NALU; or
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU and a parameter set type indirectly referenced by the VCL NALU, and the parameter information is modified according to the parameter set ID of the first video.
  • the parameter set ID of the parameter set for decoding the VCL NALU is specifically: acquiring the VCL NALU for decoding the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU according to the parameter set ID modification information of the first video, respectively.
  • Obtaining the parameter set ID for decoding the VCL NALU in the parameter set ID of the parameter set of the decoded VCL NALU is specifically: acquiring the parameter set ID according to the obtained directly referenced parameter set and the parameter set ID of the indirectly referenced parameter set a directly referenced parameter set of the VCL NALU and a parameter set of the indirect reference of the VCL NALU;
  • Decoding the VCL NALU according to the parameter set for decoding the VCL NALU, and outputting the decoded first video is specifically: a parameter set according to the obtained directly referenced VCL NALU and an indirect of the VCL NALU
  • the referenced parameter set decodes the VCL NALU and outputs the decoded first video.
  • the parameter set identifier ID modification information of the first video is modified by the parameter set identifier of the first video, so that the terminal decodes and outputs the first video according to the parameter set identifier ID modification information instead of directly according to the first video.
  • the original parameter set ID in the code stream obtains a parameter set of the VCL NALU that decodes the first video code, thereby eliminating the need for a splicer for each VCL of the first video stream.
  • the parameter set ID in the NALU is modified, not only when the terminal decodes the first video stream, but also ensures that the correct parameter set is used, and there is no need to add additional hardware resources to the splicer of the existing network, or use a new one.
  • the splicer replaces the splicer in the existing network with less operational cost than is required in the prior art.
  • FIG. 5 is a flowchart of a method for splicing a code stream according to an embodiment of the present invention, including the following steps: Step 501: The second content server receives the second video code stream and the second video parameter set.
  • the second encoder encodes the second video content, and the parameter set ID can be obtained by entropy coding. And transmitting the outputted second video code stream and the acquired parameter set of the second video to the second content server.
  • Step 502 The first content server receives the first video code stream and the first video parameter set.
  • the first encoder encodes the first video content, and the parameter set ID may be acquired by entropy coding, and the output first video code stream and the acquired parameter set ID are included as the original parameter set ID.
  • the original parameters of the first video are sent to the first content server.
  • step 501 there is no temporal sequence between step 501 and step 502.
  • Step 503 The terminal sends a first media description information acquisition request message to the application server, where the first media description information acquisition request message includes a content identifier of the second content.
  • Step 504 The application server acquires media description information of the second content according to the content identifier of the second content. Including but not limited to the following information: Parameter set of the second video The type of the second video parameter set and the parameter set ID of the parameter set corresponding to each type.
  • An optional manner for the application server to obtain the media description information of the second content is: after the application server receives the media description information acquisition request message sent by the terminal, determining, according to the content identifier of the second content, that the content server where the second content is located is the second The content server, and sending a media description information acquisition request message to the second content server.
  • the media description information acquisition request message in step 503 is sent during the session establishment process, that is, a session establishment request message, specifically, a SIP Invite request message.
  • the media description information acquisition request message sent by the application server to the second content server is also a session establishment request message.
  • the second content server receives the media description information acquisition request message, determines the second video code stream, the parameter set NALU of the second video according to the content identifier of the second content, and allocates the second video code stream identification information for the second video code stream.
  • the media description information acquisition response message is a session establishment response message corresponding to the foregoing session establishment request message, such as a SIP 200 response message.
  • the media description information acquisition response message includes a parameter set of the second video and second video code stream identification information.
  • the application server obtains and stores the parameter set of the second video carried in the media description information acquisition response message.
  • the SDP is included in the media description information acquisition response message, and the parameter set NALU of the second video is included in the SDP.
  • the information in the parameter set NALU can be represented in another manner, such as using an SDP parameter or an XML element.
  • the embodiment of the present invention is not limited herein.
  • the SDP may further include a source IP address, a port, and a synchronization source identifier (S SRC Synchronization Source) of a real time transport protocol (RTP: Real Time Transport) stream carrying the second video stream, where the RTP stream carrying the second video stream is included.
  • RTP Real Time Transport
  • the media description acquisition request message in step 503 may also be used as an initialization media description information acquisition request message (such as an RTSP Describe request message, before the session is established). HTTP Get request message, etc.) is sent.
  • the media description information acquisition request message sent by the application server to the second content server is an initialization media description information acquisition request message, and the second content server receives the initialization media description information acquisition request message and sends the second video to the application server.
  • the media description information acquisition response message of the media description information is an initialization media description information acquisition response message.
  • the initial media description information acquisition response message may be an RTSP 200 response message, an HTTP 200 response message, or the like.
  • a further optional way for the application server to obtain the media description information of the second content is that the media description acquisition request message in step 503 is sent before the session is established, and before the step 504, the second content server is the second video code stream.
  • the second video code stream identification information is allocated, and the content identifier of the second content and the media description information of the second content are sent to the application server.
  • the media description information of the second content includes a parameter set of the second video and second video code stream identification information.
  • the application server may obtain the media description information of the second content from the second content server in another manner, which is not described in detail herein.
  • Step 505 The application server acquires media description information of the first content. Including but not limited to the following information: the original parameter set of the first video and the acquisition information of the first video.
  • An optional manner for the application server to obtain the media description information of the first content is that the application server sends the first video media to the first content server according to the configured address of the first content server. Describe the information acquisition request message.
  • the first content server receives the first video media description information acquisition request message sent by the application server, and sends the media description information of the first video to the application server, where the media description information of the first video includes at least the original parameter set of the first video and The information of the video stream, the type of the parameter set of the first video, and the original parameter set ID of the parameter set corresponding to each type.
  • the information of the first video stream may be used to obtain the first video from the first content server.
  • the RTSP uniform resource locator URL: Uniform Resource Locator
  • the terminal may further include the identifier of the user in the request message for sending the media description information sent to the application server in step 503, and the media sent by the application server to the first content server.
  • the obtaining request message of the description information carries the identifier of the user, so that the first content server can determine the first content for the user according to the identifier of the user, and determine the parameter set of the first video and the obtained information of the first video code stream. Sent to the application server.
  • Step 506 The application server modifies the original parameter set ID of the first video, where the original parameter set ID of the first video is the parameter set ID included in the original parameter set of the first video in step 505.
  • the application server determines, for each type of parameter set ID, whether the original parameter set ID of the first video of the type is the parameter set ID of the second video of the type. The same, if the same, you need to modify the original parameter set ID of the first video of this type. This process is described by taking the same original parameter set ID of the first video of the type and the parameter set ID of the second video of the type as an example.
  • the application server determines the offset of the original parameter set ID of the first video of the type, and adds the value of the original parameter set ID of the first video of the type plus the offset of the original parameter set ID as the first of the type.
  • the parameter set ID of the video for decoding the parameter set of the VCL NALU; or the application server directly assigns a new value to the original parameter set ID of the first video of the type, the new value of the allocation is the same as all the same type
  • the parameter set ID of the two videos is different, and the original parameter set IDs of all other first videos of the same type are also different, and the assigned new value is used as the parameter set ID of the parameter set for decoding the VCL NALU (ie, Decoding the parameter set ID of the parameter set of the VCL NALU in the first video bitstream).
  • the application server determines whether the original parameter set ID of the first video of the type is the same as the parameter set ID of the second video of the type, and the application server is the first type of the type.
  • the original parameter set ID of the video is compared to the parameter set ID of all second videos of the type, if the original parameter set ID of the first video of the type is the same as the parameter set ID of one of the second videos of the type.
  • the application server determines that the original parameter set ID of the first video of the type is the same as the parameter set ID of the second video of the type.
  • an optional manner for the application server to determine an offset of the original parameter set ID of the first video of the certain type is: the application server determines a maximum value of all parameter IDs of the second video of the type, all the types The minimum value in the original parameter set ID of the first video, the difference between the maximum value of the parameter set IDs of all the second videos and the minimum value of the original parameter set IDs of all the first videos is calculated, and the application server compares the difference The sum of the value and a positive integer is the offset of the original parameter set ID of the first video of that type. Specifically, as an alternative, the positive integer is 1.
  • the offset can be applied to the original parameter set ID of the first video of the type, and the original parameter set ID of the first video of the different types can be set to different offsets, which is not limited herein.
  • Step 507 The application server sends a first media description information acquisition response to the terminal, where the parameter set of the second video and the parameter set of the first video are included.
  • the SDP is included in the first media description information acquisition response message
  • the parameter set of the second video is included in the SDP (may be a set of parameter encapsulation parameters of the NALU, and may also be other forms, and the implementation of the present invention includes no special It is indicated that NALU is only used as an optional example and does not limit the encapsulation form of the parameter set), the modified parameter set of the first video.
  • the parameter set ID of the modified parameter set of the first video is the parameter set ID of the parameter set of the first video for decoding the VCL NALU, specifically, the value of the original parameter set ID of the first video plus the original parameter set.
  • the offset of the ID, or the value assigned by the application server directly to the parameter set ID of the parameter set of the first video used to decode the VCL NALU.
  • Steps 501-507 In the method for the terminal to acquire the second video parameter set and the first video parameter set, the terminal is a parameter set obtained from outside the video code stream, and thus may be referred to as an out-of-band mode (or called an external mode). Steps 501-507: The method for the terminal to acquire the second video parameter set and the first video parameter set is only an optional manner in which the terminal acquires the second video parameter set and the first video parameter set in an outband manner, and the terminal may also adopt another The method obtains the second video parameter set and the first video parameter set in an out-of-band manner, and the embodiment of the present invention is no longer detailed.
  • Step 508 The application server sends the second video code stream information and the first video code stream information to the splicer.
  • the second video code stream information includes the second video code stream identification information
  • the first video code stream information includes: the first video code stream acquisition information and the first video parameter set ID modification information (eg, The parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or the parameter set type of the parameter set ID to be modified and the application server to be modified
  • the parameter set ID that is, the parameter set ID of the parameter set for decoding the VCL NALU allocated by the application server for the parameter set ID to be modified
  • the parameter set ID modification information of the first video is the parameter set ID modification information of the parameter set type directly referenced by the VCL NALU of the first video, that is, the parameter set type of the parameter set ID of the first video to be modified can only be The parameter set type directly referenced by VCL NALU.
  • Step 508 is an example in which the parameter set modification information of the first video includes the parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • the acquiring information of the first video code stream may be an RTSP URL for acquiring the first video code stream to the first content server, where the second video code stream identification information may be an RTP stream for carrying the second video code stream.
  • the parameter set type of the parameter set ID of the first video to be modified may be a parameter set type represented by a three-byte text form, for example, vps,,, "sps,,," pps ,,,, "aps” respectively represent the parameter set types as VPS, SPS, PPS, and APS.
  • the parameter set ID offset can be a 1-byte unsigned integer.
  • the second video stream and the first video stream information can be as shown in Table 1 (ie, the video configuration information table):
  • the source IP address in the second video code stream identification information is 219.133.0.1, the port is 49170, and the SSRC is 1234.
  • the information about the first video stream corresponding to the second video stream is rtsp: ⁇ example. Com/fizzle/foo;
  • the application server determines the VPS, SPS, PPS in the first video,
  • the ID of the APS needs to be modified, and the offset of the original parameter set ID of each type of the first video is 1, that is, when the parameter set ID of the first video parameter set of the corresponding type is modified, all the original parameter set IDs are The value is incremented by 1 to the value of the parameter set ID of the corresponding parameter set for decoding the VCL NALU.
  • Table 1 only contains 2 parameter set ID modification information, that is, parameter set ID modification information of PPS and parameter set ID modification information of APS. It should be noted that the offset of the original parameter set ID in the above example is 1, for the sake of simplification of the example, and does not mean that the parameter set ID offsets of different types of parameter sets are the same.
  • steps 507 and 508 have no prioritization in time.
  • Step 509 The second content server sends a second video code stream to the splicer.
  • the splicer is located at the edge router, the destination IP address of the RTP stream carrying the second video stream, the port is a multicast IP address, and the port, and the RTP stream carrying the second video stream is sent to the splicing based on the network routing policy.
  • the second content server acquires a multicast IP address and port allocated by the network management system for the second video code stream.
  • the second content server obtains the IP address and port of the splicer.
  • the session establishment request message sent by the application server to the second content server in step 504 is taken as an example for description. Specifically, the second content server obtains the IP address and port of the splicer from the session establishment request message sent by the application server to the second content server in step 504.
  • Step 510 The splicer sends the second video code stream as an output video code stream to the terminal.
  • the port is the IP address of the splicer, and the port.
  • the splicer changes the destination IP address and port ⁇ in all IP packets carrying the second video stream to the IP address and port of the corresponding terminal.
  • Step 511 The terminal acquires VCL NALU from a parameter set obtained in an out-of-band manner (the out-of-band manner is from the second video stream) according to the parameter set ID in the VCL NALU of the received second video code stream.
  • the referenced parameter set ie, the parameter set of the VCL NALU used to decode the second video stream
  • Step 512 The splicer according to the second video code stream identification information from the video configuration information table, that is, Table 1 The obtaining information of the first video code stream is found, and the first video code stream is obtained according to the obtaining information of the first video code stream.
  • the splicer determines whether splicing preparation is required. If splicing preparation is required, the splicer acquires the first video code stream from the first content server.
  • An optional manner for determining whether the splicing preparation is required is: the media description information of the second content obtained by the application server in step 504, and the second video code stream information sent by the application server to the splicer in step 508 further includes a splicing point.
  • the time information is determined by the splicer according to the pre-configured preparation threshold (the interval between the time point for starting the splicing preparation and the splicing point), whether the splicing preparation is required, that is, whether the first video code stream is obtained from the first content server.
  • the splicer sets the time for starting the splicing preparation to be the time information of the splicing point minus the preparation threshold. If the current time has reached the time for starting the splicing preparation, the splicer needs to perform the splicing preparation.
  • the format of the time information may be hh:mm:ss.fff, where hh represents hours, mm represents minutes, ss represents seconds, and fff represents milliseconds.
  • the time information of the splicing point is 01:01:01.001 for 1 hour, 1 minute, 1 second, and 1 millisecond.
  • the preparation threshold is 500, which means 500 milliseconds.
  • the time to start the stitching preparation is 01:01:00.499 for 1 hour, 1 minute, and 499 milliseconds. If the current time is equal to or later than 01:01:00.499, the splicing preparation is required to acquire the first video code stream from the first content server.
  • Another alternative method for determining whether the splicing preparation is required is that the second content server sets the time for starting the splicing preparation. If the current time has reached the time for starting the splicing preparation, the RTP belongs to the RTP stream that carries the second video stream. The packet header is inserted into the repare tag to instruct the splicer to prepare for splicing. The splicer checks whether the RTP header corresponding to the second video stream contains a prepare tag. If yes, the splicing preparation is required to obtain the first video stream from the first content server.
  • Step 513 The splicer generates a parameter set identifier ID modification message of the first video, where the first video parameter set ID modification information is included.
  • the parameter set identifier modification message may be a Parameter Set ID Change Info SEI message, or a PPS ID Change Info SEI message and/or an APS identifier modification information SEI.
  • the APS ID Change Info SEI message where the parameter set identification modification information SEI message may carry one or more modification information of the parameter set type of the parameter set ID to be modified, and the PPS identification modification information SEI message includes only the parameter to be modified.
  • the parameter set type of the set ID is the repair of the PPS
  • the APS identification modification information SEI message includes only the modification information of the parameter set type of the parameter set ID to be modified as APS.
  • the splicer determines whether the video configuration information table (ie, Table 1) includes the first video parameter set ID modification information corresponding to the second video code stream identification information, such as the parameter set type of the ID to be modified. And a parameter set ID offset corresponding to the parameter set type of the modified ID.
  • Table 1 the video configuration information table
  • the splicer If the video configuration information table includes the first video parameter set ID modification information corresponding to the second video code stream identification information, the splicer generates a parameter set identifier modification message of the first video (specifically, the splicer adopts a method for generating an SEI message)
  • the SEI message is generated, such as the parameter set identifier modification supplementary enhancement information SEI message, and the adaptive parameter set identifier modification SEI message or the picture parameter set identifier modification SEI message may also be another type of SEI message, which is not limited herein. ).
  • the parameter set type of the parameter set ID to be modified is the parameter set ID modification information of the first video of the parameter set type directly referenced by the VCL NALU.
  • the parameter set ID of the modified parameter set of the first video sent by the application server to the terminal in the outband manner is the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal obtains the VCL NALU directly.
  • the parameter set ID of the parameter set indirectly referenced by the VCL NALU in the parameter set directly referenced by the VCL NALU is the parameter set ID of the parameter set used to decode the VCL NALU. Therefore, the parameter set ID modification information of the parameter set type indirectly referenced by the VCL NALU by the application server is not sent to the splicer, and is sent to the terminal by the splicer through the parameter set ID modification message.
  • the parameter set modification information of the first video includes the parameter set type of the parameter set ID of the video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • the method for generating the parameter set identifier modification information SEI message as the parameter set identifier modification message may be:
  • the splicer obtains the first video parameter set ID modification information from the video configuration information table, and includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified.
  • the first video parameter set type of the parameter set ID to be modified only contains the parameter set type directly referenced by the VCL NALU.
  • the splicer generates the parameter set identification modification information SEI message according to the obtained first video parameter set ID modification information.
  • the first encoder is an encoder that complies with the HEVC standard, that is, the parameter set type directly referenced by the VCL NALU includes PPS and APS as an example.
  • the splicer when the first video parameter set type of the ID to be modified includes the PPS and/or the APS, the splicer generates the parameter set identifier modification information SEI message.
  • the parameter set identification modification information SEI message includes two flags (flag): a picture parameter set identification change flag (pps_id_change_flag), and an adaptive parameter set identification change flag (aps_id_change_flag), which are respectively used to indicate the parameter set ID of the corresponding type parameter set (That is, whether the PPS ID and APS ID need to be modified. If the parameter set ID of the corresponding type parameter set needs to be falsified, the parameters ps id change flag and aps id change flag have a value of 1.
  • the parameter set identifier modification information SEI message further includes an offset of the original parameter set ID of the parameter set type of the parameter set ID to be modified. If the ID of the corresponding type parameter set needs to be modified, the values of the parameter ps id change flag and the aps id change flag are equal to 1.
  • the format of the parameter set identification modification information SEI message is as shown in Table 2:
  • the parameter aps id change flag has a value of 1, indicating that the APS ID needs to be modified.
  • the value of the parameter ps id change flag is 1, indicating that the PPS ID needs to be modified.
  • the descriptor ( Descirptor ) u(l) indicates that the parameters aps id change flag and ps id change flag occupy one bit.
  • the parameter aps_id_diff (adaptive parameter set identifier offset) carries the offset of the original parameter set ID corresponding to the APS
  • the parameter ps_id_diff (picture parameter set identifier offset) carries the offset of the original parameter set ID corresponding to the PPS.
  • u(5) indicates an unsigned integer represented by 5 bits
  • u(8) indicates an unsigned integer represented by 8 bits.
  • the PPS identification modification information SEI message and the APS identification modification information SEI message may be separately generated as the parameter set identification modification message for the change of the PPS ID and the change of the APS ID.
  • the PPS identifier modification information SEI message carries the offset of the original parameter set ID corresponding to the PPS, and the message format is as follows: Table 3 shows:
  • the parameter pps_id_diff carries the offset of the original parameter set ID corresponding to the PPS.
  • the APS identification modification information SEI message carries the offset of the corresponding original parameter set ID, and its message format: ⁇ Table 4:
  • the parameter aps_id_diff carries the offset of the original parameter set ID corresponding to the APS.
  • the splicer only generates the PPS identification modification information SEI message; if the first video parameter set type of the ID needs to be modified, APS, excluding PPS, the splicer only generates the APS identification modification information SEI message; if the first video parameter set type of the ID needs to be modified to include both PPS and APS, the splicer can separately generate the PPS identification modification information SEI message and the APS identification modification. Information SEI message.
  • the first video parameter set type that needs to be modified ID includes both PPS and APS
  • the splicing can generate a parameter set identifier modification information SEI message
  • the parameter set identification information SEI message includes PPS and APS modification information.
  • Step 514 The splicer sends a parameter set identifier modification message (such as a parameter set identifier modification information SEI message) of the first video to the terminal, and the first video code stream, that is, the splicer modifies the parameter set identification message of the first video with the first The video stream is sent to the terminal together.
  • a parameter set identifier modification message such as a parameter set identifier modification information SEI message
  • the splicer Before performing the step 514, the splicer further determines whether the splicing point is reached, and sends the parameter set identification modification message and the first video code stream to the terminal when the splicing point is reached. Specifically, an optional manner of determining whether the splicing point is reached is: the splicer determines whether the current time of outputting the second video code stream has reached the starting point of the splicing point period, and if the starting point of the splicing period has been reached, the splicing point is considered to be reached. .
  • Another alternative way of determining whether the splicing point is reached is that the second content server inserts a slice tag into the header of the RTP packet corresponding to the second video stream before the splicing point and specifies a distance splicing point in the slice tag.
  • the splicer checks whether the RTP header corresponding to the second video stream contains a slice tag and the time at which the splicing point is specified is 0. If it is included, it indicates that the splicing point is reached.
  • the parameter set identification modification message is sent at least with the random access point (RAP: Random Access Point) and applied to all pictures between the associated RAP and the next RAP.
  • RAP random access point
  • the RAP is a picture in the middle of the code stream that only uses intra prediction, and the decoder can start the decoding process from the picture.
  • the splicer inserts the NALU encapsulated with the parameter set identification modification message into the associated RAP AU separator NALU, before the VCL NALU in the first video code stream.
  • Step 515 After receiving the parameter set identifier modification message (such as the parameter set identifier modification information SEI message) of the first video and the first video code stream, the terminal modifies the message according to the parameter set identifier of the first video (such as the parameter set identifier modification information SEI).
  • the terminal modifies the message according to the parameter set identifier of the first video (such as the parameter set identifier modification information SEI).
  • Message Determine the parameter set ID of the parameter set referenced by the VCL NALU in the first video stream (ie, the parameter set ID of the parameter set used to decode the VCL NALU).
  • the parameter set whose parameter set ID is the determined parameter set ID is obtained from the parameter set obtained by the out-of-band method according to the determined parameter set ID, that is, the parameter set of the VCL NALU used for decoding the first video code stream.
  • the parameter set identifier modification message of the first video is used as the parameter set identifier modification information SEI message, and the parameter set type directly referenced by the VCL NALU includes PPS and APS as an example for description.
  • Modifying information according to the parameter set identifier The SEI message determines the parameter set ID of the parameter set referenced by the VCL NALU and the method of the parameter set may be:
  • the terminal first identifies the modification information according to the parameter set identifier.
  • the SEI message determines the parameter set ID of the parameter set directly referenced by the VCL NALU, and then obtains the parameter set directly referenced by the VCL NALU from the parameter set obtained by the out-of-band method according to the parameter set ID.
  • the terminal confirms that the parameter set identification information SEI message carries the APS ID change flag, such as the parameter aps id change flag value in the message is 1, or other identifier.
  • the APS ID from the received VCL NALU of the first video stream, that is, the original parameter set ID of the adaptive parameter set referenced by the VCL NALU (ie, the original aps id of the aps id), and according to the APS ID.
  • Original The value and the parameter set identification modification information SEI message offset from the original parameter set ID corresponding to the APS calculates a new value (ie, the value of the parameter set ID used to decode the parameter set of the VCL NALU).
  • the original value + the offset of the original parameter set ID corresponding to the APS takes a new value, and uses the new aps id as a reference_aps_id (reference adaptive).
  • the value of the parameter set identifier is obtained, and then the APS including the aps_id which is the same as the reference_aps_id (that is, the same as the new_aps_id) is obtained from the parameter set obtained by the outband method, that is, the parameter set for decoding the VCL NALU in the first video bitstream.
  • the parameter set for decoding the VCL NALU in the first video bitstream from the parameter set obtained by the outband mode according to the value of the reference adaptive parameter set identifier is specifically:
  • the terminal obtains the parameter set NALU from the media description information obtained in step 507. according to
  • the nal_unit_type in the NALU header determines the type of the parameter set in the NALU. Different types of parameter sets have different formats, and the parameter set type must be determined before the parameter set can be parsed according to the corresponding format. If the type of the parameter set is APS, get the aps id in it. If the asp_id is equal to the reference aps id, that is, the parameter set corresponding to the asp_id is the APS for decoding the VCL NALU in the first video bitstream; otherwise, the next parameter set NALU is processed until the aps_id is equal to the reference aps id. APS.
  • the terminal confirms the parameter set identification modification information, and the SEI message carries the PPS ID change flag, and obtains the offset of the original parameter set ID corresponding to the PPS from the parameter set identifier modification information SEI message.
  • the terminal obtains the original parameter set ID of the PPS referenced by the VCL NALU from the VCL NALU of the received first video stream, and calculates a new value according to the original parameter set ID and the offset of the original parameter set ID corresponding to the PPS. And taking the new value as the value of the modified PPS ID (reference picture parameter set identifier), and obtaining the PPS of the modified PPS ID obtained from the parameter set NALU in step 507.
  • the terminal then obtains the parameter set indirectly referenced by VCL NALU, such as SPS and VPS, according to the parameter set ID indirectly referenced by VCL NALU contained in the parameter set directly referenced by VCL NALU.
  • VCL NALU such as SPS and VPS
  • the SPS including the same sps id as the sps id in the PPS is obtained from the parameter set NALU obtained in step 507 according to the sps_id in the PPS; and the parameter set NALU is obtained from the step 507 according to the vps_id in the SPS.
  • the VPS of the same vps id as the vps id in the SPS is obtained from the parameter set NALU obtained in step 507 according to the sps_id in the PPS.
  • Both the parameter set directly referenced by the VCL NALU and the parameter set indirectly referenced by the VCL NALU serve as a parameter set for decoding the VCL NALU of the first video stream.
  • the terminal does not generate the parameter set identification modification information SEI message in step 513, but generates the PPS identification modification information SEI message and the APS identification modification information SEI message respectively, then in step 515, The terminal determines a parameter set ID of the PPS referenced by the VCL NALU according to the PPS identifier modification information SEI message, and determines a parameter set ID of the APS referenced by the VCL NALU according to the APS identifier modification information SEI message, and according to the determined parameter set ID of the PPS And the parameter set ID of the APS obtains the parameter set ID of the PPS and the PPS and APS of the parameter set ID of the APS from the parameter set obtained by the outband mode, and acquires the SPS according to the sps id in the PPS; according to the SPS The
  • the specific acquisition method is the same as the method for obtaining the SEI message according to the parameter set identification modification information.
  • the embodiment of the present invention is not described in detail herein.
  • Step 516 Using a parameter set obtained from a parameter set obtained by an out-of-band method (for example, using the first encoder as an encoder conforming to the HEVC standard, including APS, PPS, SPS, and VPS) to the first video stream.
  • the VCL NALU in the decoding is performed.
  • the splicer only needs to generate the parameter set identifier.
  • the modification message such as (parameter set identification modification information SEI message, or PPS identification modification information SEI message and/or APS identification modification information SEI message).
  • the parameter set identifier modification information carries the parameter set ID modification information of the first video, and the terminal determines the parameter set ID of the parameter set used for decoding the VCL NALU according to the parameter set identifier modification message, without directly modifying the first video stream by using the splicer.
  • the parameter set ID in .
  • the splicer Since the operation of generating the parameter set identification modification message is relatively simple, the splicer does not need entropy decoding and entropy coding capability and the operation frequency is low, so only a small amount of computing resources and storage resources are needed. This requires only a simple upgrade of the splicer in the existing network, without the need to add additional hardware resources to the splicer in the existing network, or to replace the splicer in the existing network with a new splicer. Therefore, operators need less operating costs than the prior art.
  • the application server modifies the first video parameter set parameter set ID in step 506, and then transmits the first video parameter set including the modified parameter set ID to the terminal in step 507.
  • the application server may not modify the parameter set ID of the first video parameter set, and the media description information acquisition response sent to the terminal in step 507 indicates that the first video parameter set is used for The parameter set ID of the parameter set that decodes the VCL NALU. That is, the media description information acquisition response sent by the application server to the terminal includes a parameter set of the second video, an original parameter set of the first video, and an original parameter set of the first video for decoding the VCL.
  • the parameter set ID of the NALU parameter set, where the original parameter set of the first video is the application server in the step
  • the media description information sent to the terminal in step 507 is taken as an SDP format as an example.
  • the application server obtains a set of sub-parameters of the parameter sprop-parameter-sets (flow characteristic parameter set) in the SDP in the media description information in the media description information: ⁇ parameter set id>: ⁇ parameter set data > to carry a parameter set NALUc
  • parameter set type is a three-byte text representing the type of the parameter set.
  • the parameter set id is a 1-byte unsigned integer representing the value of the parameter set ID.
  • the value of this field can be different from the value of the original parameter set ID in the parameter set NALU. If the value of the parameter set id is different from the value of the original parameter set ID in the parameter set NALU, the value of the parameter set id field is used as the parameter set ID of the parameter set used to decode the VCL NALU.
  • the original parameter set is a set of sub-parameters of the parameter sprop-parameter-
  • the ID is the parameter set ID in the parameter set NALU sent by the first content server to the application server in step 502, that is, the parameter set ID of the first video that has not been modified.
  • the parameter set id is the parameter set ID of the parameter set that needs to be used to decode the VCL NALU.
  • the parameter set data is the Base64 (basic 64 character) encoded parameter set NALU.
  • the carried parameter set NALU is a parameter set NALU sent by the first content server to the application server.
  • the parameter sprop-parameter-sets can contain multiple sets of sub-parameters to carry multiple parameter sets NALU. If parameter-sets contains multiple sets of sub-parameters to carry multiple parameter sets NALU, each set of sub-parameters is separated by the character ",".
  • step 507 the SDP example of the parameter set NALU sent by the application server to the terminal is as follows:
  • the second video parameter set NALU received by the application server includes a VPS NALU, an SPS NALU, a PPS NALU, and an APS NALU, corresponding parameter sets.
  • the VPS ID, SPS ID, PPS ID, and APS ID in the NALU are both 0.
  • the first video parameter set NALU received by the application server also includes a VSP NALU, an SPS NALU, a PPS NALU and an APS NALU, and the VPS ID, SPS ID, PPS ID, and APS ID in the corresponding parameter set NALU are also 0. .
  • the SDP carries eight parameter sets NALU, and the first to fourth sets of sub-parameters carry the parameter set NALU of the second video, and the corresponding parameter set types are VPS, SPS, PPS, APS; the parameter set of the second video parameter set The ID does not need to be modified, so the corresponding sub-parameter arameter set id in the SDP is 0, and the parameter set ID in the parameter set NALU in the corresponding parameter set data is 0; the fifth to eighth sub-parameters carry the first video.
  • the parameter set ID of the first video parameter set needs to be modified and the new value is all 1, so the corresponding sub-parameter parameter set id in the SDP is 1, but the parameter in the parameter set NALU in the corresponding parameter set data
  • the set ID is 0.
  • the parameter set modification information of the first video includes the parameter set type of the parameter set ID of the first video to be modified and the offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified as an example. Be explained.
  • the parameter set ID in the original parameter set NALU of the first video sent by the application server to the terminal in step 507 is the original parameter set ID of the first video
  • the VCL NALU is directly
  • the parameter set ID of the parameter set indirectly referenced by the VCL NALU in the referenced parameter set is the original parameter set ID. Since the parameter set ID is based on the parameter set ID indicated in the media description response message, the terminal cannot pass the original parameter set ID.
  • the parameter set obtained in the parameter set acquired in the outband manner obtains the correct parameter set. Therefore, in step 508, the parameter set ID modification information of the first video further includes the parameter set ID modification information indirectly referenced by the VCL NALU.
  • the parameter set identification modification message also needs to include the parameter set ID modification information of the VCL NALU indirect reference.
  • the first encoder is an encoder that complies with the HEVC standard, that is, the parameter set type indirectly referenced by VCL NALU includes SPS and VPS as an example.
  • the parameter set identification modification information SEI message also needs to include vps_id_change_flag (video parameter set identification change flag), sps_id_change_flag (sequence parameter set identification change identifier), if the parameter set type of the parameter set ID to be modified includes vps, the splicer sets vps_id_change_flag to 1 And passing the corresponding parameter set ID offset by vps id diff (video parameter set identifier offset); if the parameter set type of the parameter set ID to be modified includes sps, the splicer sets sps_id_change_flag to 1, and identifies by the parameter sequence parameter set The offset (sps_id_diff ) passes the corresponding parameter set ID offset. That is, in this way, the format of the parameter set identification modification information SEI message is as shown in Table 5:
  • Table 5 The description of the parameters of the fifth table is the same as that of the second embodiment. The embodiment of the present invention is not described in detail herein.
  • Each parameter set NALU is stored in a parameter set NALU record table, and the format of the parameter set NALU record table is as shown in Table 6.
  • Parameter set type parameter set id parameter set data where, as described above, arameter set type represents the parameter set type, arameter set id represents the parameter set ID of the parameter set used to decode the VCL NALU, and the parameter set data is the Base64 encoded modified parameter.
  • the parameter set NALU record table is as follows. Table 7 shows
  • step 515 after receiving the parameter set identification modification information SEI message, the terminal obtains the reference aps id and the reference_pps_id according to the parameter set identifier tamper information SEI message, and takes the reference aps id and the reference_pps_id from the parameters learned by the sufficiency method.
  • the centralized capture contains the aps id as the reference aps id APS, including the ⁇ ps id as the reference_pps_id PPS.
  • the sps_id included in the PPS is obtained, that is, the original_sps_id, and the reference information is modified according to the parameter set identifier SEI message to obtain the reference_sps_id (reference sequence parameter set identifier value), and the included sps id is obtained from the parameter set obtained by the outband method according to the reference aps id.
  • SPS for reference sps id SPS for reference sps id.
  • the vps id in the SPS is taken as the original_vps_id, and the reference information is modified according to the parameter set identifier SEI message to obtain the reference_vps_id (reference video parameter set identifier value), and the included parameter is obtained from the parameter set obtained by the outband method according to the reference vps id.
  • the reference parameter set is obtained by using a reference aps id , a reference_pps_id, a reference sps id, or a reference vps id
  • the parameter set type is obtained from the parameter set NALU record table and the arameter set id is equal to the reference aps id , reference_pps_id, Reference sps id or reference vps id
  • the NALU record extracts the corresponding Base64 encoded parameter set NALU from the parameter set data of the record, and then decodes the VCL NALU.
  • the splicer may directly include the new value of the corresponding parameter set ID in the Parameter Set ID Change Info SEI message instead of the corresponding parameter set ID. Offset.
  • the splicer may directly include the modified PPS in the PPS identification modification information SEI message or the APS identification modification information SEI message.
  • the terminal directly obtains the corresponding parameter set ID from the parameter set identification modification message of the first video to determine the parameter set ID of the parameter set used to decode the VCL NALU.
  • the parameter set ID modification information of the first video in step 508 is a parameter set type of the parameter set ID to be modified and a parameter set ID of a parameter set for decoding the VCL NALU, where The parameter set ID of the parameter set of the VCL NALU is the value of the modified parameter set ID assigned by the application server for the parameter set ID to be modified.
  • a parameter set including a first video in which the parameter set ID is modified in the SDP is adopted.
  • NALU is taken as an example for illustration.
  • the splicer is sent to the terminal.
  • the parameter APS identifier change identifier (aps_id_change_flag) has a value of 1, indicating that the APS ID needs to be modified
  • the parameter PPS identifier change identifier (pps_id_change_flag) has a value of 1, indicating that the PPS ID needs to be modified.
  • Descirptor:u(l) indicates that the parameters aps_id_change_flag and pps_id_change_flag occupy one bit.
  • the parameter new_aps_id carries the modified APS ID
  • the parameter new_aps_id carries the modified PPS ID.
  • Descirptor: u(5) indicates an unsigned integer incremented by a 5-bit table
  • u(8) indicates an unsigned integer represented by 8 bits.
  • the message format is similar to that of the table 8, which is not described in detail in the embodiment of the present invention.
  • step 507 the original parameter set NALU and the original parameter set including the first video in the SDP are adopted.
  • the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the NALU is taken as an example for description.
  • the parameter set identifies the modification information.
  • the format of the SEI message is as shown in Table 9:
  • the first video may also be determined in the middle of the session process.
  • the address of the first content server needs to be pre-configured on the splicer.
  • the splicer determines that splicing preparation is required, the first video code stream is requested from the first content server.
  • the terminal only has a parameter set of the second video in the media description information obtained before the session is initiated or the session is established, and the terminal needs to re-acquire the media set including the parameter set of the second video and the parameter set of the first video in the middle of the session. Description. Referring to FIG. 6, another flowchart of a code stream splicing method provided by an embodiment of the present invention includes the following steps:
  • Step 606 The application server sends a first media description information acquisition response message to the terminal, where the media description information includes a parameter set of the second video.
  • the steps 607-609 are the same as the steps 509-511, and the embodiments of the present invention are not described in detail herein.
  • Step 610 The splicer sends a first video acquisition request to the first content server according to the configured address of the first content server, where the first video acquisition request further includes a user identifier and second video code stream identification information.
  • the splicer determines whether splicing preparation is required. If splicing preparation is required, the splicer sends a first video acquisition request to the first content server.
  • step 512 For the method of determining whether the splicer needs to be spliced, refer to the description of step 512, and the embodiment of the present invention will not be described in detail herein.
  • Step 611 The first content server determines the first video code stream, and sends the first video code stream to the splicer.
  • the first video acquisition request sent by the splicer in step 608 received by the first content server includes the identifier of the terminal user, the first content.
  • the server acquires the user preference of the terminal user according to the identifier of the terminal user, selects the first video according to the acquired user preference, and determines the first video code stream.
  • the first content server may further acquire an insertion policy of the second content according to the identification information of the second video stream, select the first content according to the insertion strategy, and determine the first video. Code stream.
  • the first video acquisition request in step 608 needs to include the identification information of the second video stream.
  • the first content server can also select the first content based only on the stored business policy.
  • the first content server may determine the first video bitstream in another manner, which is not described in detail herein.
  • Step 612 The first content server sends the media description information of the first video to the application server, where the media description information of the first video includes an original parameter set of the first video, a type of the parameter set of the first video, and a type corresponding to each type.
  • the original parameter set ID of the parameter set is a parameter set of the parameter set.
  • Step 613 The application server modifies the original parameter set ID of the first video.
  • the application server determines, according to the identifier of the terminal user or the second video code stream identification information, the description information of the second video corresponding to the received media description information of the first video.
  • the specific method of the application server to modify the original parameter set ID of the first video is the same as the step 506, and the embodiment of the present invention will not be described in detail herein.
  • Step 614 The application server sends the second video code stream and the first video code stream information to the splicer.
  • the second video code stream information is the second video code stream identification information
  • the first video code stream information is the first video parameter set ID modification information.
  • Step 615 The splicer generates a parameter set identifier modification message (such as a parameter set identifier modification information SEI message) that is not valid according to the first video parameter set ID modification information, and the parameter set identifies the modification message except the parameter of the first video.
  • a parameter set identifier modification message such as a parameter set identifier modification information SEI message
  • the parameter set identifies the modification message except the parameter of the first video.
  • the flag information indicating that the parameter set ID modification information is not valid is further included.
  • the parameter set identifier modification message includes only the parameter set ID modification information corresponding to the parameter set type directly referenced by the VCL NALU, and the parameter set identifier modification message is the parameter set identifier modification information SEI message as an example.
  • the modification information validation flag (change_info_enable_flag) is included in the example, and the value of the flag is set to 0.
  • other methods such as taking the value, F, The character string, the TALSE, and other identifiers are not valid, and the parameter set ID modification information carried in the parameter set ID modification information SEI message is not valid.
  • the parameter set modification type of the parameter set of the first video includes the parameter set type of the parameter set ID of the first video and the parameter set ID offset corresponding to the parameter set type of the parameter set ID to be modified, and the first encoder is The encoder according to the HEVC standard is explained. Further, the format of the parameter Set ID Change Info SEI message carrying the modification information valid flag is shown in the port table 10:
  • the splicer sets change_info_enable_flag to 0. Indicates that the parameter set ID modification information of the message does not take effect temporarily.
  • change_info_enable_flag 0.
  • the embodiments of the present invention are not described in detail herein.
  • Step 616 The splicer sends, to the terminal, a parameter set identifier modification message (ie, a parameter set identifier modification message that is not valid) and a second video code stream, where the first video that indicates that the parameter set ID modification information is not valid is sent.
  • a parameter set identifier modification message ie, a parameter set identifier modification message that is not valid
  • a second video code stream where the first video that indicates that the parameter set ID modification information is not valid is sent.
  • the splicer sends, according to the second video code stream sending step 615, a parameter set identifier modification message that includes flag information indicating that the parameter set ID modification information is not valid, such as including the indication parameter set.
  • the parameter set ID modification information SEI message of the flag information in which the ID modification information does not take effect.
  • the NALU encapsulating the parameter set identifier tampering message may be inserted in consecutive AUs.
  • the splicer In order to ensure smooth transmission, the splicer usually has a buffer. After receiving the video stream, the splicer stores the video stream in a buffer. In order to reduce the overhead caused by the parameter set identifier modification message including the flag information indicating that the parameter set ID modification information is not valid, the splicer may wait for the second video stream before the splicing point to be buffered, and determine the current to be sent. Whether the second video picture is the last RAP before the splicing point, and if so, the parameter set identification modification message including the flag information indicating that the parameter set ID modification information is not valid is sent with the second video RAP. Similarly, in order to ensure that the terminal receives the SEI message, the NALU encapsulating the parameter set identifier tampering message may be inserted into multiple consecutive AUs after the RAP AU.
  • the splicer may similarly generate PPS identification modification information including flag information indicating that the parameter set ID modification information is not valid.
  • Step 617 The terminal receives the parameter set identifier modification message of the first video that includes the flag information indicating that the parameter set ID modification information is not valid, and confirms that the flag information indicating that the parameter set ID modification information is invalid is not included, and sends the second media to the application server. Describe the information acquisition request (such as a session change request).
  • the terminal determines that the change info enable flag in the parameter set identifier modification message is 0, that is, the confirmation parameter set identifier modification message includes the flag information indicating that the parameter set ID modification information is invalid.
  • the terminal sends a media description information acquisition request to the application server, such as a SIP Re-Invite request,
  • Step 618 The application server sends a second media description information acquisition response (such as a session change response) to the terminal, where the media description information acquisition response includes a second video parameter set NALU, a first view.
  • Frequency parameter set NALU The parameter set ID in the first video parameter set NALU is the parameter set ID of the modified first video, specifically the original value of the parameter set ID of the first video plus the offset of the parameter set ID, or the application server. The value assigned directly to the parameter set ID of the modified first video.
  • Step 619 The splicer generates a parameter set identifier modification message (such as a parameter set ID modification information SEI message, or a PPS identifier modification information SEI message and/or an APS identifier tamper information SEI, including flag information indicating that the parameter set ID modification information is valid.
  • the message is that change_info_enable_flag is set to 1.
  • Step 620 The splicer sends, to the terminal, a parameter set identifier modification message and a first video code stream that include flag information indicating that the parameter set ID modification information is valid.
  • the splicer Before performing step 620, the splicer also determines if the splice point is reached. The splicer sends a parameter set identification modification message and a first video code stream to the terminal when the splicing point is reached. The specific manner in which the splicer judges whether or not the splice point is reached is the same as the same description in the flow described in FIG.
  • the parameter set identification modification message (parameter set ID modification information SEI message, or PPS identification modification information SEI message and/or APS identification modification information SEI message) is transmitted at least with the random access point and applied to the associated RAP All pictures between the next RAP and the next RAP.
  • the RAP is a picture in the middle of the code stream that only uses intra prediction, and the decoder can start the decoding process from the picture.
  • the splicer inserts the NALU encapsulated with the parameter set identifier modification message into the AU separator NALU of the associated RAP, before all VCL NALUs.
  • a parameter set identifier modification message (such as a parameter set ID modification information SEI message, or a PPS identity modification information SEI message and/or an APS identity modification information SEI message) that includes flag information indicating that the parameter set ID modification information is valid
  • the acknowledgment includes the flag information indicating that the parameter set ID modification information is valid (for example, the change_info_enable_flag in the parameter set identifier modification message is 1), and the modification is determined according to the parameter set identifier including the flag information indicating that the parameter set ID modification information is valid.
  • the parameter set ID of the parameter set for decoding VCL NALU. Obtaining the parameter set ID from the parameter set obtained by the out-of-band method according to the determined parameter set ID and determining the parameter set ID The same parameter set.
  • the method for determining the parameter set ID of the VCL NALU parameter set in the video code stream according to the parameter set identification modification message may refer to the description of step 515, which is not described in detail herein.
  • the application server may not actively perform the step 615 to 618, and the application server actively sends the media description information to the terminal (not shown in the flowchart).
  • the media description information includes a parameter set of the second video and a modified modified parameter set of the first video.
  • the modified parameter set ID of the first video is the parameter set ID of the modified first video (ie, the parameter set ID of the parameter set used to decode the VCL NALU).
  • the application server may send a session change request message including the media description information, such as a SIP Re-Invite request message or a SIP Update request message, to the terminal, or send a notification message including the media description information to the terminal, such as an RTSP Play-Notify request.
  • a session change request message including the media description information such as a SIP Re-Invite request message or a SIP Update request message
  • a notification message including the media description information such as an RTSP Play-Notify request.
  • Message RTSP Set Parameter request message, RTSP Notify request message or SIP Info request message.
  • the terminal receives the media description information sent by the application server, and obtains the second video parameter set NALU and the first video parameter set NALU. It is also possible to perform steps 513-516 without performing steps 619-622.
  • the parameter set identifier modification message in the process corresponding to FIG. 6 further includes parameter set ID modification information corresponding to the parameter set type indirectly referenced by the VCL NALU of the first video, and correspondingly, the media in step 618
  • the description information acquisition response message or the media description information that the application server actively sends to the terminal includes the original parameter set of the first video and the parameter set ID of the parameter set for decoding the VCL NALU.
  • the parameter set identifies the modification message (such as parameter set ID modification)
  • the SEI message contains the parameter set type of the parameter set ID to be modified, and does not carry other parameter set ID modification information, such as carrying the modified value or the parameter set ID to modify the offset, that is, carrying one or more * * * _id_change_flag, without carrying * * * - id - diff parameter or new_* * *_id parameter.
  • the terminal After receiving the parameter set identifier modification message, the terminal determines the type of the first video parameter set that needs to be modified according to the parameter set identifier modification message, and requests the splicer to send the first video parameter set type corresponding to the ID to be modified by the RTCP message.
  • FIG. 7 is a flowchart of another method for splicing a code stream according to an embodiment of the present invention, including the following steps:
  • Steps 701-712 are the same as steps 501-512, and embodiments of the present invention are not described in detail herein.
  • Step 713 The splicer generates a parameter set identifier modification message (such as a parameter set ID modification information SEI message).
  • the parameter set identifies a parameter set type including only the parameter set ID to be modified in the modification message, and does not carry other parameter set ID modification information, such as a parameter set ID or a parameter set ID that does not carry a parameter set for decoding the VCL NALU. Offset.
  • the media description information sent by the application server to the terminal in step 707 carries the modified parameter set of the first video (that is, the parameter set ID in the modified parameter set is used to decode the parameter set of the VCL NALU. Parameter set ID), then the parameter set type of the parameter set ID to be modified in this step is only the parameter set type directly referenced by VCL NALU.
  • the first encoder is an encoder that complies with the HEVC coding standard as an example.
  • the parameter set type of the parameter set ID to be modified is APS and/or PPS, that is, the parameter set ID modification information carries aps_id_change_flag and pps id change flag, and if the splicer receives the ID of the parameter set ID of the first video to be modified,
  • the first video parameter set type includes APS, the splicer sets aps_id_change_flag to 1, otherwise it is set to 0; if the splicer receives the first video parameter set ID modification information, the first video parameter set type of the ID to be modified includes the PPS
  • the splicer sets pps_id_change_flag to 1, otherwise it is set to 0.
  • the parameter set type of the parameter set ID to be modified in this step is further A parameter set type that contains an indirect reference to the VCL NALU.
  • the first encoder is an encoder that complies with the HEVC coding standard as an example.
  • the parameter set identifier modification message must include the vps id change flag, the sps id change flag, the aps id change flag, and the pps id change flag.
  • the specific format of the parameter set ID modification information SEI message is as shown in Table 11.
  • Step 714 The splicer sends a parameter set identifier modification message (such as a Parameter Set ID Change Info SEI message) and the first video code stream to the terminal.
  • the parameter set identifies a parameter set type including only the modified parameter set ID in the modification message, and does not carry other parameter set ID modification information, such as a parameter set ID or a original parameter set that does not carry a parameter set for decoding the VCL NALU. The offset of the ID.
  • Step 715 The terminal determines, according to the parameter set identifier modification message, a parameter set type that needs to be modified.
  • Step 716 The terminal sends an RTCP request to the splicer, where the parameter set type determined in step 715 that needs to modify the parameter set ID is included.
  • the above RTCP request may be an RTCP parameter set identifier tamper information request Parameter Set ID Change - Request message, referred to as RTCP PSIC-R message.
  • the RTCP PSIC-R message is an RTCP feedback message, as shown in Figure 8.
  • PT Payload Type
  • FMT Feedback Message Type
  • SSRC of packet sender SSRC of the sender of the RTCP feedback message
  • SSRC of media source SSRC of the RTP stream corresponding to the RTCP feedback message.
  • FCI Flight Control Information
  • the PT value of the RTCP PSIC-R message is 206, which means it is Payload Specific Feedback Message; the FMT value is 4, which means it is a PSIC message; the SSRC of packet sender is the SSRC of the terminal; and the SSRC of media source is obtained from the SDP.
  • RTP for carrying the output video stream The SSRC of the stream, that is, the SSRC in the second video stream identification information. (For the terminal, the second video stream and the first video stream are not distinguished, and the SSRC seen by the terminal is one).
  • the format of the FCI is shown in Figure 9. among them
  • SFMT Sub Feedback Message Type: The type of PSIC message. Set here to 1, representing the PSIC-R message;
  • Number of Parameter Set Type The number of parameter set types whose ID needs to be modified
  • Parameter Set Type The parameter set type identifier of the ID to be modified. There are a total of Number of Parameter Set Type Parameter Set Type.
  • the Parameter Set Type can take many forms.
  • the first encoder is an encoder that complies with the HEVC standard.
  • the value of the Parameter Set Type can be a positive integer, and the values 1, 2, 3, and 4 represent VPS, SPS, PPS, and APS, respectively.
  • Step 717 After receiving the RTCP request (such as the RTCP PSIC-R message), the splicer sends the parameter set ID modification information of the first video to the terminal by using an RTCP response (such as an RTCP PSIC-I message).
  • the RTCP PSIC-I message is also an RTCP PSIC message whose SFMT is set to 2, including Number of Parameter Set ID Change Info, indicating the parameter set identifier contained in the RTCP PSIC-I message. Modify the number of messages.
  • a parameter set identification modification information includes a parameter set type (Parameter Set Type) and an original parameter set ID offset (Parameter Set Diff).
  • the FCI format is as shown in FIG.
  • a parameter set identification modification information includes a Parameter Set Type, an Original Parameter Set ID, and a New Parameter Set ID. If the parameter set ID modification information includes a first video parameter set type of the ID to be modified and a corresponding new value of the parameter set ID, the FCI format is as follows. Figure 12 shows.
  • a parameter set identification modification information includes a Parameter Set Type and a New Parameter Set ID.
  • Step 720 The terminal receives the RTCP PSIC-I message, for each first video VCL NALU, The terminal determines the value of the parameter set ID of the VCL NALU reference parameter set according to the RTCP PSIC-I message, and obtains all referenced parameter sets, and decodes the corresponding VCL NALU by using the obtained parameter set.
  • the first video parameter set ID modification information may also be sent by the splicer to the terminal by using the transport layer signaling.
  • the splicer generates an RTCP PSCI-I message as described above for the RTCP PSCI-I message.
  • the splicer sends a first video bitstream and an RTCP PSCI-I message to the terminal, and the terminal receives the first video bitstream and the RTCP PSCI-I message.
  • the terminal For each first video VCL NALU, the terminal is based on the RTCP PSIC- The I message determines the value of the parameter set ID of the VCL NALU reference parameter set, and obtains all referenced parameter sets, and the terminal uses these parameter sets to decode the corresponding VCL NALU.
  • the splicer may also determine the sequence number of the first RTP packet used to encapsulate the first video code stream, and then Send an RTCP PSCI-I message.
  • the RTCP PSCI-I message includes the first sequence number (first sequence number) in addition to all the information of the RTCP PSCI-I message in step 717, and the first sequence number is used to encapsulate the first video stream.
  • the serial number of the RTP package In order to ensure that the terminal receives the RTCP PSCI-I message, the splicer can continuously send multiple identical RTCP PSCI-I messages.
  • the parameter set identification modification message (such as: Parameter Set ID Change Info SEI Message) is generated by the splicer.
  • the parameter set identification modification message (such as: Parameter Set ID Change Info SEI Message) can also be generated by the application server and sent to the splicer.
  • the application server sends the parameter set identifier modification information to the splicer
  • the parameter set ID modification information of the first video is directly sent in the form of a Parameter Set ID Change Info SEI Message.
  • the splicer is only responsible for the transmission of the Parameter Set ID Change Info SEI Message.
  • the application server Since the application server has entropy coding and entropy decoding capability, when the application server generates the Parameter Set ID Change Info SEI Message, the ***_id_diff or the new_***_id in the Parameter Set ID Change Info SEI Message may be entropy encoded, for example Unsigned index Columbus code.
  • FIG. 1 Unsigned index Columbus code.
  • Step 1301 Receive information of a first video code stream from an application server, where information of the first video code stream includes first The parameter set modification information of the video; specifically, the splicer receives information of the first video code stream from the application server, where the first video code stream information includes parameter set modification information of the first video. Further, the splicer can also receive information of the second video stream from the application server.
  • Step 1302 Send a first video code stream and a parameter set identifier modification message of the first video to the terminal according to the first video parameter set modification information, where the parameter set identifier modification message of the first video carries a parameter set of the first video Identifying ID modification information, so that the terminal acquires a parameter set ID of a parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal is used to decode the VCL NALU according to the The parameter set ID of the parameter set acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.
  • the splicer prior to step 1302, the splicer also pauses outputting the second video stream during the output of the second video stream.
  • the splicer also pauses outputting the second video stream during the output of the second video stream.
  • the parameter set ID modification information of the first video is a parameter set type of the parameter set ID to be modified, and the splicer further receives a real-time transmission control protocol RTCP request sent by the terminal receiving terminal, where the RTCP request includes the parameter set ID to be modified.
  • a parameter set type returning an RTCP response to the terminal, where the RTCP response carries an original parameter corresponding to the parameter set type of the parameter set ID to be modified
  • An offset of the set ID or a parameter set ID carrying a parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set type carrying the parameter set ID to be modified
  • the parameter set ID modification information is an offset of a parameter set type of the parameter set ID to be modified and an original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set ID to be modified.
  • the parameter set ID modification information further comprising a modification validation flag, the parameter for decoding the VCL NALU being obtained according to the parameter set ID of the parameter set for decoding the VCL NALU
  • the receiver further sends a parameter set identifier modification message of the first video that is not valid when the second video code stream is sent to the terminal; the second media description information acquisition request sent by the terminal is received; and the second media is returned to the terminal.
  • a description information acquisition response the second media
  • the parameter set identification modification message of the first video is: a parameter set identifier modification supplementary enhancement information SEI message, an adaptive parameter set identifier modification SEI message, or a picture parameter set identifier modification SEI message, and the splicer further Generating a parameter set identification modification message of the first video according to the first video parameter set modification information.
  • the splicer after receiving the parameter set modification information of the first video sent by the application server, the splicer does not modify the parameter set ID referenced by the VCL NALU in the first video code stream according to the parameter set modification information. Rather, the parameter set identifier modification message carrying the parameter set modification information is sent to the terminal, and the terminal obtains the parameter of the VCL NALU in the first video code stream according to the parameter set identifier modification message before decoding the VCL NALU in the first video stream. Set to decode the parameter set.
  • the splicer to have the ability to modify the entropy coding and entropy decoding of the parameter set ID of the VCL NALU reference in the first video stream, thereby eliminating the need to add extra hard on the splicer of the existing network.
  • FIG. 14 is a flowchart of a method for splicing a digital video code stream according to an example of the present invention, including the following steps:
  • Step 1401 Receive media description information of the first video sent by the first content server, where the media description information of the first video carries the type of the parameter set of the first video of the original parameter set of the first video and the parameter corresponding to each type The original parameter set ID of the set;
  • Step 1402 Receive media description information of the second video sent by the second content server, where the media description information of the second video carries the parameter set of the second video parameter set of the second video and a parameter set ID of a parameter set corresponding to each type;
  • Step 1403 Determine that the parameter set ID of the first video of the same type is the same as the parameter set ID of the parameter set of the second video, and determine the parameter set identification modification information of the first video.
  • Step 1404 Send the first to the splicer The information of the video stream, the information of the first video code stream includes the parameter set identifier modification information of the first video, and the parameter set modification identifier information of the first video is carried in the parameter set identifier modification message of the first video, so that And sending, by the splicer, the parameter set identification modification message of the first video to the terminal, and in order for the terminal to modify the message according to the parameter set identifier of the first video to decode and output the first video.
  • the application server further modifies the original parameter set ID of the original parameter set of the first video to the video coding layer for decoding the network abstraction layer unit according to the determined parameter set identification modification information of the first video.
  • the parameter set ID of the parameter set of the VCL NALU; the modified parameter set carrying the first video is sent to the terminal, and the parameter set ID in the modified parameter set is the parameter set ID of the parameter set used to decode the VCL NALU;
  • the parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an original parameter of the first video corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set identification modification message of the video includes a parameter set ID of the parameter set type of the parameter set ID to be modified, the original parameter set ID, and a parameter set for decoding the VCL NALU corresponding to the original parameter set ID. ;
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU.
  • the application server further needs to send, to the terminal, an original parameter set of the first video and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU, where the first video is The original parameter set carries the original parameter set ID;
  • the parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or
  • the parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID of the parameter set type of the parameter set ID to be modified, and a parameter set ID of a parameter set for decoding the VCL NALU corresponding to the original parameter set ID;
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU. It can be seen from the foregoing embodiment that the application server determines that the original parameter set ID of the first video parameter set of the same type is the same as the parameter set ID of the second video parameter set, and determines the offset of the original parameter set ID of the first video of the type.
  • a parameter set ID for decoding a parameter set of the VCL NALU and directly sending a parameter set identification modification message of the first video to the splicer, so that the splicer identifies the parameter set of the first video to modify the message, thereby eliminating the need for a splicer
  • 15A is a schematic structural diagram of a decoding apparatus for a digital video code stream according to an embodiment of the present invention, including a receiving module 1501, a first obtaining module 1502, a second obtaining module 1503, a decoding input module 1504, a determining module 1505, and a sending module 1506.
  • the receiving module 1501 is configured to receive a parameter set identifier modification message of the first video code stream and the first video, where the parameter set identifier modification message of the first video includes the parameter set identifier ID modification information of the first video; a first obtaining module 1502, configured to acquire, according to the parameter set ID modification information of the first video received by the receiving module 1501, a parameter set ID of a parameter set for decoding a video encoding layer VCL NALU of the network abstraction layer unit;
  • the obtaining module 1503 is configured to acquire, according to the parameter set ID of the parameter set for decoding the VCL NALU acquired by the first acquiring module 1502, a parameter set for decoding the VCL NALU, and the decoding output module 1504, according to the second
  • the parameter set for decoding the VCL NALU acquired by the obtaining module 1503 decodes the VCL NALU and outputs the decoded first video.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified;
  • the first video code stream includes an original parameter set ID for decoding a parameter set of the VCL NALU;
  • the first obtaining module 1502 is specifically configured to: according to the parameter set to be modified received by the receiving module 1501 Obtaining, by the parameter set type of the ID, the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified from the first code stream; according to the original parameter set ID and the parameter to be modified
  • the offset of the original parameter set ID corresponding to the parameter set type of the set ID is obtained by acquiring the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified and a parameter set ID of the parameter set for decoding the VCL NALU
  • the first obtaining module 1502 is specifically configured to: Obtaining, according to the parameter set type of the parameter set ID to be modified received by the receiving module 1501, from the parameter set ID modification information, the type corresponding to the parameter set type of the parameter set ID to be modified, for decoding the VCL NALU
  • the parameter set ID of the parameter set is an original parameter set ID of a parameter set type of the parameter set ID to be modified, and a parameter corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU.
  • the first acquiring module is configured to: obtain, according to the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified received by the receiving module, the original parameter set ID Decoding the parameter set ID of the parameter set of the VCL NALU; or the first obtaining module is specifically configured to obtain, according to the parameter set type of the parameter set ID to be modified received by the receiving module, from the first video code stream
  • the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified acquiring the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID according to the original parameter set ID.
  • the first video parameter set ID modification information is a parameter set to be modified.
  • the parameter set type of the ID further includes: a sending module 1506, configured to send a real-time transport control protocol RTCP request to the splicer, where the RTCP request includes a parameter set type of the parameter set ID to be modified; 1501 is further configured to: receive an RTCP response returned by the splicer, where the RTCP response carries an offset of an original parameter set ID corresponding to a parameter set type of a parameter set ID to be modified, or a parameter set ID to be modified a parameter set ID of a parameter set for decoding a VCL NALU corresponding to a parameter set type, or the original parameter set ID of a parameter set type of a parameter set ID to be modified, and a corresponding to the original parameter set ID Decoding a parameter set ID of a parameter set of the VCL NALU; the first obtaining module 1502 is specifically configured to: according to the parameter set type of the parameter set ID to be modified carried in the RTCP response received by the receiving module 1501 The offset of the original parameter set ID or
  • the parameter set ID modification information further includes a modification effective flag.
  • Receiving module The method 1501 is further configured to: before receiving the parameter set identifier modification message of the first video code stream and the first video, receive a parameter set identifier modification message of the first video that is not valid; the apparatus further includes: the determining module 1505 is further configured to: Determining, by the receiving module, that the parameter set identifier modification message of the inactive first video includes the flag information indicating that the parameter set ID modification information is not valid; the sending module 1506 is configured to send the second media description information to the application server.
  • the receiving module 1501 is further configured to: receive a second media description information acquisition response returned by the application server, where the second media description information acquisition response carries the second video parameter set, and the modified parameter set of the first video
  • the parameter set ID of the parameter set in the first video modification is the parameter set ID of the parameter set for decoding the VCL NALU.
  • the receiving module 1501 is further configured to pass the band before the second obtaining module 1503 obtains a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.
  • Obtaining a modified parameter set of the first video the parameter set ID of the modified parameter set is a parameter set ID of a parameter set for decoding the VCL NALU; or the receiving module 1501 is in the second acquiring
  • the module 1503 is further configured to obtain an original parameter set of the first video by using an out-of-band manner and to decode the VCL, before acquiring the parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU.
  • the parameter set ID of the parameter set of the NALU; the original parameter set ID is modified according to the modified parameter set ID.
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU; the first obtaining module 1502 is specifically configured to: acquire a parameter set directly received by the receiving module and the VCL NALU The parameter set ID of the parameter set directly referenced by the VCL NALU for decoding the VCL NALU, and is obtained by the second obtaining module 1503 according to the parameter set ID of the parameter set directly referenced by the VCL NALU for decoding the VCL NALU. After the parameter set directly referenced by the VCL NALU in the first video bitstream, the first obtaining module 1502 further obtains, from the directly referenced parameter set, a parameter set of a parameter set for invoking a VCL NALU indirect reference of the VCL NALU. ID; or
  • the parameter set type of the parameter set ID to be modified is the parameter set type directly referenced by the VCL NALU and the parameter set type indirectly referenced by the VCL NALU.
  • the first obtaining module 1502 is specifically configured to: respectively The parameter set ID modification information of a video acquires a parameter set ID of a parameter set of a direct reference for decoding a VCL NALU of the VCL NALU corresponding to the parameter set type directly referenced by the VCL NALU, and obtains an indirect reference with the VCL NALU Reference a parameter set ID corresponding to the parameter set of the VCL NALU corresponding to the number set type;
  • the second obtaining module 1503 is specifically configured to: obtain a directly referenced parameter of the VCL NALU according to the parameter set ID of the directly referenced parameter set acquired by the first obtaining module 1502 and the parameter set ID of the indirectly referenced parameter set. And a parameter set of the indirect reference of the VCL NALU; the decoding output module 1504 is specifically configured to: according to the directly referenced parameter set of the VCL NALU acquired by the second obtaining module 1502 and the indirect of the VCL NALU The referenced parameter set decodes the VCL NALU and outputs the decoded first video.
  • the device in FIG. 15 may be the terminal in the foregoing embodiment, and may perform any step performed by the terminal in the foregoing embodiment, and the embodiment of the present invention is not described in detail herein.
  • FIG. 16 is a splicing device for a digital video code stream according to an embodiment of the present invention, including: a receiving module
  • a module 1602 and a sending module 1603 are generated.
  • the receiving module 1601 is configured to receive, by the application server, information about the first video code stream, where the first video code stream information includes parameter set modification information of the first video.
  • the sending module 1603 is configured to send, according to the first video parameter set modification information received by the receiving module 1601, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier of the first video
  • the modification message carries the parameter set identifier ID modification information of the first video, so that the terminal acquires the parameter set ID of the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and The terminal acquires a parameter set for decoding the VCL NALU according to the parameter set ID of the parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoding.
  • the first video After the first video.
  • the receiving module 1601 is further configured to receive a real-time transmission control protocol RTCP request sent by the terminal, where The RTCP request includes a parameter set type of the parameter set ID to be modified; the sending module 1603 is further configured to return an RTCP response to the terminal, where the RTCP response carries a parameter set type corresponding to the parameter set ID to be modified.
  • the offset of the original parameter set ID, or the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or the parameter set type carrying the parameter set ID to be modified The original parameter set ID and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU.
  • the parameter set ID modification information includes a parameter set type of the parameter set ID to be modified, and an offset of the original parameter set ID corresponding to the parameter set type of the parameter set ID to be modified, or includes a to-be-modified
  • the parameter set ID modification information further includes a modification effective flag, and the sending module 1603 sends the first video code stream and the parameter set identifier modification of the first video to the terminal.
  • the receiving module 1601 is further configured to receive the second media description sent by the terminal.
  • the sending module 1603 is further configured to: return a second media description information acquisition response to the terminal, where the second media description information acquisition response carries the first video
  • the modified parameter set, the parameter set ID of the modified parameter set of the first video is a parameter set ID of a parameter set for decoding the VCL NALU.
  • the device advancement comprises a generating module 1602, configured to generate a parameter set identifier modification message of the first video according to the first video parameter set modification information.
  • the generating module 1602 is further configured to generate any message sent to the sending module 1603 according to the information received by the receiving module 1601.
  • the embodiments of the present invention are no longer detailed herein. For details, refer to the description of each step of the foregoing method embodiment.
  • the apparatus described in FIG. 16 may be a splicer in the foregoing implementation, and any of the steps performed by the splicer in the foregoing embodiment may be performed, and the embodiment of the present invention is not described in detail herein.
  • FIG. 17 is a splicing apparatus for a digital video code stream according to an embodiment of the present invention, including a first receiving module 1701, a second receiving module 1702, a determining module 1703, and a sending module 1704.
  • the first receiving module 1701 is configured to receive the media description information of the first video that is sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the parameter set of the first video. Type and original parameter set ID of the parameter set corresponding to each type;
  • the second receiving module 1702 is configured to receive media description information of the second video sent by the second content server, where the media description information of the second video carries a parameter set of the second video, a type of the second video parameter set, and The parameter set ID of the parameter set corresponding to each type;
  • a determining module 1703 configured to determine an original parameter set ID of the first video received by the first receiving module 1701 of the same type, and a second video parameter set ID received by the second receiving module 1702 Same as, and determining parameter set identification modification information of the first video;
  • the sending module 1704 is configured to send information about the first video stream to the splicer, where the information of the first video code stream includes parameter set modification information of the first video, and the parameter set modification information of the first video is carried in the first
  • the parameter set identifier of the video is modified in the message, so that the splicer sends the parameter set identification modification message of the first video to the terminal, and the terminal is configured to modify the message according to the parameter set identifier of the first video to decode and output the first message.
  • the apparatus further includes: a modifying module 1705, configured to determine, according to the determining module 1703, the parameter set identifier modification information to modify an original parameter set ID of an original parameter set of the first video to be used for decoding The parameter set ID of the parameter set of the video coding layer VCL NALU of the network abstraction layer unit;
  • the sending module is further configured to: send the modified parameter set of the first video to the terminal, where the parameter set ID in the modified parameter set is the parameter set ID of the parameter set used to decode the VCL NALU;
  • the parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of an original parameter set ID of the first video corresponding to the parameter set type of the parameter set ID to be modified, Or a parameter set type including the parameter set ID to be modified and a parameter set ID for decoding a parameter set of the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identifier modification of the first video
  • the message includes a parameter set ID of the parameter set type of the parameter set ID to be modified corresponding to the original parameter set ID for decoding the parameter set of the VCL NALU;
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU.
  • the sending module is further configured to: send, to the terminal, an original parameter set of the first video and a parameter set ID of the parameter set for decoding the VCL NALU corresponding to the original parameter set ID, where The original parameter set ID of the first video carries the original parameter set ID;
  • the parameter set identification modification message of the first video includes a parameter set type of the parameter set ID to be modified and an offset of the original parameter set ID corresponding to a parameter set type of the parameter set ID to be modified, or
  • the parameter set identification modification message of a video includes the parameter set type of the parameter set ID to be modified and the parameter set ID of the parameter set for decoding the VCL NALU corresponding to the parameter set type of the parameter set ID to be modified, or a parameter set identification modification message of a video, the original parameter set ID in a parameter set type of the parameter set ID to be modified, and a parameter set ID corresponding to the original parameter set ID for decoding a parameter set of the VCL NALU;
  • the parameter set type of the parameter set ID to be modified is a parameter set type directly referenced by the VCL NALU, and a parameter set type indirectly referenced by the VCL NALU.
  • the device shown in FIG. 17 may be an application server, and may perform any of the steps or functions performed by the application server in the foregoing embodiment.
  • the embodiments of the present invention are not described in detail herein.
  • FIG. 18 is a schematic structural diagram of a decoding apparatus or a splicing apparatus for a digital video stream according to an embodiment of the present invention, including a processor 1801, an input device 1802, an output device 1803, a memory 1804, and a communication interface 1805.
  • the computer system may specifically be a processor based computer such as a general purpose personal computer (PC), a portable device such as a tablet computer, or a smart phone.
  • Computer systems include buses, processors, memories, communication interfaces, input devices, and output devices.
  • the bus can include a path to transfer information between various components of the computer.
  • the processor 1801 can be a general purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention.
  • the computer system also includes one or more memories 1805, which may be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or Other types of dynamic storage devices that store information and instructions may also be disk storage. These memories are connected to the processor via a bus.
  • Input device 1801 can include a means for receiving data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, and the like.
  • the output device can include a means to allow output of information to the user, including display screens, printers, speakers, and the like.
  • the computer system also includes a communication interface 1804 that uses devices such as any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), and the like.
  • RAN Radio Access Network
  • WLAN Wireless Local Area Network
  • a memory 1805 such as a RAM, holds an operating system and a program for executing the inventive scheme.
  • the operating system is a program that controls the running of other programs and manages system resources.
  • the program code for carrying out the inventive arrangement is stored in a memory and is controlled by the processor 1801 for execution.
  • the memory 1805 can store instructions to cause the processor 1801 to perform the following processes:
  • the processor 1805 can also be used to perform the steps performed by the terminal or the decoding device in FIG. 1 to FIG. 15 in the method embodiment, which is not described in detail herein.
  • the processor 1801 may further perform the following process: receiving information of the first video code stream from the application server, where the information of the first video code stream includes parameter set modification of the first video Transmitting, according to the first video parameter set modification information, a first video code stream and a parameter set identifier modification message of the first video, where the parameter set identifier modification message of the first video carries a parameter set identifier of the first video
  • the ID modifies the information, so that the terminal acquires the parameter set ID of the parameter set of the video coding layer VCL NALU for decoding the network abstraction layer unit according to the parameter set ID modification information of the first video, and the terminal according to the method for decoding the VCL NALU
  • the parameter set ID of the parameter set acquires a parameter set for decoding the VCL NALU, and the terminal decodes the VCL NALU according to the parameter set for decoding the VCL NALU, and outputs the decoded first video.
  • the processor 1805 can also be used to perform the
  • the processor may further perform the following process: receiving media description information of the first video sent by the first content server, where the media description information of the first video carries the original parameter set of the first video and the first The type of the parameter set of the video and the original parameter set ID of the parameter set corresponding to each type; the media description information of the second video sent by the second content server, where the media description information of the second video carries the second video a parameter set second video parameter set type and a parameter set ID of a parameter set corresponding to each type; determining a parameter set ID of the first video of the same type is the same as a parameter set ID of a parameter set of the second video, and determining The parameter set of the first video identifies the modification information; the information of the first video stream is sent to the splicer, the information of the first video code stream includes parameter set identification modification information of the first video, and the parameter set of the first video is modified.
  • the identifier information is carried in the parameter set identifier modification message of the first video, so that the splicer sends the parameter set identifier modification message of the first video to the terminal. And, in order for the terminal to modify the message according to the parameter set identification of the first video, and output the first video.
  • the processor 1805 can also be used to perform the steps performed by the application server in FIG. 1 to FIG. 17 in the embodiment of the method.
  • the embodiments of the present invention are not described in detail herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and details are not described herein again.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed.
  • the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise.
  • the components displayed for the unit may or may not be physical units, ie may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute the method of the various embodiments of the present application. Part of the steps.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention porte sur un procédé de décodage de flux de code vidéo numérique, un procédé d'épissage de flux de code vidéo numérique et un appareil. Au moyen d'un message de modification d'identification d'un ensemble de paramètres d'informations de modification d'une identification (ID) d'un ensemble de paramètres contenant une première vidéo de la première vidéo, un terminal décode et produit en sortie la première vidéo en fonction des informations de modification d'une identification (ID) d'un ensemble de paramètres. Non seulement un ensemble de paramètres correct peut être utilisé lorsqu'un terminal décode un premier flux de code vidéo, mais également des ressources matérielles additionnelles n'ont pas besoin d'être ajoutées à un dispositif d'épissage d'un réseau existant ou il n'est pas nécessaire de remplacer le dispositif d'épissage du réseau existant par un nouveau dispositif d'épissage, ce qui nécessite des coûts d'exploitation inférieurs à ceux nécessaires dans l'état antérieur de la technique.
PCT/CN2012/082483 2012-09-29 2012-09-29 Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil WO2014047938A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2012/082483 WO2014047938A1 (fr) 2012-09-29 2012-09-29 Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil
CN201280002009.5A CN103959796B (zh) 2012-09-29 2012-09-29 数字视频码流的解码方法拼接方法和装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/082483 WO2014047938A1 (fr) 2012-09-29 2012-09-29 Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil

Publications (1)

Publication Number Publication Date
WO2014047938A1 true WO2014047938A1 (fr) 2014-04-03

Family

ID=50386903

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2012/082483 WO2014047938A1 (fr) 2012-09-29 2012-09-29 Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil

Country Status (2)

Country Link
CN (1) CN103959796B (fr)
WO (1) WO2014047938A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020239743A1 (fr) * 2019-05-28 2020-12-03 Canon Kabushiki Kaisha Procédé et appareil de codage et de décodage d'un flux binaire vidéo permettant de fusionner des régions d'intérêt

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109587478B (zh) * 2017-09-29 2023-03-31 华为技术有限公司 一种媒体信息的处理方法及装置
CN110248221A (zh) * 2019-06-18 2019-09-17 北京物资学院 一种视频广告动态插入方法及装置

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004056169A (ja) * 2002-07-16 2004-02-19 Matsushita Electric Ind Co Ltd 画像データ受信装置、画像データ送信装置
US20050254526A1 (en) * 2004-05-12 2005-11-17 Nokia Corporation Parameter sets update in streaming applications
CN101453653A (zh) * 2007-12-07 2009-06-10 中兴通讯股份有限公司 一种传播数字音视频参数集的方法
CN101582908A (zh) * 2009-06-30 2009-11-18 杭州华三通信技术有限公司 一种参数配置方法和设备
CN101606389A (zh) * 2007-01-08 2009-12-16 汤姆森特许公司 用于视频流拼接的方法及装置
CN101663893A (zh) * 2007-04-18 2010-03-03 汤姆森许可贸易公司 编码系统
WO2010122147A1 (fr) * 2009-04-24 2010-10-28 Ge Healthcare Uk Limited Procédé et appareil pour l'analyse de données multi-paramètres
CN102577387A (zh) * 2009-10-30 2012-07-11 松下电器产业株式会社 解码方法、解码装置、编码方法以及编码装置
US20120230430A1 (en) * 2011-03-10 2012-09-13 Stephan Wenger Parameter Set Maintenance in Video Coding

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004056169A (ja) * 2002-07-16 2004-02-19 Matsushita Electric Ind Co Ltd 画像データ受信装置、画像データ送信装置
US20050254526A1 (en) * 2004-05-12 2005-11-17 Nokia Corporation Parameter sets update in streaming applications
CN101606389A (zh) * 2007-01-08 2009-12-16 汤姆森特许公司 用于视频流拼接的方法及装置
CN101663893A (zh) * 2007-04-18 2010-03-03 汤姆森许可贸易公司 编码系统
CN101453653A (zh) * 2007-12-07 2009-06-10 中兴通讯股份有限公司 一种传播数字音视频参数集的方法
WO2010122147A1 (fr) * 2009-04-24 2010-10-28 Ge Healthcare Uk Limited Procédé et appareil pour l'analyse de données multi-paramètres
CN101582908A (zh) * 2009-06-30 2009-11-18 杭州华三通信技术有限公司 一种参数配置方法和设备
CN102577387A (zh) * 2009-10-30 2012-07-11 松下电器产业株式会社 解码方法、解码装置、编码方法以及编码装置
US20120230430A1 (en) * 2011-03-10 2012-09-13 Stephan Wenger Parameter Set Maintenance in Video Coding

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020239743A1 (fr) * 2019-05-28 2020-12-03 Canon Kabushiki Kaisha Procédé et appareil de codage et de décodage d'un flux binaire vidéo permettant de fusionner des régions d'intérêt
US20220217355A1 (en) * 2019-05-28 2022-07-07 Canon Kabushiki Kaisha Method and apparatus for encoding and decoding a video bitstream for merging regions of interest

Also Published As

Publication number Publication date
CN103959796B (zh) 2017-11-17
CN103959796A (zh) 2014-07-30

Similar Documents

Publication Publication Date Title
US20230283863A1 (en) Retrieving and accessing segment chunks for media streaming
TWI846795B (zh) 用於經串流媒體資料之多個解碼器介面
KR20180089341A (ko) 미디어 데이터를 송수신하기 위한 인터페이스 장치 및 방법
CN112154672B (zh) 一种检索媒体数据的方法、设备及可读存储介质
CA3104173C (fr) Methode de signalement de renseignements de biens de sous-titres et dispositif connexe
JP6254291B2 (ja) Ｄａｓｈのロバストなライブ動作
WO2018187318A1 (fr) Types de segments en tant que délimiteurs et identifiants de ressources adressables
CN116055781A (zh) 动态条件性广告插入
US9369508B2 (en) Method for transmitting a scalable HTTP stream for natural reproduction upon the occurrence of expression-switching during HTTP streaming
US20180176278A1 (en) Detecting and signaling new initialization segments during manifest-file-free media streaming
KR102549656B1 (ko) 미디어 데이터 스트리밍을 위한 sei 트랙들의 시스템 레벨 시그널링
CN112771876B (zh) 检索媒体数据的方法和设备以及发送媒体数据的方法和设备
CN116762346A (zh) 媒体数据的后台数据业务分布
WO2014047938A1 (fr) Procédé de décodage de flux de code vidéo numérique, procédé d'épissage et appareil
JP2024525323A (ja) リアルタイム拡張現実通信セッション
US20240259454A1 (en) Method, An Apparatus, A Computer Program Product For PDUs and PDU Set Handling
TW202423095A (zh) 回應於網路中斷的視訊內容的自動產生
Wenger et al. RFC 6190: RTP Payload Format for Scalable Video Coding
BR112016016434B1 (pt) Método de transmissão adaptativa dinâmica através de http, dispositivo para receber, a partir de um dispositivo de servidor, dados relacionados a dados de mídia de transmissão contínua dash, método e dispositivo de sinalização

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12885922

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12885922

Country of ref document: EP

Kind code of ref document: A1