WO2020038309A1 - 一种实现视频流切换的方法、装置和系统 - Google Patents

一种实现视频流切换的方法、装置和系统 Download PDF

Info

Publication number
WO2020038309A1
WO2020038309A1 PCT/CN2019/101241 CN2019101241W WO2020038309A1 WO 2020038309 A1 WO2020038309 A1 WO 2020038309A1 CN 2019101241 W CN2019101241 W CN 2019101241W WO 2020038309 A1 WO2020038309 A1 WO 2020038309A1
Authority
WO
WIPO (PCT)
Prior art keywords
message
frame
video stream
switching
video
Prior art date
Application number
PCT/CN2019/101241
Other languages
English (en)
French (fr)
Inventor
张锦辉
熊婕
李吉
刘文斌
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to KR1020237010789A priority Critical patent/KR102671249B1/ko
Priority to JP2021509807A priority patent/JP7224439B2/ja
Priority to KR1020217006921A priority patent/KR20210041052A/ko
Priority to EP19851740.1A priority patent/EP3836552A4/en
Publication of WO2020038309A1 publication Critical patent/WO2020038309A1/zh
Priority to US17/180,568 priority patent/US11483495B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/633—Control signals issued by server directed to the network components or client
    • H04N21/6338—Control signals issued by server directed to the network components or client directed to network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/64322—IP
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/38—Transmitter circuitry for the transmission of television signals according to analogue transmission standards

Definitions

  • Embodiments of the present invention relate to the field of multimedia communication technologies, and in particular, to a method, an apparatus, and a system for implementing video stream switching.
  • SDI serial digital interface
  • IP Internet Protocol
  • a transmission device determines that a video stream needs to be switched, it stops sending data packets of the video stream to be cut out and starts sending data of the video stream to be cut in. Message.
  • Embodiments of the present invention provide a method, a device, and a system for implementing video stream switching, which are used to solve the problem of damage to a video picture caused by video stream switching in the prior art.
  • a method for switching a video stream includes: a transmission device receives a first video stream and a second video stream, and sends the first video stream through a target egress port. After switching instructions for switching the first video stream to the second video stream sent through the target outbound port, switch reference information (such as a sequence number) of a first reference frame tail message of the first video stream is determined according to the switching instruction.
  • the switching reference information of the switch switches the first video stream sent through the target egress port to the second video stream.
  • the handover reference information of the first reference frame tail message, the second reference frame tail message, the first handover message and the second handover message may all be serial numbers or all timestamps, or they may all include the serial number and Timestamp.
  • the switch message of the cut-out stream and the cut-in stream is determined according to the switching reference information of the end-of-frame message of the cut-out stream (that is, the first video stream) and the cut-in stream (that is, the second video stream).
  • the switching reference information (such as a serial number), and the video stream is switched according to the switching reference information of the switching message. Because the position of the data message sent before and after the switching is considered when the video stream is switched, the damage to the transmitted video image by the switching operation can be effectively reduced.
  • the first reference frame tail message and the second reference frame tail message may specifically be switching reference information for determining the first reference frame tail message and the second reference frame tail message, respectively.
  • the reference frame tail message switching reference information received before the penultimate frame-end message of the first video stream and the penultimate frame-end message of the second video stream, where x is a natural number and can be 1 .
  • the timestamps of the first reference frame tail message and the second reference frame tail message determined according to this implementation manner are relatively close, so it is possible to more accurately determine the first handover message and the second handover message with the same time stamp. , Thereby further reducing the damage caused by the switching operation to the transmitted video picture.
  • the value of x is 1, and the video frame to which the video data in the first handover packet belongs is the first reference frame.
  • the video frame to which the video data in the second reference frame tail message belongs is the next video frame in sequence or the subsequent video frame in the next video frame in sequence.
  • the sequence number of the first switching message may be calculated according to the sequence number of the first reference frame tail message and the number of messages of a single video frame encapsulating the first video stream, and according to the second
  • the sequence number of the reference frame tail message and the number of messages of a single video frame encapsulating the second video stream calculate the sequence number of the second handover message, and determine the sequence number of the first handover message and
  • the difference between the sequence number of the first reference frame tail message may be greater than or equal to the number of messages of a single video frame encapsulating the first video stream, and the determined sequence number of the second switching message and the The difference between the sequence numbers of the second reference frame tail messages may be greater than or equal to the number of messages of a single video frame encapsulating the second video stream.
  • the time stamp of the first switching message may be calculated according to the time stamp of the first reference frame tail message and the duration of a single video frame of the first video stream, and according to the second reference frame tail
  • the timestamp of the message and the duration of a single video frame of the second video stream calculate the timestamp of the second handover message, and the timestamp of the first handover message determined is the same as the end of the first reference frame
  • the difference between the timestamp of the message may be greater than or equal to the duration of a single video frame of the first video stream, and the timestamp of the determined second switch message and the timestamp of the second reference frame tail message are determined.
  • the difference may be greater than or equal to the duration of a single video frame of the second video stream.
  • video stream switching is performed at the position of at least one frame after the reference frame tail message, which can effectively avoid damage to the video picture currently being transmitted.
  • the switching reference information of the first reference frame tail message and the first determines the handover reference information of the first handover message, and determines the second handover message according to the handover reference information of the second reference frame tail message and the synchronization relationship.
  • the switching of the video stream can be realized more accurately, thereby further reducing the switching operation Damage to the transmitted video picture.
  • the first reference video frame tail message switching reference information and the synchronization relationship may be used to determine the first A switch reference information of the aligned frame tail message, and the switch reference information of the first switch frame is determined according to the switch reference information of the first aligned frame foot message, and the switch of the second reference frame tail message may also be performed.
  • the reference information and the synchronization relationship determine the switching reference information of the second aligned frame-tail message of the second video stream, and determine the switching reference of the second switching message according to the switching reference information of the second aligned frame-tail message Information, the first aligned frame-tail message and the second aligned frame-tail message are time-corresponding frame tail messages.
  • the first aligned frame trailer message and the second aligned frame trailer message determined in consideration of the synchronization relationship are temporally corresponding frame trailer messages. Therefore, according to the first aligned frame trailer message, The first handover message and the second handover message determined separately from the second aligned frame tail message also correspond in time, so the damage to the transmitted video picture by the handover operation can be further reduced.
  • the switching reference information of the first aligned frame tail message may be directly used as the switching reference information of the first switching message and the first
  • the switching reference information of the second aligned frame-tail message is used as the switching reference information of the second switching message, that is, the first aligned frame-tail message is directly used as the first switching message and the second aligned frame-tail message is used as the second switch. Message.
  • switching of a video stream at a boundary of a video frame can be implemented, and damage to a transmitted video image by the switching operation can be effectively avoided.
  • the cancellation of the next video frame for encapsulating the video frame to which the video data in the first aligned frame tail message belongs can be performed.
  • the switching reference information of the data message of the hidden area data is determined as the switching reference information of the first switching message, and will be used to encapsulate the next video frame of the video frame to which the video data in the second aligned frame tail message belongs.
  • the switching reference information of the data message of the blanking area data is determined as the switching reference information of the second switching message.
  • the video stream at the data message in the blanking area of the encapsulated video frame. Since the blanking area data is not valid video data, it is invisible to the human eye. Damage to valid video data.
  • the sequence number of the first reference frame tail message and m The sum of * M1 is determined as the sequence number of the first aligned frame-tailed message, and the sum of the sequence number of the second reference frame-tailed message and (m + 1) * M2 is determined as the second-aligned frame-tailed message
  • the serial number of the text when it is determined that the synchronization relationship is that the first video stream is earlier than the second video stream, the sequence number of the first reference frame tail message and m
  • * M1 is determined as the sequence number of the first aligned frame-tailed message, and the sum of the sequence number of the second reference frame-tailed message and (m + 1) * M2 is determined as the second-aligned frame-tailed message.
  • the sum of the sequence number of the first reference frame tail message and (n + 1) * M1 is determined as the first aligned frame tail
  • the sequence number of the message, and the sum of the sequence number of the second reference frame tail message and n * M2 is determined as the sequence number of the second aligned frame tail message.
  • the sum of the sequence number of the first reference frame trailer message and k * M1 may be determined as the first aligned frame trailer
  • the sequence number of the text, and the sum of the sequence number of the second reference frame tail message and k * M2 is determined as the sequence number of the second aligned frame tail message.
  • M1 is the number of data messages required to encapsulate a video frame of the first video stream
  • M2 is the number of data messages required to encapsulate a video frame of the second video stream
  • m, n and k are Is a natural number.
  • the transmission device may specifically according to the timestamp and The time stamp of the second reference frame tail message determines a synchronization relationship between the first video stream and the second video stream.
  • the synchronization relationship can reflect that the transmission device receives the first reference frame tail message and The synchronization status at the time of the second reference frame tail message, so the first handover determined according to the combination of the first reference frame tail message's switching reference information, the second reference frame tail message's switching reference information, and the synchronization relationship
  • the switching reference information of the message and the switching reference information of the second switching message are more in line with the true synchronization status of the video stream, which can further reduce the damage to the transmitted video picture by the switching operation.
  • the time stamps of data packets sent from different video source devices at the same time may be different.
  • IP networks use packets as a unit, and the jitter of packet forwarding will also introduce some synchronization deviations. Therefore, it can be considered that if the time stamp deviation of the reference frame tail message of the two video streams is less than a preset time difference threshold, the two video streams can be considered to be synchronized.
  • the transmission device may specifically determine that the first video stream is earlier than the second video stream when determining t1-t2> ⁇ t1; and determine that the first video stream is later than the second video stream when determining t2-t1> ⁇ t1 Video stream; when determining
  • t1 is the timestamp of the first reference frame tail message
  • t2 is the timestamp of the second reference frame tail message
  • ⁇ t1 is a preset first time difference threshold
  • ⁇ t2 is a preset second time difference threshold
  • ⁇ t1 and ⁇ t2 is a value greater than 0 and less than T, and ⁇ t2 may be the same as ⁇ t1 or less than ⁇ t1.
  • ⁇ t1 and ⁇ t2 can be preset according to the following conditions: T / 2 ⁇ t1 ⁇ T, such as ⁇ t1 can be 3 * T / 4, 0 ⁇ t2 ⁇ T / 2, such as ⁇ t2 can be T / 4, and T is a single video frame. duration.
  • the determined synchronization relationship excludes the influence of the accuracy deviation of the device synchronization itself, the jitter of message forwarding, etc., and can more accurately reflect the true synchronization status between the video streams.
  • the determined switching of the first switching message The reference information and the handover reference information of the second handover message are more in line with the true synchronization status of the video stream, which can further reduce the damage of the transmitted video picture by the handover operation.
  • a transmission device in a second aspect, includes a processing unit and a communication unit.
  • the communication unit is configured to receive a first video stream and a second video stream, send the first video stream through a target output port of the transmission device, and receive a switching instruction, which is used to indicate that the target video is to be sent through the target output port.
  • the first video stream to the second video stream.
  • the processing unit is configured to determine switching reference information (such as a sequence number) of a first reference frame tail message of the first video stream and a reference reference of a second reference frame tail message of the second video stream according to the switching instruction. Information, determining the switching reference information of the first switching message of the first video stream according to the switching reference information of the first reference frame tail message, and determining the second based on the switching reference information of the second reference frame tail message Reference information for switching the second switching message of the video stream.
  • switching reference information such as a sequence number
  • the communication unit is further configured to switch the first video stream sent through the target egress port to the second video stream according to the switching reference information of the first switching message and the switching reference information of the second switching message.
  • the handover reference information of the first reference frame tail message, the second reference frame tail message, the first handover message and the second handover message may all be serial numbers or all timestamps, or they may all include the serial number and Timestamp.
  • the switching message of the cut-out stream and the cut-in stream is determined according to the switching reference information of the end-of-frame message of the cut-out stream (that is, the first video stream) and the cut-in stream (that is, the second video stream).
  • the switching reference information (such as a serial number), and the video stream is switched according to the switching reference information of the switching message. Because the position of the data message sent before and after the switching is considered when the video stream is switched, the damage to the transmitted video image by the switching operation can be effectively reduced.
  • the first reference frame tail message and the second reference frame tail message may specifically be switching reference information for determining the first reference frame tail message and the second The reference frame tail message switching reference information received before the penultimate frame-end message of the first video stream and the penultimate frame-end message of the second video stream, where x is a natural number and can be 1 .
  • the timestamps of the first reference frame tail message and the second reference frame tail message determined according to this implementation manner are relatively close, so it is possible to more accurately determine the first handover message and the second handover message with the same time stamp. , Thereby further reducing the damage caused by the switching operation to the transmitted video picture.
  • the value of x is 1, and the video frame to which the video data in the first handover packet belongs is the first reference frame.
  • the video frame to which the video data in the trailer message belongs is the next video frame in sequence or the subsequent video frame to the next video frame in sequence; the video frame to which the video data in the second switching message belongs is the first
  • the video frame to which the video data in the second reference frame tail message belongs is the next video frame in sequence or the subsequent video frame in the next video frame in sequence.
  • video stream switching is performed at the position of at least one frame after the reference frame tail message, which can effectively avoid damage to the video picture currently being transmitted.
  • the processing unit may specifically according to the switching reference information of the first reference frame tail message, And the synchronization relationship between the first video stream and the second video stream determines the handover reference information of the first handover message, and determines the handover reference information according to the handover reference information of the second reference frame end message, and the synchronization relationship.
  • Handover reference information for the second handover message may specifically according to the switching reference information of the first reference frame tail message, And the synchronization relationship between the first video stream and the second video stream determines the handover reference information of the first handover message, and determines the handover reference information according to the handover reference information of the second reference frame end message, and the synchronization relationship.
  • the switching of the video stream can be realized more accurately, thereby further reducing the switching operation Damage to the transmitted video picture.
  • the processing unit may determine the first video according to the switching reference information of the first reference frame tail message and the synchronization relationship.
  • Switching reference information, the first aligned end-of-frame message and the second aligned end-of-frame message are corresponding end-of-frame messages in time.
  • the first aligned frame trailer message and the second aligned frame trailer message determined in consideration of the synchronization relationship are temporally corresponding frame trailer messages. Therefore, according to the first aligned frame trailer message, The first handover message and the second handover message determined separately from the second aligned frame tail message also correspond in time, so the damage to the transmitted video picture by the handover operation can be further reduced.
  • the processing unit may directly use the switching reference information of the first aligned frame tail message as the switching reference information of the first switching message.
  • the switching reference information of the second aligned frame tail message is used as the switching reference information of the second switching message, that is, the first aligned frame tail message is directly used as the first switching message and the second aligned frame tail message is used as Second switching message.
  • switching of a video stream at a boundary of a video frame can be implemented, and damage to a transmitted video image by the switching operation can be effectively avoided.
  • the processing unit may encapsulate the next video of the video frame to which the video data in the first aligned frame trailer message belongs.
  • the switching reference information of the data message of the blanking area data of the frame is determined as the switching reference information of the first switching message, and will be used to encapsulate the next frame of the video frame to which the video data in the second aligned frame tail message belongs.
  • the switching reference information of the data message of the blanking area data of the video frame is determined as the switching reference information of the second switching message.
  • the transmission device may specifically according to the timestamp and The time stamp of the second reference frame tail message determines a synchronization relationship between the first video stream and the second video stream.
  • the synchronization relationship can reflect that the transmission device receives the first reference frame tail message and The synchronization status at the time of the second reference frame tail message, so the first handover determined according to the combination of the first reference frame tail message's switching reference information, the second reference frame tail message's switching reference information, and the synchronization relationship
  • the switching reference information of the message and the switching reference information of the second switching message are more in line with the true synchronization status of the video stream, which can further reduce the damage to the transmitted video picture by the switching operation.
  • the time stamps of data packets sent from different video source devices at the same time may be different.
  • IP networks use packets as a unit, and the jitter of packet forwarding will also introduce some synchronization deviations. Therefore, it can be considered that if the time stamp deviation of the reference frame tail message of the two video streams is less than a preset time difference threshold, the two video streams can be considered to be synchronized.
  • the processing unit may specifically determine that the first video stream is earlier than the second video stream when determining t1-t2> ⁇ t1; and determine that the first video stream is later than the first video stream when determining t2-t1> ⁇ t1.
  • t1 is the timestamp of the first reference frame tail message
  • t2 is the timestamp of the second reference frame tail message
  • ⁇ t1 is a preset first time difference threshold
  • ⁇ t2 is a preset second time difference threshold
  • ⁇ t1 and ⁇ t2 is a value greater than 0 and less than T, and ⁇ t2 may be the same as ⁇ t1 or less than ⁇ t1.
  • ⁇ t1 and ⁇ t2 can be preset according to the following conditions: T / 2 ⁇ t1 ⁇ T, such as ⁇ t1 can be 3 * T / 4, 0 ⁇ t2 ⁇ T / 2, such as ⁇ t2 can be T / 4, and T is a single video frame. duration.
  • the determined synchronization relationship excludes the influence of the accuracy deviation of the device synchronization itself, the jitter of message forwarding, etc., and can more accurately reflect the true synchronization status between the video streams.
  • the determined switching of the first switching message The reference information and the handover reference information of the second handover message are more in line with the true synchronization status of the video stream, which can further reduce the damage of the transmitted video picture by the handover operation.
  • a system for switching video streams includes a controller and the transmission device according to the second aspect or any one of the implementation manners of the second aspect, wherein the controller is configured to The transmission device sends a switching instruction.
  • a computer-readable storage medium stores instructions, and when the computer-readable storage medium runs on the computer, the computer executes the first aspect or any implementation of the first aspect. Way described.
  • a computer program product containing instructions that, when run on a computer, causes the computer to execute the method according to the first aspect or any one of the implementation manners of the first aspect.
  • FIG. 1 is a schematic diagram of a networking structure of a multimedia communication system 100 according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method 100 according to an embodiment of the present invention.
  • 3A-3C and 4A-4C are schematic diagrams of handover provided by an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a transmission device 200 according to Embodiment 2 of the present invention.
  • FIG. 6 is a schematic structural diagram of a transmission device 1000 according to Embodiment 3 of the present invention.
  • FIG. 1 is a schematic diagram of a networking structure of a multimedia communication system 100 according to an embodiment of the present invention.
  • the multimedia communication system 100 includes a video source device 110 and a transmission device 120.
  • the video source device 110 sends a video stream to the video terminal through the transmission device 120.
  • the transmission device 120 may receive video streams from multiple video source devices 110, and forward only one of the video streams to the video terminal, and may also switch the forwarded video streams.
  • the transmission device 120 may specifically be a switch, a router, etc., and the video source device may be an IP camera, a device that performs IP encapsulation and transmission of video data, and the like.
  • the multimedia communication system 100 may specifically be a TV production and broadcast system, such as a system for a TV station production and broadcast network, a studio system, and a broadcast system, which have a net switching requirement for video streams carried on an IP network.
  • the multimedia communication system 100 may further include a controller 130.
  • the controller 130 is configured to send a switching instruction to the transmission device 120 to instruct the transmission device 120 to switch the video stream. Accordingly, the transmission device 120 After receiving the switching instruction, the video stream is switched.
  • each video frame of a video is encapsulated in multiple data packets for transmission, and the last data packet among the multiple data packets that encapsulates each video frame is referred to as a frame tail packet.
  • the data message in the embodiment of the present invention carries a sequence number and / time stamp, and information used to determine whether the data message is a frame-tail message, and may specifically be a Real-time Transport Protocol (RTP) message. Text, messages based on private protocols, etc.
  • RTP is used to carry the audio and video data of the production network.
  • the standard SMPTE 2022-6 Transport of High Bit Rate Media Signals IP Networks (HBRMT)
  • HBRMT Transport of High Bit Rate Media Signals IP Networks
  • the video frame in the embodiment of the present invention refers to an uncompressed video frame, which may be a YUV video frame or an RGB video frame.
  • the number of messages of a single video frame of the same video stream (that is, the number of packets required to encapsulate a video frame of the video stream)
  • the number of data packets) is generally the same, usually up to thousands, it can be preset, or it can be based on the difference between the sequence numbers of two adjacent frame-tail messages received in the video stream. Value calculated.
  • the duration of a single video frame of the same video stream is usually the same, it can be preset, or it can be calculated based on the difference between the timestamps of the two adjacent frame-end messages of the received video stream. .
  • the handover message in the embodiment of the present invention refers to a data message in a handover position.
  • the handover reference information of the handover message is information for video stream handover in the handover message, and may include the handover.
  • At least one of a sequence number of a message hereinafter may be simply referred to as a switching sequence number
  • a time stamp of the switching message hereinafter may be simply referred to as a switching time stamp.
  • FIG. 2 is a schematic flowchart of a method provided in Embodiment 1 of the present invention.
  • the method in Embodiment 1 of the present invention can be applied to the multimedia communication system 100 shown in FIG. 1 and executed by a transmission device.
  • the method 100 includes the following steps:
  • the transmission device (such as the transmission device 120 shown in FIG. 1) receives the first video stream and the second video stream and sends the first video stream through a target output port of the transmission device.
  • the transmission device forwards the received data packet of the first video stream through the target out port and does not forward the received data packet of the second video stream through the target out port.
  • the transmission device may receive multiple video streams corresponding to the target output port, send one video stream of the multiple video streams through the target output port, and send the video streams sent through the target output port in the multiple video streams.
  • the video stream is switched.
  • the first video stream is a video stream that is being sent through the target output port when step S110 is performed, and the second video stream is one of the other video streams of the plurality of video streams.
  • the transmission device receives a switching instruction for instructing to switch the first video stream sent to the second video stream through the target egress port.
  • the switching instruction may be received from a controller (such as the controller 130 shown in FIG. 1).
  • the switching instruction may include the port number of the target egress port, the identifier of the first video stream (such as the multicast address of the first video stream), and the identifier of the second video stream (such as the multicast address of the second video stream) To instruct the transmission device to switch the first video stream sent through the target out port to a second video stream, both to instruct the transmission device to stop forwarding the first video stream through the target out port and start passing through the target out port Forward the second video stream.
  • the transmission device determines, according to the switching instruction, switching reference information of a reference frame tail message of the first video stream (hereinafter referred to as a first reference frame tail message) and a reference frame of the second video stream. Switching reference information for the tail message (hereinafter referred to as the second reference frame tail message).
  • the switching reference information of the first reference frame tail message and the switching reference information of the second reference frame tail message may both be a serial number, both a timestamp, or both include a serial number and a timestamp.
  • the penultimate frame-end message of the first video stream and the penultimate frame-end message of the second video stream received before executing step S130 may be determined as the first reference frame-tail respectively.
  • the value of x can be 1.
  • the reference frame tail message of any video stream in the first video stream and the second video stream can also be referred to as the latest frame tail message of the video stream when step S130 is performed.
  • the switching reference information of the first reference frame tail message and the switching reference information of the second reference frame tail message may be determined according to a time when the transmission device receives the switching instruction.
  • Step S130 is executed immediately after receiving the switching instruction, that is, the execution time of steps S120 and S130 is very close, then the penultimate x-th frame of the first video stream received before the switching instruction is received
  • the switching reference information of the tail message and the switching reference information of the received last penultimate x-th frame of the second video stream are determined as the switching reference information of the first reference frame tail message and the second Reference switching information of the reference frame tail message, that is, the first reference frame tail message and the second reference frame tail message are the penultimate number of the first video stream received before receiving the switching instruction, respectively.
  • the first reference frame tail message and the second reference frame tail message when x is 1 are shown in FIGS. 3A-3C.
  • the frame tail message 101 and the frame tail message 201 are determined as the first reference frame tail message and the second reference frame tail message, respectively, and the sequence numbers are 20 and 10, respectively.
  • the frame tail message 103 and the frame tail message 203 are respectively determined as the first reference frame tail message and the second reference frame tail message, and the sequence numbers are 10 and 20, respectively.
  • the frame tail message 105 and the frame tail message 205 are respectively determined as the first reference frame tail message and the second reference frame tail message, and the sequence numbers are 10 and 10, respectively.
  • Step S130 is performed after receiving at least one frame trailer message of the first video stream and at least one frame trailer message of the second video stream after receiving the switching instruction.
  • Step S130 may be performed when at least one frame trailer message of the first video stream and at least one frame trailer message of the second video stream are received, and the last frame trailer message of the first video stream received at this time may be executed.
  • the switching reference information of the last frame tail message of the second video stream are determined as the switching reference information of the first reference frame tail message and the switching reference information of the second reference frame tail message, respectively.
  • the frame tail message 107 and the frame tail message 207 are respectively determined as the first reference frame tail message and the second reference frame tail message, and the sequence numbers are 30 and 20, respectively.
  • the frame tail message 109 and the frame tail message 209 are respectively determined as the first reference frame tail message and the second reference frame tail message, and the sequence numbers are 20 and 30, respectively.
  • the frame tail message 111 and the frame tail message 211 are respectively determined as the first reference frame tail message and the second reference frame tail message, and the sequence numbers are 20 and 20, respectively. If the number of messages in a single video frame is calculated for the first video stream and the second video stream according to the sequence number of the adjacent frame tail message after receiving the switching instruction, the first video stream may also be received separately.
  • step S130 is performed to switch the reference information of the last frame trailer message of the first video stream received at this time and the second video.
  • the switching reference information of the last frame tail message of the stream is determined as the switching reference information of the first reference frame tail message and the switching reference information of the second reference frame tail message, respectively.
  • S140 Determine handover reference information of a handover message of the first video stream (hereinafter referred to as a first handover message) according to the handover reference information of the first reference frame tail message, and according to the second reference frame
  • the switching reference information of the tail message determines the switching reference information of the switching message of the second video stream (hereinafter referred to as a second switching message).
  • Switching reference information of the first reference frame tail message, switching reference information of the second reference frame tail message, switching reference information of the first handover message, and switching reference of the second handover message The information can be all serial numbers, all timestamps, or both serial numbers and timestamps.
  • the sequence number of the first handover message may be referred to as the first handover sequence number
  • the sequence number of the second handover message may be referred to as the second handover sequence number
  • the timestamp of the first handover message is referred to as The first handover timestamp and the sequence number of the second handover message are referred to as the second handover timestamp.
  • the video frame to which the video data in the first switching message belongs is the next video frame in the sequence of the video frame to which the video data in the first reference frame tail message belongs or the subsequent video frame in the sequence.
  • Video frame; the video frame to which the video data in the second switch message belongs is the next video frame in sequence or the next video in sequence to the video frame in the second reference frame tail message The subsequent video frames of the frame.
  • the first switching sequence number may be calculated according to a sequence number of the first reference frame tail message and a number of messages of a single video frame encapsulating the first video stream, and according to the second reference frame tail
  • the sequence number of the message and the number of messages of a single video frame encapsulating the second video stream calculate the second switching sequence number, and the determined first switching sequence number and the first reference frame tail message
  • the difference between the sequence number of the second video frame may be greater than or equal to the number of packets of a single video frame encapsulating the first video stream.
  • the difference may be greater than or equal to the number of packets of a single video frame encapsulating the second video stream.
  • the first switching timestamp may be calculated according to a time stamp of the first reference frame tail message and a duration of a single video frame of the first video stream, and according to the second reference frame tail message, The time stamp and the length of a single video frame of the second video stream are used to calculate the second switching time stamp.
  • the difference between the determined first switching time stamp and the time stamp of the first reference frame tail message may be Greater than or equal to the duration of a single video frame of the first video stream
  • the difference between the determined second switching timestamp and the timestamp of the second reference frame end message may be greater than or equal to the second video
  • the duration of a single video frame of a stream may be calculated according to a time stamp of the first reference frame tail message and a duration of a single video frame of the first video stream, and according to the second reference frame tail message, The time stamp and the length of a single video frame of the second video stream are used to calculate the second switching time stamp.
  • the transmission device may first determine the switching of the aligned frame trailer message (hereinafter referred to as the first aligned frame trailer message) of the first video stream according to the switching reference information of the first reference frame trailer message.
  • Reference information and then determine the handover reference information of the first handover message according to the handover reference information of the first aligned frame tail message, for example, use the handover reference information of the first aligned frame tail message as the first
  • the switching reference information of the switching message is, for example, the switching reference information of the data message that is used to encapsulate the blanking area data of the next video frame of the video frame to which the video data in the first aligned frame tail message belongs.
  • the handover reference information of the first handover message and the first aligned frame trailer message is a subsequent frame trailer message of the first reference frame trailer message.
  • the switching reference information of the aligned frame tail message (hereinafter referred to as the second aligned frame tail message) of the second video stream may be determined according to the switching reference information of the second reference frame tail message of the second video stream.
  • determining the handover reference information of the second handover message according to the handover reference information of the second aligned frame tail message for example, using the handover reference information of the second aligned frame tail message as the second handover message
  • Switching reference information of another for example, determining the switching reference information of a data message used to encapsulate the blanking area data of the next video frame of the video frame to which the video data in the second aligned frame tail message belongs.
  • Switching reference information of the second handover message, and the second aligned frame trailer message is a subsequent frame trailer message of the second reference frame trailer message.
  • the first aligned end-of-frame message and the second aligned end-of-frame message are time-corresponding end-of-frame messages, that is, for the first aligned end-of-frame message, the second aligned end-of-frame message is of the second video.
  • the end-of-frame message that is closest to the timestamp of the first aligned end-of-frame message in all the end-of-frame messages is the video frame closest to its time among all the video frames of the second video.
  • Switching reference information for the first reference frame tail message, switching reference information for the second reference frame tail message, switching reference information for the first aligned frame tail message, and switching reference for the second aligned frame tail message may all be a serial number, both a timestamp, or both include a serial number and a timestamp.
  • the switching reference information of the aligned frame-tail message of any one of the first video stream and the second video stream may be determined through one or a combination of Embodiment A and Embodiment B below, and the video stream may be determined.
  • Embodiment A is used to determine the sequence number of the handover message
  • Embodiment B is used to determine the time stamp of the handover message.
  • the transmission device may first calculate the sequence number of the first aligned frame tail message according to the sequence number of the first reference frame tail message and the number of messages of a single video frame of the first video stream. , And then determining the sequence number of the first handover message according to the sequence number of the first aligned frame tail message.
  • the sequence number of the second aligned frame tail message may be calculated according to the sequence number of the second reference frame tail message of the second video stream and the number of messages of a single video frame of the second video stream. The sequence number of the second aligned frame tail message determines the sequence number of the second handover message.
  • the sequence number 20 of the first reference frame tail message 101 and the sequence number 10 of the second reference frame tail message 201 are recorded before the switching instruction is received, and the alignment frame tail message is determined immediately after receiving the switching instruction.
  • the sequence number of the text where the determined sequence number of the first aligned frame-tail message is the sequence number 30 of the frame-tail message 102, that is, the first-aligned frame-tail message is the frame-end message 102, and the determined second-aligned frame
  • the sequence number of the trailer message is the sequence number 30 of the trailer message 202, that is, the second aligned trailer message is the trailer message 202.
  • the end-of-frame message 102 and the end-of-frame message 202 are end-of-frame messages corresponding in time.
  • sequence number of the aligned end-of-frame message of any one of the first video stream and the second video stream can be determined through one of the following two implementation manners (implementation manner A1 and implementation manner A2) of implementation manner A. .
  • Embodiment A1 Considering the problem that different video streams may be out of synchronization, in Embodiment A1, the transmission device first determines the synchronization relationship between the first video stream and the second video stream, and then for the first video Any one of the video stream and the second video stream, based on the sequence number of the reference frame tail message of the video stream and the synchronization relationship, calculates the sequence number of the aligned frame tail message of the video stream.
  • the synchronization relationship between the first video stream and the second video stream may specifically be that the first video stream is earlier than the second video stream, the first video stream is later than the second video stream, or The first video stream is synchronized with the second video stream. Out-of-sync between video streams with the need to switch between them usually does not exceed one frame.
  • the sequence number of the first aligned frame-tail message of the first video stream and the sequence of the second aligned frame-tail message of the second video stream are specifically determined as follows: number.
  • the sequence number of the first reference frame tail message is denoted as S11
  • the sequence number of the first aligned frame tail message is denoted as S12
  • the The sequence number is represented as S21
  • the sequence number of the second aligned frame-tail message is represented as S22
  • the number of messages of a single video frame of the first video stream is represented as M1
  • the single of the second video stream is represented as a single
  • M2 The number of packets in a video frame is expressed as M2.
  • M1 and M2 may be the same or different, and are usually the same. In the example of the following manner, it is assumed that M1 and M2 are both 10.
  • the sequence number S12 of the end-of-frame message is aligned
  • the first reference frame tail message is a frame tail message 101 with a sequence number of 20
  • the end-of-frame message is an end-of-frame message 201 with a sequence number of 10
  • the first reference frame tail message is a frame tail message 103 with a sequence number of 10
  • the sequence number S12 of the end-of-frame message is aligned
  • the first reference frame tail message is a frame tail message 105 with a sequence number of 10
  • the second reference The end-of-frame message is an end-of-frame message 205 with a sequence number of 10
  • the first reference frame tail message is a frame tail message 111 with a sequence number of 20
  • the second reference The end-of-frame message is an end-of-frame message 211 with a sequence number of 20
  • the sequence number of the first aligned frame-tail message and the second aligned frame-tail message corresponding to the sequence number of the second aligned frame-tail message (that is, the first aligned frame-tail message and the second aligned frame-tail message).
  • the timestamps of the messages are basically the same, and the timestamps of the switch messages determined based on them are also basically the same. Therefore, it can be ensured that the cut-out flow and the cut-in flow are exactly connected in time dimension during the switchover, and no multi-frame will occur. Or fewer frames.
  • one of the following two implementations may be used according to the first alignment, respectively.
  • the sequence number of the end-of-frame message and the sequence number of the second aligned end-of-frame message determine the first switching sequence number and the second switching sequence number.
  • Embodiment X1 is used to implement switching of a video stream at a frame boundary.
  • a sequence number of a first aligned frame-tail message of the first video stream is used as the first switching sequence number
  • a second aligned frame-tail message of the second video stream is used.
  • the serial number is used as the second handover serial number.
  • the sequence number 30 of the first aligned frame tail message 102 is used as the first switching sequence number, that is, the first aligned frame tail message 102 is used as the switching message of the first video stream
  • the second aligned frame is The sequence number 30 of the tail message 202 is used as the second switching sequence number, that is, the second aligned frame tail message 202 is used as the switching message of the second video stream.
  • Embodiment X2 is used to switch a video stream in a blanking area of a video frame.
  • a complete image scanning signal is composed of a sequence of line signals separated by a horizontal blanking interval, which is called a frame. After scanning a frame at the scanning point, it is necessary to return from the lower right corner of the image to the upper left corner of the image and start a new frame scanning. This time interval is called vertical blanking, also called field blanking.
  • SMPTE RP-168 specifies switching lines or switching areas. Switching two SDI signals in the switching line or switching area will not cause damage to the effective video data. Because SMPTE2022-6 carries SDI signals in IP packets, it is necessary to switch in the blanking area when performing net switching on video streams encapsulated with SMPTE2022-6.
  • first video stream and the second video stream are both SMPTE 2022-6 encoded video streams
  • video stream switching can be performed at the data message in the encapsulated blanking area.
  • the sequence number of the data message used to encapsulate the blanking area data of the next video frame of the video frame corresponding to the sequence number of the first aligned frame tail message may be specifically determined as the The first switching sequence number. For example, determining the sum of the sequence number of the first aligned frame tail message and ⁇ M1 of the first video stream as the first switching sequence number, or Wherein, OL1 is the number of bytes occupied by each row of pixels in a single video frame of the first video stream, and SL1 is a switching line number of the first video stream.
  • Switching line numbers can depend on the SDI video standard. Currently SMPTE RP-168 specifies switching line numbers for most video formats. The switching line number SL1 of the first video stream and the switching line number SL2 of the second video stream are generally the same.
  • the transmission device may first calculate the timestamp of the first aligned frame tail message according to the timestamp of the first reference frame tail message and the duration of a single video frame of the first video stream, and then The time stamp of the first aligned frame tail message determines the time stamp of the first handover message. And the time stamp of the second aligned frame tail message may be calculated according to the time stamp of the second reference frame tail message of the second video stream and the duration of a single video frame of the second video stream, and then based on the second Aligning the time stamp of the end-of-frame message determines the second switch message time stamp.
  • the time stamp of the aligned frame tail message of any one of the first video stream and the second video stream can be determined through one of the following two implementation manners (embodiment B1 and implementation B2) of implementation B. .
  • Embodiment B1 Considering the problem that different video streams may be out of sync, in Embodiment B1, the transmission device first determines the synchronization relationship between the first video stream and the second video stream, and then for the first video Any one of the video stream and the second video stream calculates the time stamp of the aligned frame-tail message of the video stream according to the time stamp of the reference frame-tail message of the video stream and the synchronization relationship.
  • the synchronization relationship between the first video stream and the second video stream may specifically be that the first video stream is earlier than the second video stream, the first video stream is later than the second video stream, or The first video stream is synchronized with the second video stream. Out-of-sync between video streams with the need to switch between them usually does not exceed one frame.
  • the timestamp of the first aligned frame-tail message of the first video stream and the time of the second aligned frame-tail message of the second video stream are specifically determined as follows: stamp.
  • stamp For ease of expression, in the embodiment of the present invention, the timestamp of the first reference frame tail message is represented as t11, the timestamp of the first aligned frame tail message is represented as t12, and the The timestamp is represented as t21, the timestamp of the second aligned frame tail message is represented as t22, the duration of a single video frame of the first video stream is represented as T1, and the time of a single video frame of the second video stream is The duration is expressed as T2.
  • T1 and T2 can be the same or different, and are usually the same.
  • the time stamp t12 of the frame-tailed message is aligned
  • the timestamp of the first aligned frame-tailed message and the timestamp of the second aligned frame-tailed message determined in Embodiment B1 are basically the same, and the timestamps of the handover packets determined based on this are also basically the same. Therefore, It can be ensured that the cut-out flow and the cut-in flow are exactly connected in the time dimension when switching, and the problem of multiple frames or fewer frames does not occur.
  • one of the following two implementations may be used according to the first The timestamp of the aligned end-of-frame message and the timestamp of the second aligned end-of-frame message determine the first switching time stamp and the second switching time stamp.
  • Embodiment Y1 is used to implement switching of a video stream at a frame boundary.
  • a timestamp of a first aligned frame-tailed message of the first video stream is used as the first switching timestamp
  • a second aligned frame-tailed message of the second video stream is used.
  • the timestamp is used as the second switching timestamp.
  • Embodiment Y2 is used to implement switching of a video stream in a blanking area of a video frame.
  • the time stamp of the data message used to encapsulate the blanking area data of the next video frame of the video frame corresponding to the time stamp of the first aligned frame tail message may be specifically determined as the time stamp.
  • the sum of the timestamp of the second aligned frame-tail message of the second video stream and ⁇ t2 is determined as the second switching timestamp
  • ⁇ t1 and ⁇ t2 are calculated according to the same method.
  • Switching line numbers can depend on the SDI video standard. Currently SMPTE RP-168 specifies switching line numbers for most video formats.
  • the switching line number SL1 of the first video stream and the switching line number SL2 of the second video stream are generally the same.
  • L2 of the first video stream and L2 of the second video stream are also generally the same.
  • the transmission device may specifically determine the first time stamp according to a time stamp of a reference frame tail message of the first video stream and a time stamp of a reference frame tail message of the second video stream.
  • the time stamps of data packets sent from different video source devices at the same time may be different.
  • IP networks use packets as a unit, and the jitter of packet forwarding will also introduce some synchronization deviations. Therefore, it can be considered that the two video streams can be considered to be synchronized if the time stamp deviation of the two reference frame tail messages is less than a preset time difference threshold.
  • the time difference threshold used to determine synchronization is determined according to the clock synchronization accuracy of the video source device in the network.
  • the transmission device may specifically determine that the first video stream is earlier than the second video stream when determining t1-t2> ⁇ t1; and determine that the first video stream is later than the first video stream when determining t2-t1> ⁇ t1. Two video streams; when
  • t1 is the timestamp of the first reference frame tail message
  • t2 is the timestamp of the second reference frame tail message
  • ⁇ t1 is a preset first time difference threshold
  • ⁇ t2 is a preset second time difference threshold
  • ⁇ t1 and ⁇ t2 are both values greater than 0 and less than T
  • ⁇ t2 may be the same as ⁇ t1 or smaller than ⁇ t1.
  • ⁇ t1 and ⁇ t2 can be preset according to the following conditions: T / 2 ⁇ t1 ⁇ T, such as ⁇ t1 can be 3 * T / 4, 0 ⁇ t2 ⁇ T / 2, such as ⁇ t2 can be T / 4, and T is a single video frame. duration.
  • T may be preset or calculated according to preset video information (such as a frame rate).
  • T may also be a data message according to the received video stream of any of the first video stream and the second video stream. Calculated, for example, calculated based on a difference between timestamps of two adjacent frame-tail messages of the video stream.
  • the transmission device switches the first video stream sent through the target outbound port to the first switching stream according to the switching reference information of the first switching message and the switching reference information of the second switching message. Two video streams.
  • both the handover reference information of the first handover message and the handover reference information of the second handover message are serial numbers, and the transmission device may specifically not send or receive information through the target outbound port.
  • the transmission device may determine whether to send the data packet through the target outbound port according to the serial number of the received data packet of the first video stream. If the data packet is less than or equal to the first switching sequence number, it passes as usual. The target egress port sends the data packet, otherwise the data packet is not sent through the target egress port.
  • the transmission device may determine whether to send the data packet through the target outbound port according to the sequence number of the data packet of the received second video stream. If it is less than or equal to the second switching sequence number, it will not pass as usual.
  • the target egress port sends the data packet, otherwise the data packet is sent through the target egress port.
  • both the handover reference information of the first handover message and the handover reference information of the second handover message are timestamps, and the transmission device may specifically not send or receive through the target outbound port.
  • the transmission device may also send and receive a data packet with a timestamp of the first video stream that is less than or equal to the first switching timestamp through the target output port, and does not send and receive through the target output port.
  • a data packet whose time stamp of the second video stream is less than or equal to the second switching time stamp
  • the handover reference information of the first handover message and the handover reference information of the second handover message both include a sequence number and a time stamp.
  • the absolute value Dif1 of the difference between the time stamp of the data packet and the time stamp of the first switching packet is greater than the threshold value 1 and the data packet Timestamp is greater than the timestamp of the first switch message, or Dif1 is less than or equal to threshold 1 and the sequence number of the data message is greater than the sequence number of the first switch message, and does not pass through the destination egress port Forwarding the data message.
  • the absolute value of the difference between the timestamp of the data message and the timestamp of the second handover message is Dif2 greater than the threshold value 2 and the data message Timestamp greater than the timestamp of the second handover message, or Dif2 is less than or equal to threshold 2 and the sequence number of the data message is greater than the sequence number of the second handover message, and is forwarded through the destination egress The data message.
  • the received data packet of the first video stream if Dif1 is greater than the threshold 1 and the timestamp of the data packet is less than the timestamp of the first switching packet, or Dif1 is less than or equal to the threshold 1 and The serial number of the data packet is less than or equal to the serial number of the first handover packet, and the data packet is forwarded through the target outbound port.
  • the received data packet of the second video stream if Dif2 is greater than threshold 2 and the timestamp of the data packet is less than the timestamp of the second handover packet, or Dif2 is less than or equal to threshold 2 and the The sequence number of the data message is smaller than the sequence number of the second handover message, and the data message is not forwarded through the target outbound port.
  • threshold 1 and threshold 2 can be the same or different, and both can be set to a value greater than or equal to 1 / 4T, and specifically can be set to a value greater than or equal to 1 / 4T and less than or equal to 7 / 4T, such as 3 / 4T , Where T is the duration of a single video frame of the first video stream or the second video stream.
  • the sequence number or timestamp of the received data stream is less than or equal to the switching sequence number of the data stream. Or the possibility of switching the time-stamped data message is very small, so it is no longer necessary to check the sequence number or timestamp of the data message of the received data stream one by one to determine whether to send the data message through the target outbound port.
  • M is the number of packets of a single video frame of the video stream
  • T is the duration of a single video frame of the video stream.
  • the transmission device may include multiple outbound ports. If steps S110 and S150 of the method 100 are performed by a unit that is only responsible for processing a data packet corresponding to the target outbound port, in steps S110 and S150, The unit can discard data packets that do not need to be forwarded through the target outbound port.
  • the transmission device when the transmission device forwards the data packet of the second video stream, it may also modify the data packet in the second video stream. information.
  • the transmission device may modify the value of the first field in the data message of the second video stream so that the second video sent by the target out port after the switch
  • the value of the first field in the data packet of the stream is the same as the value of the first field in the data packet of the first video stream sent through the target outbound port before switching.
  • the first field may be a multicast address field or a payload type field. If RTP is used for video stream transmission, the first field may also be a Synchronization Source (SSRC) field.
  • SSRC Synchronization Source
  • the first field may also be an HBRMT layer parameter defined by SMPTE 2022-6, such as FRCount, where FRCount is the frame count (Frame count) in the high bit rate media payload header defined by SMPTE 2022-6. ) Field, which indicates the number of video frames accumulated in the video stream.
  • FRCount is the frame count (Frame count) in the high bit rate media payload header defined by SMPTE 2022-6.
  • Field which indicates the number of video frames accumulated in the video stream.
  • the transmission device may modify the value of the first field in the data packet of the video stream to a preset value when sending any video stream of the multiple video streams through the target egress port, such as, Modify the multicast address in the data message of the video stream to a preset multicast address (such as 239.0.2.1).
  • the transmission device may receive multiple video streams corresponding to the target egress port, send one video stream of the plurality of video streams through the target egress port, and pass through the plurality of video streams.
  • the video stream sent by the target egress port is switched.
  • the multiple video streams include stream 1, stream 2, and stream 3.
  • the transmission device first forwards stream 2 and then switches to stream 1 and stream 3 in sequence.
  • the transmission device may also record the value of the first field of the video stream when the video stream (such as stream 2) is forwarded through the target outbound port for the first time, and then switch to other videos later When streaming, modify the value of the first field in the data packets of other video streams to the value of the first field of the recorded video stream.
  • the value of the first field in the data packet of stream 1 is modified to the value of the first field in stream 2 and when switching to stream 3, the first field in the data message of stream 3 is changed.
  • the value of one field is also modified to the value of the first field of stream 2.
  • the transmission device may also modify the second field in the data message of the second video stream.
  • the value is such that the value of the second field in the data message of the second video stream sent through the target out port after switching and the second field in the data message of the first video stream sent through the target out port before switching
  • the values are continuous.
  • the value of the second field may be one or more of a sequence number, a frame number, a time stamp, and the like.
  • the transmission device may also perform step S115 to record the target information of the latest frame-tail messages of the first video stream and the second video stream.
  • the recorded target information includes the sequence number and time stamp of the latest frame-tail message. At least one of.
  • the reference frame tail message may be determined according to the target information of the latest frame tail message recorded in step S115. For example, for any video stream in the first video stream and the second video stream, The target information (such as a sequence number) of the latest frame trailer message of the video stream recorded in step S115 for the last time before step S130 is performed is used as the reference information (such as the sequence number) of the reference frame trailer message of the video stream.
  • the transmission device may start performing step S115 before step S120.
  • the transmission device may receive multiple video streams corresponding to the target egress port, send one video stream of the plurality of video streams through the target egress port, and pass through the plurality of video streams.
  • the video stream sent by the target egress port is switched.
  • the transmission device starts to forward the video streams in the multiple video streams through the target output port, the target information of the latest frame tail message of each video stream in the multiple video streams is recorded.
  • steps S130-S140 may be performed using target information of the latest frame trailer messages of the two video streams that have been recorded to determine the switching reference information.
  • step S115 may be executed before step S120.
  • the transmission device may also execute step S115 after step S120. That is, after receiving the switching instruction, the target information of the latest frame-tail message of each of the two video streams to be switched begins to be recorded, and then according to the latest frame-tail messages of the two video streams that are recorded Steps S130-S140 are performed on the target information to determine the switching reference information of the switching message.
  • step S115 may be started after step S120.
  • step S115 is started before step S120, for any video stream in the multiple video streams, the transmission device may receive the first frame tail message in sequence in the video stream, and send the frame.
  • the target information of the tail message is recorded as the target information of the latest frame tail message of the video stream, and the target information of the latest frame tail message of the video stream is updated according to the data message of the video stream received subsequently.
  • steps S130-S140 are executed immediately after receiving the switching instruction to calculate switching reference information (such as a sequence number) of the switching message based on the target information (such as a sequence number) of the latest frame-end message recorded before receiving the switching instruction. As shown in FIG.
  • the first switching sequence number is calculated according to the sequence number 20 of the latest frame trailer message 101 of the first video stream recorded before receiving the switching instruction, and according to the sequence of the latest frame tail message 201 of the second video stream Number 10 calculates the second handover sequence number.
  • step S115 is executed after step S120, for any video stream in the first video stream and the second video stream, the transmission device may receive the first frame in the video stream in sequence
  • the target information of the frame tail message is recorded as the target information of the latest frame tail message of the video stream. If you have not received the first end-of-frame message of the other video stream when receiving the end-of-frame message of another video stream, you can update the target information of the latest end-of-frame message of the video stream Until the first end-of-frame message of another video stream is received and the target information of the latest end-of-frame message of the other video stream is recorded accordingly.
  • the number of messages of a single video frame is calculated according to the sequence numbers of adjacent frame tail messages for the first video stream and the second video stream, at least two are received for each video stream before the switching sequence number is calculated.
  • Each frame tail message each time the frame tail message is received, the target information of the frame tail message is recorded as the target information of the latest frame tail message of the video stream to which it belongs.
  • the difference between the sequence numbers of the two adjacent frame-tail messages can be used as the number of messages of a single video frame of the video stream to which it belongs, and the timestamps of the two adjacent frame-tail messages can also be used. The difference is the duration of a single video frame of the video stream.
  • the first switching sequence number is calculated according to the sequence number 30 of the latest frame tail message 107 of the first video stream recorded after receiving the switching instruction, and the sequence of the latest frame tail message 207 of the second video stream is calculated.
  • No. 20 calculates the second switching sequence number.
  • the transmission device may specifically update the target information of the latest frame trailer message of any video stream in the following manner.
  • the transmission device may update the sequence number of the latest frame trailer message of the video stream to the sequence number of the frame trailer message each time the frame trailer message of the video stream is received, and update the The timestamp of the latest trailer message is updated to that of the trailer message.
  • the transmission device may also consider the problems of out-of-frame message and end-of-frame message loss that may occur during the update. If there is an inter-frame message out of order, that is, the non-tail-end message of the next frame arrives before the end-of-frame message of the previous frame or the end-of-frame message of the previous frame is processed by the transmission device, or there is a end-of-frame message The message is lost, and the sequence number of the latest frame-tail message of the video stream may also be the sequence number of the frame-tail message estimated based on the non-frame-tail message whose sequence number is greater than that of the un-received frame-tail message. Accordingly, the transmission device can be updated in the following manner.
  • the received data message of the video stream is not a frame-tail message and the sequence number of the data message is greater than the sequence number of the latest frame-tail message of the video stream and the number of messages of a single video frame of the video stream.
  • the sequence number of the latest frame-end message of the recorded video stream is increased by M.
  • the target information further includes a time stamp, correspondingly, the time stamp of the latest frame tail message of the video stream recorded is increased by the duration T of a single video frame.
  • an update operation may be directly performed, that is, the sequence number of the latest frame trailer message of the video stream is updated to the sequence number of the data message, and the The time stamp of the latest frame trailer message of the video stream is updated to the time stamp of the data message. It can also be further determined whether the time stamp of the data message is greater than the time stamp of the latest frame tail message of the video stream recorded, and The update operation is performed only when it is greater than.
  • step S115 The specific implementation of step S115 will be described below in a specific implementation manner.
  • the transmitting device establishes a video stream switching table. If step S115 is performed before step S120, the switching table may include a record of each video stream in the plurality of video streams. If step S115 is performed after step S120, the switching table may include only two video streams to be switched (Ie, the first video stream and the second video stream), as shown in Table 1.
  • the switching table may include fields FT_TS and FT_SN, and the value of FT_TS and the value of FT_SN corresponding to each video stream are the timestamp and sequence number of the latest frame trailer message of the video stream, respectively.
  • the video stream switching table may further include any one or any combination of the fields Flow, Outport, State, FrameTime, and FramePackets.
  • the value of Flow is used to identify the corresponding video stream.
  • the value of Outport corresponding to each video stream is used to record the port number of the corresponding output port of the video stream, that is, which port of the transmission device the video stream is forwarded from.
  • the embodiment of the present invention Stream to switch.
  • the State value corresponding to each video stream is used to record the current state of the video stream. Among them, SW_Forwarding indicates that the video stream is a video stream being forwarded by the transmission device, and SW_Discard indicates that the video stream is that the transmission device does not pass the corresponding outbound port. Forwarded video stream.
  • the value of FrameTime corresponding to each video stream is used to record the duration of a single video frame of the video stream. If RTP is used for video stream transmission, the unit of FrameTime is the same as the unit of the TS field in the RTP header.
  • the value of FramePackets corresponding to each video stream is used to record the number of packets of a single video frame of the video stream.
  • the video stream switching table may further include SW_SN, and a value of SW_SN corresponding to each video stream is used to record a switching sequence number of the video stream.
  • video stream A Any video stream in the video stream switching table is hereinafter referred to as video stream A.
  • message a For video stream A, when any data message (hereinafter referred to as message a) of video stream A is received, it is first determined whether the message a is an end-of-frame message.
  • the value of the M (Marker, boundary) field in the RTP message header is used to indicate whether the RTP message is a frame-tail message. If the value of M is 1, it means that the RTP message is an end-of-frame message. If it is 0, it means that the RTP message is not an end-of-frame message.
  • the value of the Sequence Number (SN) field in the RTP message header indicates the sequence number of the RTP message in an RTP stream.
  • the sequence number of each RTP message in the same RTP stream monotonically increases by 1.
  • the RTP packet header also includes a timestamp (Timestamp, TS).
  • TS Timestamp
  • Each video source device in a multimedia communication system (such as a TV production and broadcast system) will keep the clock synchronized.
  • the value of this field is very close for RTP packets sent at the same time. Therefore, the value of this field can be used to judge the difference in different video streams. Whether the video frames at the same time are used to align different video streams when switching, to achieve the connection in the time dimension of the switch.
  • the values of the TS field and the SN field of the message a are recorded in the first switch table
  • the FT_TS field and FT_SN field corresponding to the middle video stream A.
  • the value of the TS field of the message a and the FT_TS corresponding to the video stream A in the first switching table The values of the fields are compared. If the value of the TS field of the message a is greater than the value of the FT_TS field corresponding to the video stream A in the first switching table, the message a is the latest frame end message. Therefore, the FT_TS corresponding to the video stream A in the first switching table is The values of the field and the FT_SN field are updated to the values of the TS field and the SN field of the message a, respectively. If it is less, it is not updated.
  • a non-frame trailer message of a frame may reach the transmission device before the frame trailer message of the previous frame or a frame trailer message of the previous frame is processed. If the message of one frame arrives before the end of frame of the previous frame or is processed before the end of frame of the previous frame, it indicates that the inter-frame message is out of order.
  • message a is not a trailer message, and the sequence number of message a is greater than the sum of the sequence number of the latest trailer message of video stream A and M, that is, message a arrives before the trailer message of the previous frame. , Then the frame tail message of the previous frame may be out of order or packet loss.
  • the sum of the value of the FT_TS field and T can be used as the estimated time stamp of the frame tail message, and the value of the FT_SN field
  • the sum of M and M is used as the estimated sequence number of the end-of-frame message, and the estimated time stamp and sequence number are used to update the values of the FT_TS field and FT_SN field corresponding to the first switching table video stream A, that is, the value of the FT_TS field is increased by T And increase the value of the FT_SN field by M.
  • Embodiment 2 of the present invention proposes a transmission device 200.
  • the transmission device 200 includes a processing unit 210 and a communication unit 220.
  • the communication unit 220 is configured to receive a first video stream and a second video stream, send the first video stream through a target output port of the transmission device, and receive a switching instruction, where the switching instruction is used to indicate that The first video stream sent by the target egress port is switched to the second video stream.
  • a processing unit 210 configured to determine switching reference information of a first reference frame tail message of the first video stream and switching reference information of a second reference frame tail message of the second video stream according to the switching instruction, Determining the switching reference information of the first switching message of the first video stream according to the switching reference information of the first reference frame tail message, and determining the switching information according to the switching reference information of the second reference frame tail message Reference information for switching the second switching message of the second video stream.
  • the communication unit 220 is further configured to switch the first video stream sent through the target out port to the first video stream according to the switching reference information of the first switching message and the switching reference information of the second switching message.
  • the second video stream is further configured to switch the first video stream sent through the target out port to the first video stream according to the switching reference information of the first switching message and the switching reference information of the second switching message.
  • the functional unit described in Embodiment 2 of the present invention may be used to implement the operations performed by the transmission device in the method described in Embodiment 1 above.
  • the communication unit 220 is configured to communicate with other devices, such as receiving control signaling, receiving a data message, forwarding a data message, and modifying information in a data message when the data message is being forwarded; and the processing unit 210 is used to determine a switching sequence number of a cut-in flow and a cut-out flow.
  • the processing unit 210 may include multiple processing sub-units, each corresponding to an output port (such as the foregoing target output port) that needs to switch video streams.
  • the communication unit 220 may include multiple sending subunits, each corresponding to an egress port that needs to switch video streams.
  • the processing subunit of each outbound port determines the switching reference information (such as the sequence number) of the inbound and outbound flow switching packets of the outbound port.
  • the sending subunit corresponding to each outbound port needs to pass through the outbound port for forwarding. Data packets of the video stream forwarded by the outbound port, and directly discard data packets of the video stream that do not need to be forwarded through the outbound port.
  • the transmission device 200 provided in Embodiment 2 of the present invention since the position of the data message sent before and after the switching is taken into consideration when the video stream is switched, it is possible to effectively reduce the damage to the transmitted video image during the switching operation.
  • Embodiment 3 of the present invention provides a transmission device 1000.
  • the transmission device 1000 includes a processor 1010 and a port 1020.
  • the processor 1010 is configured to implement an operation performed by the transmission device in the method 100 provided in Embodiment 1 of the present invention, and when communicating with other devices, communicate with the other devices through the port 1020.
  • the processor 1010 may specifically be a central processing unit (CPU), a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention.
  • the transmission device 1000 may further include a memory 1030.
  • the processor 1010 and the memory 1030 can communicate with each other through a bus.
  • the memory 1030 is configured to store computer operation instructions.
  • the memory 1030 may be a high-speed random access memory (RAM) or a non-volatile memory (non-volatile memory).
  • the processor 1010 may specifically execute computer operation instructions stored in the memory 1030, and cause the transmission device 1000 to perform operations performed by the transmission device in the method described in Embodiment 1 by executing the computer operation instructions.
  • the memory 1030 may include a processing module 1031 and a communication module 1032, which respectively store functions for implementing the processing unit 210 and the communication unit 220 shown in FIG. 5. Accordingly, the processor 1010 specifically implements the functions of the processing unit 210 by executing instructions in the processing module 1031 and implements the functions of the communication unit 220 by executing instructions in the communication module 1032.
  • the transmission device 1000 provided in Embodiment 3 of the present invention, when a video stream is switched, a position of a data message sent before and after switching is taken into consideration. Therefore, it is possible to effectively reduce the damage to the transmitted video image during the switching operation.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer.
  • the available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (Solid State Disk (SSD)), and the like.
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium for example, a DVD
  • a semiconductor medium for example, a solid state disk (Solid State Disk (SSD)

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本发明实施例公开了一种实现视频流切换的方法、装置和系统。所述方法包括：传输设备接收第一视频流和第二视频流，通过目标出端口发送第一视频流，在接收到切换指令时确定第一视频流的第一基准帧尾报文的切换参考信息和第二视频流的第二基准帧尾报文的切换参考信息，分别根据第一基准帧尾报文和第二基准帧尾报文的切换参考信息确定第一视频流的第一切换报文的切换参考信息和第二视频流的第二切换报文的切换参考信息，进而根据第一切换报文和第二切换报文的切换参考信息将通过目标出端口发送的第一视频流切换为第二视频流。由于在视频流切换时考虑了切换前后发送的数据报文所处的位置，因此，可以有效减轻切换操作对传输的视频画面的损伤。

Description

一种实现视频流切换的方法、装置和系统 技术领域
本发明实施例涉及多媒体通信技术领域，尤其涉及一种实现视频流切换的方法、装置和系统。
背景技术
在多媒体通信技术领域，如电视(Television,TV)制播领域，存在对不同视频源设备发送的视频流进行切换的场景。如，在对足球比赛进行直播时，可以在传输设备(如路由器)上对接收的通过对拍摄球场的不同位置所得到的视频流进行切换，以实现在视频终端上对球场不同角度的视频画面进行切换显示。
在传统的TV制播领域，通常是以串行数字接口(serial digital interface，SDI)格式传输视频流。由于用于传输SDI信号的同轴电缆最大长度通常受到限制，且比较昂贵，因此在TV制播领域，已经开始通过因特网协议(Internet Protocol，IP)网络传输视频流，如将SDI信号承载在IP报文中传输。
在实现对通过IP网络传输的视频流进行切换的现有技术中，传输设备在确定需要切换视频流时即停止发送要切出的视频流的数据报文并开始发送要切入的视频流的数据报文。
根据该现有技术实现视频流的切换，往往导致视频画面出现损伤。
发明内容
本发明实施例提供一种实现视频流切换的方法、装置和系统，用于解决现有技术中存在的视频流切换导致的视频画面出现损伤的问题。
第一方面，提供了一种实现视频流切换的方法，该方法包括：传输设备接收第一视频流和第二视频流，通过目标出端口发送该第一视频流，在接收到用于指示将通过该目标出端口发送的该第一视频流切换为该第二视频流的切换指令后，根据该切换指令确定该第一视频流的第一基准帧尾报文的切换参考信息(如序列号)和该第二视频流的第二基准帧尾报文的切换参考信息，根据该第一基准帧尾报文的切换参考信息确定该第一视频流的第一切换报文的切换参考信息，并根据该第二基准帧尾报文的切换参考信息确定该第二视频流的第二切换报文的切换参考信息，进而根据该第一切换报文的切换参考信息和该第二切换报文的切换参考信息将通过该目标出端口发送的该第一视频流切换为该第二视频流。该第一基准帧尾报文、第二基准帧尾报文、第一切换报文和第二切换报文的切换参考信息可以均为序列号或均为时间戳，也可以均包括序列号和时间戳。
根据第一方面的实现方式，根据切出流(即第一视频流)和切入流(即第二视频流)的帧尾报文的切换参考信息确定切出流和切入流的切换报文的切换参考信息(如序列号)，并根据切换报文的切换参考信息进行视频流的切换。由于在视频流切换时考虑了切换前后发送的数据报文所处的位置，因此，可以有效减轻切换操作对传输的视频画面的损伤。
在第一方面的第一种实现方式中，该第一基准帧尾报文和该第二基准帧尾报文具体可以分别为确定该第一基准帧尾报文的切换参考信息和该第二基准帧尾报文的切换参考信息之前接收的该第一视频流的倒数第x个帧尾报文和该第二视频流的倒数第x个帧尾报文，其中x为自然数，可以为1。
根据该实现方式确定的第一基准帧尾报文和第二基准帧尾报文的时间戳比较接近，因此可以更加准确地确定出时间戳比较一致的第一切换报文和第二切换报文，从而进一步减轻切换操作对传输的视频画面的损伤。
基于第一方面的第一种实现方式，在第一方面的第二种实现方式中，x取值为1，所述第一切换报文中的视频数据所属视频帧为所述第一基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧；所述第二切换报文中的视频数据所属视频帧为所述第二基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧。具体可以根据所述第一基准帧尾报文的序列号和封装所述第一视频流的单个视频帧的报文个数计算所述第一切换报文的序列号、并根据所述第二基准帧尾报文的序列号和封装所述第二视频流的单个视频帧的报文个数计算所述第二切换报文的序列号,确定的所述第一切换报文的序列号与所述第一基准帧尾报文的序列号的差值可以大于或等于封装所述第一视频流的单个视频帧的报文个数，确定的所述第二切换报文的序列号与所述第二基准帧尾报文的序列号的差值可以大于或等于封装所述第二视频流的单个视频帧的报文个数。具体还可以根据所述第一基准帧尾报文的时间戳和所述第一视频流的单个视频帧的时长计算所述第一切换报文的时间戳、并根据所述第二基准帧尾报文的时间戳和所述第二视频流的单个视频帧的时长计算所述第二切换报文的时间戳,确定的所述第一切换报文的时间戳与所述第一基准帧尾报文的时间戳的差值可以大于或等于所述第一视频流的单个视频帧的时长，确定的所述第二切换报文的时间戳与所述第二基准帧尾报文的时间戳的差值可以大于或等于所述第二视频流的单个视频帧的时长。
根据该实现方式实现在基准帧尾报文后至少一帧的位置处进行视频流切换，可以有效避免对当前正在传输的视频画面的损伤。
基于第一方面或第一方面的第一种或第二种实现方式，在第一方面的第三种实现方式中，可以根据该第一基准帧尾报文的切换参考信息、以及该第一视频流和该第二视频流之间的同步关系确定该第一切换报文的切换参考信息，并根据该第二基准帧尾报文的切换参考信息、以及该同步关系确定该第二切换报文的切换参考信息。
根据该实现方式，由于在计算第一切换报文和第二切换报文的切换参考信息时考虑了视频流之间的同步关系，可以更加精准地实现视频流的切换，从而进一步减轻切换操作对传输的视频画面的损伤。
基于第一方面的第三种实现方式，在第一方面的第四种实现方式中，可以根据该第一基准帧尾报文的切换参考信息、以及该同步关系确定该第一视频流的第一对齐帧尾报文的切换参考信息，并根据该第一对齐帧尾报文的切换参考信息确定该第一切换报文的切换参考信息，还可以根据该第二基准帧尾报文的切换参考信息、以及该同步关系确定该第二视频流的第二对齐帧尾报文的切换参考信息，并根据该第二对齐帧尾报文的切换参考信息确定该第二切换报文的切换参考信息，该第一对齐帧尾报文和该第二对齐帧尾报文为在时间上对应的帧尾报文。
根据该实现方式，在考虑同步关系的情况下确定的第一对齐帧尾报文和第二对齐帧尾报文为在时间上对应的帧尾报文，因此，根据第一对齐帧尾报文和第二对齐帧尾报文分别确定出的第一切换报文和第二切换报文在时间上也是对应的，因此可以进一步减轻切换操作对传输的视频画面的损伤。
基于第一方面的第四种实现方式，在第一方面的第五种实现方式中，可以直接将第一对齐帧尾报文的切换参考信息作为第一切换报文的切换参考信息并将第二对齐帧尾报文的切换 参考信息作为第二切换报文的切换参考信息，即直接将第一对齐帧尾报文作为第一切换报文并将第二对齐帧尾报文作为第二切换报文。
根据该实现方式，可以实现在视频帧的边界进行视频流的切换，可以有效避免切换操作对传输的视频画面的损伤。
基于第一方面的第四种实现方式，在第一方面的第六种实现方式中，可以将用于封装该第一对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为该第一切换报文的切换参考信息，并将用于封装该第二对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为该第二切换报文的切换参考信息。
根据该实现方式，可以实现在封装视频帧的消隐区的数据报文处进行视频流的切换，由于消隐区数据并非有效的视频数据，对人眼是不可见的，因此，可以避免对有效的视频数据产生损伤。
以切换参考信息包括序列号为例，在具体实现时，在确定该同步关系为该第一视频流早于该第二视频流时，可以将该第一基准帧尾报文的序列号与m*M1的和确定为该第一对齐帧尾报文的序列号，并将该第二基准帧尾报文的序列号与(m+1)*M2的和确定为该第二对齐帧尾报文的序列号。在确定该同步关系为该第一视频流晚于该第二视频流时，将该第一基准帧尾报文的序列号与(n+1)*M1的和确定为该第一对齐帧尾报文的序列号，将该第二基准帧尾报文的序列号与n*M2的和确定为该第二对齐帧尾报文的序列号。或，在确定该同步关系为该第一视频流和该第二视频流同步时，可以将该第一基准帧尾报文的序列号与k*M1的和确定为该第一对齐帧尾报文的序列号，将该第二基准帧尾报文的序列号与k*M2的和确定为该第二对齐帧尾报文的序列号。其中，M1为封装该第一视频流的一个视频帧所需的数据报文的数量，M2为封装该第二视频流的一个视频帧所需的数据报文的数量，m、n和k均为自然数。
基于第一方面的第三种至第六种中的任意一种实现方式，在第一方面的第七种实现方式中，该传输设备具体可以根据该第一基准帧尾报文的时间戳和该第二基准帧尾报文的时间戳确定该第一视频流和该第二视频流的同步关系。
根据该实现方式，由于是根据该第一基准帧尾报文和第二基准帧尾报文确定该同步关系，因此该同步关系可以反映出该传输设备接收到该第一基准帧尾报文和第二基准帧尾报文时的同步状况，所以，根据第一基准帧尾报文的切换参考信息、第二基准帧尾报文的切换参考信息、以及该同步关系的组合确定的第一切换报文的切换参考信息和第二切换报文的切换参考信息更加符合视频流的真实同步状况，从而可以进一步减轻切换操作对传输的视频画面的损伤。
考虑到设备同步本身就有精度偏差，同一个时刻从不同视频源设备发出的数据报文，其时间戳可能会有偏差。另外，IP网络以报文为单位，报文转发的抖动也会引入一些同步偏差。因此，可以认为，如果两个视频流的基准帧尾报文的时间戳偏差小于预设的时差阈值即可认为这两个视频流同步。
相应地，传输设备具体可以在确定t1-t2>Δt1时，确定该第一视频流早于该第二视频流；在确定t2-t1>Δt1时，确定该第一视频流晚于该第二视频流；在确定|t1-t2|<Δt2时，确定该第一视频流与该第二视频流同步。t1为该第一基准帧尾报文的时间戳；t2为该第二基准帧尾报文的时间戳；Δt1为预设的第一时差阈值，Δt2为预设的第二时差阈值，Δt1和Δt2均为大于0且小于T的值，Δt2可以和Δt1相同也可以小于Δt1。可以按照如下条件预设Δt1和Δt2：T/2<Δt1<T，如Δt1可以为3*T/4，0<Δt2<T/2，如Δt2可以为T/4，T 为单个视频帧的时长。
根据该实现方式，确定出的同步关系排除了设备同步本身的精度偏差、报文转发的抖动等的影响，更能反映出视频流之间的真实同步状况，确定的第一切换报文的切换参考信息和第二切换报文的切换参考信息也更加符合视频流的真实同步状况，从而可以进一步减轻切换操作对传输的视频画面的损伤。
第二方面，提供了一种传输设备，该传输设备包括处理单元和通信单元。
该通信单元，用于接收第一视频流和第二视频流，通过该传输设备的目标出端口发送该第一视频流，并接收切换指令，该切换指令用于指示将通过该目标出端口发送的该第一视频流切换为该第二视频流。该处理单元，用于根据该切换指令确定该第一视频流的第一基准帧尾报文的切换参考信息(如序列号)和该第二视频流的第二基准帧尾报文的切换参考信息，根据该第一基准帧尾报文的切换参考信息确定该第一视频流的第一切换报文的切换参考信息，并根据该第二基准帧尾报文的切换参考信息确定该第二视频流的第二切换报文的切换参考信息。该通信单元，还用于根据该第一切换报文的切换参考信息和该第二切换报文的切换参考信息将通过该目标出端口发送的该第一视频流切换为该第二视频流。该第一基准帧尾报文、第二基准帧尾报文、第一切换报文和第二切换报文的切换参考信息可以均为序列号或均为时间戳，也可以均包括序列号和时间戳。
根据第二方面的实现方式，根据切出流(即第一视频流)和切入流(即第二视频流)的帧尾报文的切换参考信息确定切出流和切入流的切换报文的切换参考信息(如序列号)，并根据切换报文的切换参考信息进行视频流的切换。由于在视频流切换时考虑了切换前后发送的数据报文所处的位置，因此，可以有效减轻切换操作对传输的视频画面的损伤。
在第二方面的第一种实现方式中，该第一基准帧尾报文和该第二基准帧尾报文具体可以分别为确定该第一基准帧尾报文的切换参考信息和该第二基准帧尾报文的切换参考信息之前接收的该第一视频流的倒数第x个帧尾报文和该第二视频流的倒数第x个帧尾报文，其中x为自然数，可以为1。
根据该实现方式确定的第一基准帧尾报文和第二基准帧尾报文的时间戳比较接近，因此可以更加准确地确定出时间戳比较一致的第一切换报文和第二切换报文，从而进一步减轻切换操作对传输的视频画面的损伤。
基于第二方面的第一种实现方式，在第二方面的第二种实现方式中，x取值为1，所述第一切换报文中的视频数据所属视频帧为所述第一基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧；所述第二切换报文中的视频数据所属视频帧为所述第二基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧。
根据该实现方式实现在基准帧尾报文后至少一帧的位置处进行视频流切换，可以有效避免对当前正在传输的视频画面的损伤。
基于第二方面或第二方面的第一种或第二种实现方式，在第二方面的第三种实现方式中，该处理单元具体可以根据该第一基准帧尾报文的切换参考信息、以及该第一视频流和该第二视频流之间的同步关系确定该第一切换报文的切换参考信息，并根据该第二基准帧尾报文的切换参考信息、以及该同步关系确定该第二切换报文的切换参考信息。
根据该实现方式，由于在计算第一切换报文和第二切换报文的切换参考信息时考虑了视频流之间的同步关系，可以更加精准地实现视频流的切换，从而进一步减轻切换操作对传输的视频画面的损伤。
基于第二方面的第三种实现方式，在第二方面的第四种实现方式中，该处理单元可以根据该第一基准帧尾报文的切换参考信息、以及该同步关系确定该第一视频流的第一对齐帧尾报文的切换参考信息，并根据该第一对齐帧尾报文的切换参考信息确定该第一切换报文的切换参考信息，还可以根据该第二基准帧尾报文的切换参考信息、以及该同步关系确定该第二视频流的第二对齐帧尾报文的切换参考信息，并根据该第二对齐帧尾报文的切换参考信息确定该第二切换报文的切换参考信息，该第一对齐帧尾报文和该第二对齐帧尾报文为在时间上对应的帧尾报文。
根据该实现方式，在考虑同步关系的情况下确定的第一对齐帧尾报文和第二对齐帧尾报文为在时间上对应的帧尾报文，因此，根据第一对齐帧尾报文和第二对齐帧尾报文分别确定出的第一切换报文和第二切换报文在时间上也是对应的，因此可以进一步减轻切换操作对传输的视频画面的损伤。
基于第二方面的第四种实现方式，在第二方面的第五种实现方式中，该处理单元可以直接将第一对齐帧尾报文的切换参考信息作为第一切换报文的切换参考信息并将第二对齐帧尾报文的切换参考信息作为第二切换报文的切换参考信息，即直接将第一对齐帧尾报文作为第一切换报文并将第二对齐帧尾报文作为第二切换报文。
根据该实现方式，可以实现在视频帧的边界进行视频流的切换，可以有效避免切换操作对传输的视频画面的损伤。
基于第二方面的第四种实现方式，在第二方面的第六种实现方式中，该处理单元可以将用于封装该第一对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为该第一切换报文的切换参考信息，并将用于封装该第二对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为该第二切换报文的切换参考信息。
根据该实现方式，可以实现在封装视频帧的消隐区的数据报文处进行视频流的切换，由于消隐区数据并非有效的视频数据，对人眼是不可见的，因此，可以避免对有效的视频数据产生损伤。
基于第二方面的第三种至第六种中的任意一种实现方式，在第二方面的第七种实现方式中，该传输设备具体可以根据该第一基准帧尾报文的时间戳和该第二基准帧尾报文的时间戳确定该第一视频流和该第二视频流的同步关系。
根据该实现方式，由于是根据该第一基准帧尾报文和第二基准帧尾报文确定该同步关系，因此该同步关系可以反映出该传输设备接收到该第一基准帧尾报文和第二基准帧尾报文时的同步状况，所以，根据第一基准帧尾报文的切换参考信息、第二基准帧尾报文的切换参考信息、以及该同步关系的组合确定的第一切换报文的切换参考信息和第二切换报文的切换参考信息更加符合视频流的真实同步状况，从而可以进一步减轻切换操作对传输的视频画面的损伤。
考虑到设备同步本身就有精度偏差，同一个时刻从不同视频源设备发出的数据报文，其时间戳可能会有偏差。另外，IP网络以报文为单位，报文转发的抖动也会引入一些同步偏差。因此，可以认为，如果两个视频流的基准帧尾报文的时间戳偏差小于预设的时差阈值即可认为这两个视频流同步。
相应地，该处理单元具体可以在确定t1-t2>Δt1时，确定该第一视频流早于该第二视频流；在确定t2-t1>Δt1时，确定该第一视频流晚于该第二视频流；在确定|t1-t2|<Δt2时，确定该第一视频流与该第二视频流同步。t1为该第一基准帧尾报文的时间戳；t2为该第二基 准帧尾报文的时间戳；Δt1为预设的第一时差阈值，Δt2为预设的第二时差阈值，Δt1和Δt2均为大于0且小于T的值，Δt2可以和Δt1相同也可以小于Δt1。可以按照如下条件预设Δt1和Δt2：T/2<Δt1<T，如Δt1可以为3*T/4，0<Δt2<T/2，如Δt2可以为T/4，T为单个视频帧的时长。
根据该实现方式，确定出的同步关系排除了设备同步本身的精度偏差、报文转发的抖动等的影响，更能反映出视频流之间的真实同步状况，确定的第一切换报文的切换参考信息和第二切换报文的切换参考信息也更加符合视频流的真实同步状况，从而可以进一步减轻切换操作对传输的视频画面的损伤。
第三方面，提供了一种实现视频流切换的系统，该系统包括：控制器和第二方面或第二方面的任意一种实现方式所述的传输设备，其中，该控制器，用于向该传输设备发送切换指令。
第四方面，提供了提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种实现方式所述的方法。
第五方面，提供了提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种实现方式所述的方法。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍。
图1为本发明实施例提供的多媒体通信系统100的组网结构示意图；
图2为本发明实施例提供的方法100的流程示意图；
图3A-3C和图4A-4C为本发明实施例提供的切换示意图；
图5是本发明实施例2提供的传输设备200的结构示意图；
图6是本发明实施例3提供的传输设备1000的结构示意图；
具体实施方式
下面结合附图，对本发明的实施例进行描述。
图1为本发明实施例提供的一种多媒体通信系统100的组网结构示意图，多媒体通信系统100包括视频源设备110和传输设备120。其中，视频源设备110通过传输设备120向视频终端发送视频流。传输设备120可以接收来自多个视频源设备110的视频流，并向视频终端仅转发其中一路视频流，还可以对转发的视频流进行切换。传输设备120具体可以为交换机、路由器等，视频源设备可以为IP摄像机、对视频数据进行IP封装传输的设备等。多媒体通信系统100具体可以是TV制播系统，如电视台制播网络的总控系统、演播室系统、播出系统等对承载在IP网络上的视频流有净切换需求的系统。
如图1所示，多媒体通信系统100还可以包括控制器(Controller)130，控制器130用于向传输设备120发送用于指示传输设备120进行视频流切换的切换指令，相应地，传输设备120收到该切换指令后进行视频流切换。
在本发明实施例中，视频的每个视频帧封装在多个数据报文中传输，后续将封装每个视频帧的多个数据报文中的最后一个数据报文称为帧尾报文。本发明实施例中的数据报文携带序列号和/时间戳、以及用于判断该数据报文是否为帧尾报文的信息，具体可以是实时传输协议(Real-time Transport Protocol，RTP)报文、基于私有协议的报文等。根据动画和电视工程师协会(The Society of Motion Picture and Television Engineers，SMPTE)2022和 SMPTE 2110标准，采用RTP承载制播网的音视频数据。如，标准SMPTE 2022-6(Transport of High Bit Rate Media Signals over IP Networks(HBRMT))通过RTP封装SDI格式的视频数据并通过基于IP的网络传输。
本发明实施例中的视频帧指的是未压缩视频帧，可以是YUV视频帧或RGB视频帧，同一视频流的单个视频帧的报文个数(即封装该视频流的一个视频帧所需的数据报文的数量)一般是相同的，通常可以多达数千个，可以是预先设置的，也可以是根据接收的该视频流的两个相邻的帧尾报文的序列号的差值计算得出的。同一个视频流的单个视频帧的时长通常是相同的，可以是预先设置的，也可以是根据接收的该视频流的两个相邻的帧尾报文的时间戳的差值计算得出的。
本发明实施例中的切换报文指的是处于切换位置的数据报文，所述切换报文的切换参考信息为所述切换报文中的用于视频流切换的信息，可以包括所述切换报文的序列号(后续可以简称为切换序列号)和所述切换报文的时间戳(后续可以简称为切换时间戳)中的至少一个。进行视频流切换时，根据要切出的视频流(后续简称为切出流)的切换报文的切换参考信息和要切入的视频流(后续简称为切入流)的切换参考信息进行切换，如，对于切出流，不再转发序列号大于切出流的切换序列号的数据报文，对于切入流，开始转发序列号大于切入流的切换序列号的数据报文。
下面结合图2、图3A-3C和图4A-4C介绍本发明实施例一提供的方法100。图2是本发明实施例1提供的方法流程示意图，本发明实施例1的方法可以应用于图1所示的多媒体通信系统100，由传输设备执行。
如图2所示，方法100包括如下步骤：
S110：传输设备(如图1所示的传输设备120)接收第一视频流和第二视频流并通过该传输设备的目标出端口发送所述第一视频流。
具体地，该传输设备通过该目标出端口转发接收的第一视频流的数据报文并不通过所述目标出端口转发接收的第二视频流的数据报文。
该传输设备可能接收多个与该目标出端口对应的视频流，通过该目标出端口发送该多个视频流中的一个视频流，并在该多个视频流中对通过该目标出端口发送的视频流进行切换。第一视频流为执行步骤S110时正在通过该目标出端口发送的一个视频流，第二视频流为该多个视频流中的其它视频流中的一个视频流。
S120：所述传输设备接收用于指示将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流的切换指令。
具体可以是从控制器(如图1所示的控制器130)接收该切换指令。
该切换指令可以包括该目标出端口的端口号、该第一视频流的标识(如第一视频流的组播地址)和该第二视频流的标识(如第二视频流的组播地址)，以指示该传输设备将通过该目标出端口发送的该第一视频流切换为第二视频流，既指示该传输设备停止通过该目标出端口转发该第一视频流并开始通过该目标出端口转发该第二视频流。
S130：所述传输设备根据所述切换指令确定所述第一视频流的基准帧尾报文(后续称为第一基准帧尾报文)的切换参考信息和所述第二视频流的基准帧尾报文(后续称为第二基准帧尾报文)的切换参考信息。
所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息可以均为序列号、均为时间戳、或均包括序列号和时间戳。
具体可以将执行步骤S130之前接收的所述第一视频流的倒数第x个帧尾报文和所述第二视频流的倒数第x个帧尾报文分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，其中x为自然数。x取值可以为1，x为1时，第一视频流和第二视频流中任一视频流的基准帧尾报文又可以称为执行步骤S130时该视频流的最新帧尾报文。
在具体实现时，可以根据所述传输设备接收到所述切换指令的时间确定所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息。如下为两种示例实施方式。
实施方式M：收到切换指令后立即执行步骤S130，即步骤S120和步骤S130的执行时间非常接近，则可以将接收到所述切换指令之前接收的所述第一视频流的倒数第x个帧尾报文的切换参考信息和接收的所述第二视频流的倒数第x个帧尾报文的切换参考信息分别确定为所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息，即所述第一基准帧尾报文和所述第二基准帧尾报文分别为接收到所述切换指令之前接收的所述第一视频流的倒数第x个帧尾报文和接收的所述第二视频流的倒数第x个帧尾报文。x为1时的第一基准帧尾报文和所述第二基准帧尾报文如图3A-3C所示。其中，如图3A所示，将帧尾报文101和帧尾报文201分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为20和10。如图3B所示，将帧尾报文103和帧尾报文203分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为10和20。如图3C所示，将帧尾报文105和帧尾报文205分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为10和10。
实施方式N：收到切换指令后再在分别接收到第一视频流的至少一个帧尾报文和第二视频流的至少一个帧尾报文后执行步骤S130。可以是在分别接收到第一视频流的至少一个帧尾报文和第二视频流的至少一个帧尾报文时执行步骤S130，将此时接收的第一视频流的最后一个帧尾报文的切换参考信息和第二视频流的最后一个帧尾报文的切换参考信息分别确定为所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息。如图4A所示，将帧尾报文107和帧尾报文207分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为30和20。如图4B所示，将帧尾报文109和帧尾报文209分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为20和30。如图4C所示，将帧尾报文111和帧尾报文211分别确定为所述第一基准帧尾报文和所述第二基准帧尾报文，序列号分别为20和20。如果在接收到切换指令后对于第一视频流和第二视频流均根据相邻帧尾报文的序列号计算单个视频帧的报文个数，则还可以是在分别接收到第一视频流的至少两个帧尾报文和第二视频流的至少两个帧尾报文时执行步骤S130，将此时接收的第一视频流的最后一个帧尾报文的切换参考信息和第二视频流的最后一个帧尾报文的切换参考信息分别确定为所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息。
S140：根据所述第一基准帧尾报文的切换参考信息确定所述第一视频流的切换报文(后续称为第一切换报文)的切换参考信息，并根据所述第二基准帧尾报文的切换参考信息确定所述第二视频流的切换报文(后续称为第二切换报文)的切换参考信息。
所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息可以均为序列号、均为时间戳、或均包括序列号和时间戳。
为便于表述，后续还可将第一切换报文的序列号称为第一切换序列号、将第二切换报文的序列号称为第二切换序列号、将第一切换报文的时间戳称为第一切换时间戳、将第二切换报文的序列号称为第二切换时间戳。
所述第一切换报文中的视频数据所属视频帧为所述第一基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧；所述第二切换报文中的视频数据所属视频帧为所述第二基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧。当x取值为1时，根据该实现方式实现在基准帧尾报文后至少一帧的位置处进行视频流切换，可以有效避免对当前正在传输的视频画面的损伤。
具体可以根据所述第一基准帧尾报文的序列号和封装所述第一视频流的单个视频帧的报文个数计算所述第一切换序列号、并根据所述第二基准帧尾报文的序列号和封装所述第二视频流的单个视频帧的报文个数计算所述第二切换序列号,确定的所述第一切换序列号与所述第一基准帧尾报文的序列号的差值可以大于或等于封装所述第一视频流的单个视频帧的报文个数，确定的所述第二切换序列号与所述第二基准帧尾报文的序列号的差值可以大于或等于封装所述第二视频流的单个视频帧的报文个数。
具体还可以根据所述第一基准帧尾报文的时间戳和所述第一视频流的单个视频帧的时长计算所述第一切换时间戳、并根据所述第二基准帧尾报文的时间戳和所述第二视频流的单个视频帧的时长计算所述第二切换时间戳,确定的所述第一切换时间戳与所述第一基准帧尾报文的时间戳的差值可以大于或等于所述第一视频流的单个视频帧的时长，确定的所述第二切换时间戳与所述第二基准帧尾报文的时间戳的差值可以大于或等于所述第二视频流的单个视频帧的时长。
在具体实现时，所述传输设备可以先根据该第一基准帧尾报文的切换参考信息确定该第一视频流的对齐帧尾报文(后续称为第一对齐帧尾报文)的切换参考信息，再根据该第一对齐帧尾报文的切换参考信息确定该第一切换报文的切换参考信息，如，将所述第一对齐帧尾报文的切换参考信息作为所述第一切换报文的切换参考信息，又如，将用于封装所述第一对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为所述第一切换报文的切换参考信息，该第一对齐帧尾报文为该第一基准帧尾报文的后续帧尾报文。并且可以先根据该第二视频流的第二基准帧尾报文的切换参考信息确定该第二视频流的对齐帧尾报文(后续称为第二对齐帧尾报文)的切换参考信息，再根据该第二对齐帧尾报文的切换参考信息确定该第二切换报文的切换参考信息，如，将所述第二对齐帧尾报文的切换参考信息作为所述第二切换报文的切换参考信息，又如，将用于封装所述第二对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的切换参考信息确定为所述第二切换报文的切换参考信息，该第二对齐帧尾报文为该第二基准帧尾报文的后续帧尾报文。该第一对齐帧尾报文和该第二对齐帧尾报文为在时间上对应的帧尾报文，即对于第一对齐帧尾报文，第二对齐帧尾报文是第二视频的所有帧尾报文中与第一对齐帧尾报文的时间戳最接近的帧尾报文。可以理解的是，对于第一对齐帧尾报文对应的视频帧，第二对齐帧尾报文对应的视频帧为第二视频的所有视频帧中与其时间上最接近的视频帧。所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、第一对齐帧尾报文的切换参考信息、第二对齐帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息可以均为序列号、均为时间戳、或均包括序列号和时间戳。
具体可以通过如下实施方式A和实施方式B中的一种或其组合确定第一视频流和第二视频流中的任一视频流的对齐帧尾报文的切换参考信息并确定所述视频流的切换报文的切换参考信息。其中，实施方式A用于确定所述切换报文的序列号，实施方式B用于确定所述切换报文的时间戳。
实施方式A：
在实施方式A中，该传输设备可以先根据该第一基准帧尾报文的序列号和该第一视频流的单个视频帧的报文个数计算该第一对齐帧尾报文的序列号，再根据该第一对齐帧尾报文的序列号确定该第一切换报文的序列号。并且可以先根据该第二视频流的第二基准帧尾报文的序列号和该第二视频流的单个视频帧的报文个数计算该第二对齐帧尾报文的序列号，再根据该第二对齐帧尾报文的序列号确定该第二切换报文序列号。
如图3A所示，收到切换指令之前记录了第一基准帧尾报文101的序列号20和第二基准帧尾报文201的序列号10，收到切换指令后立即确定对齐帧尾报文的序列号，其中确定的第一对齐帧尾报文的序列号为帧尾报文102的序列号30，即第一对齐帧尾报文为帧尾报文102，确定的第二对齐帧尾报文的序列号为帧尾报文202的序列号30，即第二对齐帧尾报文为帧尾报文202。帧尾报文102和帧尾报文202为在时间上对应的帧尾报文。假定第一视频流和第二视频流的帧率一样且起始帧号一样，帧尾报文102和帧尾报文202均为帧号为n+2的视频帧的帧尾报文。
具体可以通过实施方式A的如下两种实施方式(实施方式A1和实施方式A2)中的一种确定第一视频流和第二视频流中的任一视频流的对齐帧尾报文的序列号。
实施方式A1：考虑到不同视频流之间可能会不同步的问题，在实施方式A1中，传输设备先确定所述第一视频流和所述第二视频流的同步关系，进而对于第一视频流和第二视频流中的任一视频流，根据该视频流的基准帧尾报文的序列号、以及该同步关系，计算该视频流的对齐帧尾报文的序列号。所述第一视频流和所述第二视频流的同步关系，具体可以是所述第一视频流早于所述第二视频流、所述第一视频流晚于所述第二视频流或所述第一视频流与所述第二视频流同步。具有互相切换的需求的视频流之间的失步通常不会超过一帧。
确定第一视频流与第二视频流的同步关系之后，具体通过如下方式确定第一视频流的第一对齐帧尾报文的序列号和第二视频流的第二对齐帧尾报文的序列号。为便于表述，在本发明实施例中，将第一基准帧尾报文的序列号表示为S11，将第一对齐帧尾报文的序列号表示为S12，将第二基准帧尾报文的序列号表示为S21，将第二对齐帧尾报文的序列号表示为S22，将所述第一视频流的单个视频帧的报文个数表示为M1，将所述第二视频流的单个视频帧的报文个数表示为M2，M1与M2可以相同，也可以不同，通常是相同的。在如下方式的示例中，假定M1和M2均为10。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流早于所述第二视频流时，根据公式S12＝S11+m*M1确定所述第一对齐帧尾报文的序列号S12，并根据公式S22＝S21+(m+1)*M2确定所述第二对齐帧尾报文的序列号S22，m为自然数，可以取1。
假定m为1且采用实施方式M。如图3A所示，所述第一基准帧尾报文为帧尾报文101，序列号为20，第一对齐帧尾报文102的序列号为20+10＝30，所述第二基准帧尾报文为帧尾报文201，序列号为10，第二对齐帧尾报文202的序列号为10+2*10＝30。
假定m为1且采用实施方式N。如图4A所示，所述第一基准帧尾报文为帧尾报文107，序列号为30，第一对齐帧尾报文108的序列号为30+10＝40，所述第二基准帧尾报文为帧尾报文207，序列号为20，第二对齐帧尾报文208的序列号为20+2*10＝40。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流晚于所述第二视频流时，根据公式S12＝S11+(n+1)*M1确定所述第一对齐帧尾报文的序列号S12，并根据公式S22＝S21+n*M2确定所述第二对齐帧尾报文的序列号S22，n为自然数，可以取1。
假定n为1且采用实施方式M。如图3B所示，所述第一基准帧尾报文为帧尾报文103， 序列号为10，第一对齐帧尾报文104的序列号为10+2*10＝30，所述第二基准帧尾报文为帧尾报文203，序列号为20，第二对齐帧尾报文204的序列号为20+10＝30。
假定n为1且采用实施方式N。如图4B所示，所述第一基准帧尾报文为帧尾报文109，序列号为20，第一对齐帧尾报文110的序列号为20+2*10＝40，所述第二基准帧尾报文为帧尾报文209，序列号为30，第二对齐帧尾报文210的序列号为30+10＝40。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流和所述第二视频流同步时，根据公式S12＝S11+k*M1确定所述第一对齐帧尾报文的序列号S12，并根据公式S22＝S21+k*M2确定所述第二对齐帧尾报文的序列号S22，k为自然数，可以取1。
假定k为1且采用实施方式M。如图3C所示，所述第一基准帧尾报文为帧尾报文105，序列号为10，第一对齐帧尾报文106的序列号为10+10＝20，所述第二基准帧尾报文为帧尾报文205，序列号为10，第二对齐帧尾报文206的序列号为10+10＝20。
假定k为1且采用实施方式N。如图4C所示，所述第一基准帧尾报文为帧尾报文111，序列号为20，第一对齐帧尾报文112的序列号为20+10＝30，所述第二基准帧尾报文为帧尾报文211，序列号为20，第二对齐帧尾报文212的序列号为20+10＝30。
通过实施方式A1确定的第一对齐帧尾报文的序列号和第二对齐帧尾报文的序列号所对应的帧尾报文(即第一对齐帧尾报文和第二对齐帧尾报文)的时间戳基本是一致的，据此确定的切换报文的时间戳基本上也是一致的，因此，可以保证切换时切出流和切入流在时间维度上正好衔接，不会发生多帧或少帧的问题。
实施方式A2：在实施方式A2中不考虑不同视频流之间的同步关系，而是假定视频流之间是同步的，可以直接根据公式S12＝S11+k*M1确定所述第一对齐帧尾报文的序列号S12，并根据公式S22＝S21+k*M2确定所述第二对齐帧尾报文的序列号S22。
在确定第一对齐帧尾报文的序列号和第二对齐帧尾报文的序列号之后，可以通过如下两种实施方式(实施方式X1和实施方式X2)中的一种分别根据第一对齐帧尾报文的序列号和第二对齐帧尾报文的序列号确定第一切换序列号和第二切换序列号。
实施方式X1：实施方式X1用于实现在帧边界处切换视频流。在实施方式X1中，将所述第一视频流的第一对齐帧尾报文的序列号作为所述第一切换序列号，并将所述第二视频流的第二对齐帧尾报文的序列号作为所述第二切换序列号。
如图3A所示，将第一对齐帧尾报文102的序列号30作为第一切换序列号，即将第一对齐帧尾报文102作为第一视频流的切换报文，将第二对齐帧尾报文202的序列号30作为第二切换序列号，即将第二对齐帧尾报文202作为第二视频流的切换报文。
实施方式X2：实施方式X2用于实现在视频帧的消隐区切换视频流。
在视频领域，在将光信号转换为电信号的扫描过程中，扫描总是从图像的左上角开始，水平向右行进，同时扫描点也以较慢的速率向下移动。当扫描点到达图像右侧边缘时，扫描点快速返回左侧，重新开始在第1行的起点下面进行第2行扫描，行与行之间的返回过程称为水平消隐。一幅完整的图像扫描信号，由水平消隐间隔分开的行信号序列构成，称为一帧。扫描点扫描完一帧后，要从图像的右下角返回到图像的左上角，开始新一帧的扫描，这一时间间隔，叫做垂直消隐，也称场消隐。水平消隐、垂直消隐在SDI信号中不承载可视内容，统称消隐区。为了指导在SDI格式的视频流之间进行切换，SMPTE RP-168规定了切换行或切换区。在切换行或切换区切换两路SDI信号，不会对有效的视频数据产生损伤。因为SMPTE2022-6是将SDI信号承载在IP报文中，所以对采用SMPTE 2022-6封装的视频流进行净切换时，应该在消隐区切换。
综上，如果所述第一视频流和所述第二视频流均为采用SMPTE 2022-6编码的视频流，可以在封装消隐区的数据报文处进行视频流切换。
对于第一视频流，具体可以将用于封装所述第一对齐帧尾报文的序列号所对应的视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第一切换序列号。如，将所述第一视频流的第一对齐帧尾报文的序列号和ΔM1的和确定为所述第一切换序列号，
Figure PCTCN2019101241-appb-000001
Figure PCTCN2019101241-appb-000002
或
Figure PCTCN2019101241-appb-000003
其中，OL1为所述第一视频流的单个视频帧中每一行像素所占的字节数，SL1为所述第一视频流的切换行号。
对于第二视频流，将用于封装所述第二对齐帧尾报文的序列号所对应的视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第二切换序列号。如，将所述第二视频流的第二对齐帧尾报文的序列号和ΔM2的和确定为所述第二切换序列号，
Figure PCTCN2019101241-appb-000004
Figure PCTCN2019101241-appb-000005
或
Figure PCTCN2019101241-appb-000006
其中，OL2为所述第二视频流的单个视频帧中每一行像素所占的字节数，SL2为所述第二视频流的切换行号。
切换行号可以取决于SDI视频标准，目前SMPTE RP-168规定了大部分视频格式的切换行号。第一视频流的切换行号SL1和第二视频流的切换行号SL2通常是相同的。
对于所述第一视频流和所述第二视频流中的任一视频流，如果采用4:2:2采样方式，所述视频流的单个视频帧中每一行像素所占的字节数OL＝PL*BS*2/8，其中，PL为所述视频流的视频帧的每一行的总像素数(包括行消隐像素的数量和视频有效像素的数量)，BS为像素位深。可以理解的是，如果第一视频流和第二视频流的像素位深和采样方式是相同的，通常OL1和OL2是相同的。
实施方式B：
在实施方式B中，该传输设备可以先根据该第一基准帧尾报文的时间戳和该第一视频流的单个视频帧的时长计算该第一对齐帧尾报文的时间戳，再根据该第一对齐帧尾报文的时间戳确定该第一切换报文的时间戳。并且可以先根据该第二视频流的第二基准帧尾报文的时间戳和该第二视频流的单个视频帧的时长计算该第二对齐帧尾报文的时间戳，再根据该第二对齐帧尾报文的时间戳确定该第二切换报文时间戳。
具体可以通过实施方式B的如下两种实施方式(实施方式B1和实施方式B2)中的一种确定第一视频流和第二视频流中的任一视频流的对齐帧尾报文的时间戳。
实施方式B1：考虑到不同视频流之间可能会不同步的问题，在实施方式B1中，传输设备先确定所述第一视频流和所述第二视频流的同步关系，进而对于第一视频流和第二视频流中的任一视频流，根据该视频流的基准帧尾报文的时间戳、以及该同步关系，计算该视频流的对齐帧尾报文的时间戳。所述第一视频流和所述第二视频流的同步关系，具体可以是所述第一视频流早于所述第二视频流、所述第一视频流晚于所述第二视频流或所述第一视频流与所述第二视频流同步。具有互相切换的需求的视频流之间的失步通常不会超过一帧。
确定第一视频流与第二视频流的同步关系之后，具体通过如下方式确定第一视频流的第一对齐帧尾报文的时间戳和第二视频流的第二对齐帧尾报文的时间戳。为便于表述，在本发明实施例中，将第一基准帧尾报文的时间戳表示为t11，将第一对齐帧尾报文的时间戳表示为t12，将第二基准帧尾报文的时间戳表示为t21，将第二对齐帧尾报文的时间戳表示为t22，将所述第一视频流的单个视频帧的时长表示为T1，将所述第二视频流的单个视频帧的时长表示为T2，T1与T2可以相同，也可以不同，通常是相同的。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流早于所述第二视频流时，根据公式t12＝t11+m*T1确定所述第一对齐帧尾报文的时间戳t12，并根据公式 t22＝t21+(m+1)*T2确定所述第二对齐帧尾报文的时间戳t22，m为自然数，可以取1。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流晚于所述第二视频流时，根据公式t12＝t11+(n+1)*T1确定所述第一对齐帧尾报文的时间戳t12，并根据公式t22＝t21+n*T2确定所述第二对齐帧尾报文的时间戳t22，n为自然数，可以取1。
在确定的所述第一视频流和所述第二视频流的同步关系为所述第一视频流和所述第二视频流同步时，根据公式t12＝t11+k*T1确定所述第一对齐帧尾报文的时间戳t12，并根据公式t22＝t21+k*T2确定所述第二对齐帧尾报文的时间戳t22，k为自然数，可以取1。
通过实施方式B1确定的第一对齐帧尾报文的时间戳和第二对齐帧尾报文的时间戳基本是一致的，据此确定的切换报文的时间戳基本上也是一致的，因此，可以保证切换时切出流和切入流在时间维度上正好衔接，不会发生多帧或少帧的问题。
实施方式B2：在实施方式B2中不考虑不同视频流之间的同步关系，而是假定视频流之间是同步的，可以直接根据公式t12＝t11+k*T1确定所述第一对齐帧尾报文的时间戳t12，并根据公式t22＝t21+k*T2确定所述第二对齐帧尾报文的时间戳t22。
在确定第一对齐帧尾报文的切换参考信息和第二对齐帧尾报文的时间戳之后，可以通过如下两种实施方式(实施方式Y1和实施方式Y2)中的一种分别根据第一对齐帧尾报文的时间戳和第二对齐帧尾报文的时间戳确定第一切换时间戳和第二切换时间戳。
实施方式Y1：实施方式Y1用于实现在帧边界处切换视频流。在实施方式Y1中，将所述第一视频流的第一对齐帧尾报文的时间戳作为所述第一切换时间戳，并将所述第二视频流的第二对齐帧尾报文的时间戳作为所述第二切换时间戳。
实施方式Y2：实施方式Y2用于实现在视频帧的消隐区切换视频流。
对于第一视频流，具体可以将用于封装所述第一对齐帧尾报文的时间戳所对应的视频帧的下一视频帧的消隐区数据的数据报文的时间戳确定为所述第一切换时间戳。如，将所述第一视频流的第一对齐帧尾报文的时间戳和Δt1的和确定为所述第一切换时间戳，Δt1为属于区间[tx，ty]的值，如Δt1为tx或(tx+ty)/2，其中，tx＝T1*SL1/L1，ty＝T1*(SL1+1)/L1，其中，SL1为所述第一视频流的切换行号,L1为所述第一视频流的一个视频帧的像素的总行数。
对于第二视频流，将用于封装所述第二对齐帧尾报文的时间戳所对应的视频帧的下一视频帧的消隐区数据的数据报文的时间戳确定为所述第二切换时间戳。如，将所述第二视频流的第二对齐帧尾报文的时间戳和Δt2的和确定为所述第二切换时间戳，Δt2为属于区间[tm，tn]的值，如Δt2为tm或(tm+tn)/2，其中，tm＝T2*SL2/L2，ty＝T2*(SL2+1)/L2，其中，SL2为所述第二视频流的切换行号,L2为所述第二视频流的一个视频帧的像素的总行数。需要说明的是，根据相同的方法计算Δt1和Δt2，如，利用公式Δt1＝T1*SL1/L1计算Δt1(即Δt1为tx)并利用公式Δt2＝T2*SL2/L2计算Δt2(即Δt2为tm)。
切换行号可以取决于SDI视频标准，目前SMPTE RP-168规定了大部分视频格式的切换行号。第一视频流的切换行号SL1和第二视频流的切换行号SL2通常是相同的。第一视频流的L2和第二视频流的L2通常也是相同的。
在本发明实施例1中，该传输设备具体可以根据所述第一视频流的基准帧尾报文的时间戳、以及所述第二视频流的基准帧尾报文的时间戳确定所述第一视频流和所述第二视频流的同步关系。
考虑到设备同步本身就有精度偏差，同一个时刻从不同视频源设备发出的数据报文，其时间戳可能会有偏差。另外，IP网络以报文为单位，报文转发的抖动也会引入一些同步偏差。 因此，可以认为，如果两个基准帧尾报文的时间戳偏差小于预设的时差阈值即可认为这两个视频流同步。用于判断同步与否的时差阈值可以根据网络中视频源设备的时钟同步精度决定。
传输设备具体可以在确定t1-t2>Δt1时，确定所述第一视频流早于所述第二视频流；在确定t2-t1>Δt1时，确定所述第一视频流晚于所述第二视频流；在确定|t1-t2|<Δt2时，确定所述第一视频流与所述第二视频流同步。t1为所述第一基准帧尾报文的时间戳；t2为所述第二基准帧尾报文的时间戳；Δt1为预设的第一时差阈值，Δt2为预设的第二时差阈值，Δt1和Δt2均为大于0且小于T的值，Δt2可以和Δt1相同也可以小于Δt1。可以按照如下条件预设Δt1和Δt2：T/2<Δt1<T，如Δt1可以为3*T/4，0<Δt2<T/2，如Δt2可以为T/4，T为单个视频帧的时长。在本发明实施例中，T可以是预先设置的或根据预先设置的视频信息(如帧率)计算得到的。另外，由于具有互相切换的需求的视频流的单个视频帧的时长通常是一样的，因此，T也可以是根据接收的第一视频流和第二视频流中的任一视频流的数据报文计算得到的，如，根据该视频流的两个相邻的帧尾报文的时间戳的差值计算得出。
S150：所述传输设备根据所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流。
在一种实现方式中，所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均为序列号，该传输设备具体可以不通过所述目标出端口发送接收的所述第一视频流的序列号大于所述第一切换序列号的数据报文，并通过所述目标出端口发送接收的所述第二视频流的序列号大于所述第二切换序列号的数据报文。
由于第一切换序列号和第二切换序列号为计算得出的尚未接收到的数据报文的序列号，在计算出第一切换序列号和第二切换序列号时，通常还会接收到第一视频流的序列号小于或等于第一切换序列号的数据报文和第二视频流的序列号小于或等于第二切换序列号的数据报文。该传输设备在步骤S150中可以根据接收的第一视频流的数据报文的序列号确定是否要通过该目标出端口发送该数据报文，如果小于或等于该第一切换序列号，则照常通过该目标出端口发送该数据报文，否则不通过该目标出端口发送该数据报文。相应地，该传输设备可以根据接收的第二视频流的数据报文的序列号确定是否要通过该目标出端口发送该数据报文，如果小于或等于该第二切换序列号，则照常不通过该目标出端口发送该数据报文，否则通过该目标出端口发送该数据报文。
在另一种实现方式中，所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均为时间戳，该传输设备具体可以不通过所述目标出端口发送接收的所述第一视频流的时间戳大于所述第一切换时间戳的数据报文，并通过所述目标出端口发送接收的所述第二视频流的时间戳大于所述第二切换时间戳的数据报文。另外，该传输设备还可以通过所述目标出端口发送接收的所述第一视频流的时间戳小于或等于所述第一切换时间戳的数据报文，并不通过所述目标出端口发送接收的所述第二视频流的时间戳小于或等于所述第二切换时间戳的数据报文
在再一种实现方式中，所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均包括序列号和时间戳。对于接收的所述第一视频流的数据报文，如果所述数据报文的时间戳与所述第一切换报文的时间戳的差值的绝对值Dif1大于阈值1且所述数据报文的时间戳大于所述第一切换报文的时间戳，或者Dif1小于或等于阈值1且所述数据报文的序列号大于所述第一切换报文的序列号，不通过所述目标出端口转发所述数据报文。对于接收的所述第二视频流的数据报文，如果所述数据报文的时间戳与所述第二切换报文的时间戳的差值的绝对值Dif2大于阈值2且所述数据报文的时间戳大于所述第二切换报文的时间戳，或者Dif2 小于或等于阈值2且所述数据报文的序列号大于所述第二切换报文的序列号，通过所述目标出端口转发所述数据报文。另外，对于接收的所述第一视频流的数据报文，如果Dif1大于阈值1且所述数据报文的时间戳小于所述第一切换报文的时间戳，或者Dif1小于或等于阈值1且所述数据报文的序列号小于或等于所述第一切换报文的序列号，通过所述目标出端口转发所述数据报文。对于接收的所述第二视频流的数据报文，如果Dif2大于阈值2且所述数据报文的时间戳小于所述第二切换报文的时间戳，或者Dif2小于或等于阈值2且所述数据报文的序列号小于所述第二切换报文的序列号，不通过所述目标出端口转发所述数据报文。
其中，阈值1和阈值2可以相同也可以不同，均可以设置为大于或等于1/4T的值，具体可以设置为大于或等于1/4T且小于或等于7/4T的值，如3/4T，其中，T为所述第一视频流或所述第二视频流的单个视频帧的时长。
对于第一视频流和第二视频流中的任一视频流，可以在确定该视频流进入稳定状态后，如确定在开始执行步骤S150之后已处理该数据流的预定个数(如M、2M个)的数据报文、或在开始执行步骤S130之后经过了预设时长(如T、2T)，此时再接收到该数据流的序列号或时间戳小于或等于该数据流的切换序列号或切换时间戳的数据报文的可能性很小，因此可以不再逐报文检查接收的该数据流的数据报文的序列号或时间戳来判断是否通过该目标出端口发送该数据报文，而是基于用于表明该数据报文所属的数据流(即该数据流)是否为需要发送的数据流的状态信息来确定是否通过该目标出端口发送该数据报文，从而提高转发效率。其中,M为该视频流的单个视频帧的报文个数，T为该视频流的单个视频帧的时长。
需要说明的是，该传输设备可以包括多个出端口，如果方法100的步骤S110和S150由仅负责处理该目标出端口所对应的数据报文的单元来执行，则在步骤S110和S150中，该单元可以丢弃不需要通过所述目标出端口转发的数据报文。
为了减少对下游设备的影响，以实现下游设备对切换不感知，所述传输设备在转发所述第二视频流的数据报文时，还可以修改所述第二视频流的数据报文中的信息。
具体地，为了提升切换前后的视频流的一致性，该传输设备可以修改所述第二视频流的数据报文中的第一字段的值以使得切换后通过该目标出端口发出的第二视频流的数据报文中的第一字段的值和切换前通过该目标出端口发出的第一视频流的数据报文中的第一字段的值相同。所述第一字段可以为组播地址字段或载荷类型字段。如果采用RTP进行视频流传输，该第一字段还可以为同步源(Synchronization source，SSRC)字段。所述第一字段还可以为SMPTE 2022-6定义的HBRMT层参数，如FRCount，FRCount是SMPTE 2022-6定义的高码率媒体载荷头(High Bit Rate Media Payload Header)中的帧计数(Frame Count)字段，表示该视频流中累积的视频帧数。
对于一个出端口，可能需要在多条输入视频流间反复切换。所以，当前的切出流很可能是之前的切入流。
该传输设备可以在通过该目标出端口发送该多个视频流中的任一视频流时，均将该视频流的数据报文中的第一字段的值，修改为预设的值，如，将该视频流的数据报文中的组播地址修改为预设的组播地址(如239.0.2.1)。
如S110中所述，该传输设备可能接收多个与该目标出端口对应的视频流，通过该目标出端口发送该多个视频流中的一个视频流，并在该多个视频流中对通过该目标出端口发送的视频流进行切换。如，该多个视频流包括流1、流2和流3，该传输设备首先转发的是流2，后续依次切换到流1和流3。该传输设备还可以在第一次通过该目标出端口转发该多个视频流中的一个视频流(如流2)时，记录该视频流的第一字段的值，并在后续切换到其它视频流 时，将其它视频流的数据报文中的第一字段的值修改为记录的该视频流的第一字段的值。如，在切换到流1时将流1的数据报文中的第一字段的值修改为流2的第一字段的值，并在切换到流3时将流3的数据报文中的第一字段的值也修改为流2的第一字段的值。
为了提升切换前后的视频流的连续性，所述传输设备在切换后发送所述第二视频流的数据报文时，还可以修改所述第二视频流的数据报文中的第二字段的值以使得切换后通过该目标出端口发出的第二视频流的数据报文中的第二字段的值和切换前通过该目标出端口发出的第一视频流的数据报文中的第二字段的值连续。所述第二字段的值可以为序列号、帧号和时间戳等中的一个或多个。
为了保证第二字段的值(如序列号)连续，具体可以将要发送的第二视频流的数据报文中的第二字段的值增加ΔV，其中，ΔV＝V1-V2，V1为所述第一视频流的序列号等于所述第一切换序列号的数据报文在被通过该目标出端口发出时所携带的第二字段的值,V2为所述第二视频流的序列号等于所述第二切换序列号的数据报文在被接收时所携带的第二字段的值。
在执行步骤S130之前，传输设备还可以执行步骤S115以记录第一视频流和第二视频流的最新帧尾报文的目标信息，记录的目标信息包括最新帧尾报文的序列号和时间戳中的至少一个。相应地，在步骤S130中可以根据在步骤S115中记录的最新帧尾报文的目标信息确定基准帧尾报文，如，对于第一视频流和第二视频流中的任一视频流，将执行步骤S130之前最后一次通过步骤S115记录的该视频流的最新帧尾报文的目标信息(如序列号)作为该视频流的基准帧尾报文的切换参考信息(如序列号)。
该传输设备可以在步骤S120之前开始执行步骤S115。如S110中所述，该传输设备可能接收多个与该目标出端口对应的视频流，通过该目标出端口发送该多个视频流中的一个视频流，并在该多个视频流中对通过该目标出端口发送的视频流进行切换。则在该传输设备开始通过该目标出端口转发该多个视频流中的视频流时开始记录该多个视频流中的每个视频流的最新帧尾报文的目标信息。当后续要对其中两个视频流进行切换时，可以利用已经记录的这两个视频流的最新帧尾报文的目标信息执行步骤S130-S140以确定切换参考信息。当实现上述实施方式M时，可以在步骤S120之前开始执行步骤S115。
该传输设备也可以在步骤S120之后开始执行步骤S115。即，在接收到切换指令后开始记录要进行切换的两个视频流中的每个视频流的最新帧尾报文的目标信息，并进而根据记录的这两个视频流的最新帧尾报文的目标信息执行步骤S130-S140以确定切换报文的切换参考信息。当实现上述实施方式N时，可以在步骤S120之后开始执行步骤S115。
如果在步骤S120之前开始执行步骤S115，对于该多个视频流中的任一视频流，该传输设备可以在接收到该视频流中在顺序上的第一个帧尾报文时，将该帧尾报文的目标信息记录为该视频流的最新帧尾报文的目标信息，并根据后续接收的该视频流的数据报文更新该视频流的最新帧尾报文的目标信息。进而，在接收到切换指令后立即执行步骤S130-S140以根据接收到切换指令前记录的最新帧尾报文的目标信息(如序列号)计算切换报文的切换参考信息(如序列号)。如图3A所示，根据收到切换指令前记录的第一视频流的最新帧尾报文101的序列号20计算第一切换序列号，根据第二视频流的最新帧尾报文201的序列号10计算第二切换序列号。
如果在步骤S120之后开始执行步骤S115，对于所述第一视频流和所述第二视频流中的任一视频流，该传输设备可以在接收到该视频流中在顺序上的第一个帧尾报文时，将该帧尾报文的目标信息记录为该视频流的最新帧尾报文的目标信息。如果在收到其中一路视频流的另一帧尾报文时还没有收到另一视频流的第一个帧尾报文，则可以对该视频流的最新帧尾报 文的目标信息进行更新,直至收到另一视频流的第一个帧尾报文并相应记录另一个视频流的最新帧尾报文的目标信息。另外，如果对于第一视频流和第二视频流均根据相邻帧尾报文的序列号计算单个视频帧的报文个数，则在计算切换序列号之前对于每个视频流都至少接收两个帧尾报文，每次接收到帧尾报文时均将该帧尾报文的目标信息记录为所属视频流的最新帧尾报文的目标信息。此外，可以将接收的两个相邻帧尾报文的序列号的差值作为所属视频流的单个视频帧的报文个数，还可以将接收的两个相邻帧尾报文的时间戳的差值作为所属视频流的单个视频帧的时长。
如图4A所示，根据收到切换指令后记录的第一视频流的最新帧尾报文107的序列号30计算第一切换序列号，根据第二视频流的最新帧尾报文207的序列号20计算第二切换序列号。
传输设备具体可以通过如下方式对任一视频流的最新帧尾报文的目标信息进行更新。
所述传输设备可以在每次接收到该视频流的帧尾报文时，将该视频流的最新帧尾报文的序列号更新为该帧尾报文的序列号，并将该视频流的最新帧尾报文的时间戳更新为该帧尾报文的时间戳。
所述传输设备也可以在更新时考虑可能发生的帧间报文乱序和帧尾报文丢失问题。如果存在帧间报文乱序，即后一帧的非帧尾报文先于前一帧的帧尾报文到达或先于前一帧的帧尾报文被传输设备处理，或存在帧尾报文丢失，该视频流的最新帧尾报文的序列号也可以是根据序列号大于未被收到的帧尾报文的非帧尾报文估算的该帧尾报文的序列号。相应地，该传输设备可以采用如下方式进行更新。
在确定接收的该视频流的数据报文不是帧尾报文且所述数据报文的序列号大于该视频流的最新帧尾报文的序列号与该视频流的单个视频帧的报文个数M的和时，将记录的该视频流的最新帧尾报文的序列号增加M。如果该目标信息还包括时间戳，相应地，将记录的该视频流的最新帧尾报文的时间戳增加单个视频帧的时长T。在确定接收的该视频流的数据报文为帧尾报文时，可以直接执行更新操作，即将该视频流的最新帧尾报文的序列号更新为该数据报文的序列号，并将该视频流的最新帧尾报文的时间戳更新为该数据报文的时间戳，也可以进一步判断该数据报文的时间戳是否大于记录的该视频流的最新帧尾报文的时间戳，并仅在大于时执行该更新操作。
下面以一种具体实施方式对步骤S115的具体实现进行详细描述。
传输设备建立视频流切换表。如果在步骤S120之前执行步骤S115，该切换表可以包括该多个视频流中的每个视频流的记录，如果在步骤S120之后执行步骤S115，该切换表可以仅包括要切换的两个视频流(即第一视频流和第二视频流)的记录，如表1所示。
该切换表可以包括字段FT_TS和FT_SN，每个视频流对应的FT_TS的值和FT_SN的值分别为该视频流的最新帧尾报文的时间戳和序列号。
该视频流切换表还可以包括字段Flow、Outport、State、FrameTime和FramePackets中的任意一个或任意组合。Flow的值用于标识对应的视频流。每个视频流对应的Outport的值用于记录该视频流对应的出端口的端口号，即表示该视频流从该传输设备的哪个端口转发出去，本发明实施例对对应同一个出端口的视频流进行切换。每个视频流对应的State的值用于记录该视频流的当前状态，其中，SW_Forwarding表明该视频流为传输设备在转发的视频流，SW_Discard表明该视频流为该传输设备不通过对应的出端口转发的视频流。每个视频流对应的FrameTime的值用于记录该视频流的单个视频帧的时长，如果采用RTP进行视频流传输，FrameTime的单位与RTP头中的TS字段的单位相同。每个视频流对应的FramePackets的值用于记录该视频流的单个视频帧的报文个数。
该视频流切换表还可以包括SW_SN，每个视频流对应的SW_SN的值用于记录该视频流的切换序列号。
Outport Flow State FrameTime FramePackets FT_TS FT_SN
6 1 SW_Forwarding T1 M1 TS1 SN1
6 2 SW_Discard T2 M2 TS2 SN2
表1
后续将所述视频流切换表中的任一视频流称为视频流A。
对于视频流A，在接收到视频流A的任一数据报文(后续称为报文a)时，首先判断报文a是否为帧尾报文。
以RTP报文为例，根据RFC 3550制定的RTP报文头格式，RTP报文头中的M(Marker，边界)字段的值用于表示该RTP报文是否为帧尾报文。如果M的值为1，表示该RTP报文为帧尾报文，如果为0，表示该RTP报文不是帧尾报文。
RTP报文头中的序列号(Sequence number，SN)字段的值表示该RTP报文在一条RTP流中的序列号，同一条RTP流的各RTP报文的序列号单调递增1。RTP报文头还包括时间戳(Timestamp，TS)。多媒体通信系统(如TV制播系统)中的各视频源设备会保持时钟同步，同一时刻发出的RTP报文，该字段的值非常接近，因此，可以利用该字段的值来判断不同视频流中的视频帧是否是同一时刻的视频帧，用于在切换时对齐不同的视频流，达到切换在时间维度上的衔接。
如果确定报文a为帧尾报文，且是视频流A在顺序上的的第一个帧尾报文，将报文a的TS字段的值和SN字段的值分别记录到第一切换表中视频流A对应的FT_TS字段和FT_SN字段中。
如果确定报文a为帧尾报文，且不是视频流A在顺序上的的第一个帧尾报文，将报文a的TS字段的值和第一切换表中视频流A对应的FT_TS字段的值进行比较。如果报文a的TS字段的值大于第一切换表中视频流A对应的FT_TS字段的值，则报文a为最新帧尾报文，因此，将第一切换表中视频流A对应的FT_TS字段和FT_SN字段的值分别更新为报文a的TS字段和SN字段的值。如果小于，则不更新。
如果确定报文a不是帧尾报文，则判断是否发生了帧间乱序。报文在网络中传输时有可能发生乱序，报文所经过的传输设备的处理器对接收的报文进行多线程并发处理时也可能导致报文在处理上出现乱序。因此，一帧的非帧尾报文可能先于前一帧的帧尾报文到达传输设备或先于前一帧的帧尾报文被处理。如果一帧的报文先于前一帧的帧尾报文到达或先于前一帧的帧尾报文被处理，则表明发生了帧间报文乱序。
如果报文a不是帧尾报文，且报文a的序列号大于视频流A的最新帧尾报文的序列号与M的和，即报文a先于前一帧的帧尾报文到达了，则前一帧的帧尾报文可能发生了乱序或丢包，此时可以将FT_TS字段的值与T的和作为估计的该帧尾报文的时间戳，并将FT_SN字段的值和M的和作为估计的该帧尾报文的序列号，并利用估计的时间戳和序列号更新第一切换表视频流A对应的FT_TS字段和FT_SN字段的值，即将FT_TS字段的值增加T，并将FT_SN字段的值增加M。
根据本发明实施例1，本发明实施例2提出了一种传输设备200，如图5所示，传输设备200包括：处理单元210和通信单元220。
通信单元220，用于接收第一视频流和第二视频流，通过所述传输设备的目标出端口发送所述第一视频流，并接收切换指令，所述切换指令用于指示将通过所述目标出端口发送的 所述第一视频流切换为所述第二视频流。
处理单元210，用于根据所述切换指令确定所述第一视频流的第一基准帧尾报文的切换参考信息和所述第二视频流的第二基准帧尾报文的切换参考信息，根据所述第一基准帧尾报文的切换参考信息确定所述第一视频流的第一切换报文的切换参考信息，并根据所述第二基准帧尾报文的切换参考信息确定所述第二视频流的第二切换报文的切换参考信息。
通信单元220，还用于根据所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流。
本发明实施例2中描述的功能单元可以用来实施上述实施例1所述的方法中传输设备执行的操作。具体地，所述通信单元220用于与其它设备通信，如接收控制信令、接收数据报文、转发数据报文、在转发数据报文时修改数据报文中的信息等；所述处理单元210用于确定切入流和切出流的切换序列号等。
处理单元210可以包括多个处理子单元，分别对应一个需要切换视频流的出端口(如上述目标出端口)。通信单元220可以包括多个发送子单元，分别对应一个需要切换视频流的出端口。每个出端口的处理子单元确定该出端口的切入流和切出流的切换报文的切换参考信息(如序列号)，每个出端口对应的发送子单元通过该出端口转发需要通过该出端口转发的视频流的数据报文、并直接丢弃不需要通过该出端口转发的视频流的数据报文。
根据本发明实施例2提供的传输设备200，由于在视频流切换时考虑了切换前后发送的数据报文所处的位置，因此，可以有效减轻切换操作对传输的视频画面的损伤。
根据本发明实施例1，本发明实施例3提供了一种传输设备1000。如图6所示，传输设备1000包括处理器1010和端口1020。
处理器1010，用于实施本发明实施例1提供的方法100中的传输设备执行的操作，其中，在与其它设备通信时通过端口1020与其它设备通信。处理器1010具体可以是中央处理器(central processing unit，CPU)，或者是特定集成电路(Application Specific Integrated Circuit，ASIC)，或者是被配置成实施本发明实施例的一个或多个集成电路。
如图6所示，传输设备1000还可以包括存储器1030。处理器1010和存储器1030之间可以通过总线完成相互间的通信。存储器1030，用于存放计算机操作指令，具体可以是高速随机存取存储器(Random Access Memory，RAM)，也可以是非易失性存储器(non-volatile memory)。处理器1010具体可以执行存储器1030中存放的计算机操作指令，通过执行所述计算机操作指令使得传输设备1000执行上述实施例1所述的方法中传输设备所执行的操作。在具体实现时，存储器1030可以包括处理模块1031和通信模块1032，分别存储用于实现图5所示的处理单元210和通信单元220的功能。相应地，处理器1010具体通过执行处理模块1031中的指令实现处理单元210的功能并通过执行通信模块1032中的指令实现通信单元220的功能。
根据本发明实施例3提供的传输设备1000，由于在视频流切换时考虑了切换前后发送的数据报文所处的位置，因此，可以有效减轻切换操作对传输的视频画面的损伤。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算 机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。

Claims (22)

  1. 一种实现视频流切换的方法，其特征在于，包括：
    传输设备接收第一视频流和第二视频流；
    所述传输设备通过目标出端口发送所述第一视频流；
    所述传输设备接收切换指令，所述切换指令用于指示将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流；
    所述传输设备根据所述切换指令确定所述第一视频流的第一基准帧尾报文的切换参考信息和所述第二视频流的第二基准帧尾报文的切换参考信息；
    所述传输设备根据所述第一基准帧尾报文的切换参考信息确定所述第一视频流的第一切换报文的切换参考信息，并根据所述第二基准帧尾报文的切换参考信息确定所述第二视频流的第二切换报文的切换参考信息；
    所述传输设备根据所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流。
  2. 根据权利要求1所述的方法，其特征在于，
    所述第一基准帧尾报文和所述第二基准帧尾报文分别为确定所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息之前接收的所述第一视频流的倒数第x个帧尾报文和所述第二视频流的倒数第x个帧尾报文，其中x为自然数。
  3. 根据权利要求2所述的方法，其特征在于，x取值为1；
    所述第一切换报文中的视频数据所属视频帧为所述第一基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧；所述第二切换报文中的视频数据所属视频帧为所述第二基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧。
  4. 根据权利要求1-3任意一项所述的方法，其特征在于，
    所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均包括序列号。
  5. 根据权利要求1-4任意一项所述的方法，其特征在于，
    所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均包括时间戳。
  6. 根据权利要求4所述的方法，其特征在于，
    所述根据所述第一基准帧尾报文的序列号确定所述第一视频流的第一切换报文的序列号包括：根据所述第一基准帧尾报文的序列号、以及所述第一视频流和所述第二视频流之间的同步关系确定所述第一视频流的第一对齐帧尾报文的序列号，并将所述第一对齐帧尾报文的序列号作为所述第一切换报文的序列号，或将用于封装所述第一对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第一切换报文的序列号；
    所述根据所述第二基准帧尾报文的序列号确定所述第二视频流的第二切换报文的序列号包括：根据所述第二基准帧尾报文的序列号、以及所述同步关系确定所述第二视频流的第二对齐帧尾报文的序列号，并将所述第二对齐帧尾报文的序列号作为所述第二切换报文的序列号，或将用于封装所述第二对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第二切换报文的序列号，所述第一对齐帧尾报文和所述第二对齐帧尾报文为在时间上对应的帧尾报文。
  7. 根据权利要求6所述的方法，其特征在于，
    在确定所述同步关系为所述第一视频流早于所述第二视频流时，将所述第一基准帧尾报文的序列号与m*M1的和确定为所述第一对齐帧尾报文的序列号，并将所述第二基准帧尾报文的序列号与(m+1)*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，m为自然数；
    在确定所述同步关系为所述第一视频流晚于所述第二视频流时，将所述第一基准帧尾报文的序列号与(n+1)*M1的和确定为所述第一对齐帧尾报文的序列号，将所述第二基准帧尾报文的序列号与n*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，n为自然数；或
    在确定所述同步关系为所述第一视频流和所述第二视频流同步时，将所述第一基准帧尾报文的序列号与k*M1的和确定为所述第一对齐帧尾报文的序列号，将所述第二基准帧尾报文的序列号与k*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，k为自然数。
  8. 根据权利要求6或7所述的方法，其特征在于，所述传输设备具体根据所述第一基准帧尾报文的时间戳和所述第二基准帧尾报文的时间戳确定所述第一视频流和所述第二视频流的同步关系。
  9. 根据权利要求1-8任意一项所述的方法，其特征在于，所述传输设备通过所述目标出端口发出的第二视频流的数据报文中的第一字段的值与通过所述目标出端口发出的所述第一视频流的数据报文中的第一字段的值是相同的，所述第一字段为组播地址字段或载荷类型字段。
  10. 根据权利要求1-9任意一项所述的方法，其特征在于，所述传输设备通过所述目标出端口发出的所述第二视频流的数据报文的第二字段的值与通过所述目标出端口发出的所述第一视频流的数据报文中的第二字段的值是连续的，所述第二字段为序列号字段、帧号字段或时间戳字段。
  11. 一种传输设备，其特征在于，所述传输设备包括处理单元和通信单元；
    所述通信单元，用于接收第一视频流和第二视频流，通过所述传输设备的目标出端口发送所述第一视频流，并接收切换指令，所述切换指令用于指示将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流；
    所述处理单元，用于根据所述切换指令确定所述第一视频流的第一基准帧尾报文的切换参考信息和所述第二视频流的第二基准帧尾报文的切换参考信息，根据所述第一基准帧尾报文的切换参考信息确定所述第一视频流的第一切换报文的切换参考信息，并根据所述第二基准帧尾报文的切换参考信息确定所述第二视频流的第二切换报文的切换参考信息；
    所述通信单元，还用于根据所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息将通过所述目标出端口发送的所述第一视频流切换为所述第二视频流。
  12. 根据权利要求11所述的传输设备，其特征在于，
    所述第一基准帧尾报文和所述第二基准帧尾报文分别为确定所述第一基准帧尾报文的切换参考信息和所述第二基准帧尾报文的切换参考信息之前接收的所述第一视频流的倒数第x个帧尾报文和所述第二视频流的倒数第x个帧尾报文，其中x为自然数。
  13. 根据权利要求12所述的传输设备，其特征在于，x取值为1；
    所述第一切换报文中的视频数据所属视频帧为所述第一基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧；所述第二切换报文中的视频数据所属视频帧为所述第二基准帧尾报文中的视频数据所属视频帧在顺序上的下一视频帧或在顺序上的下一视频帧的后续视频帧。
  14. 根据权利要求11-13任意一项所述的方法，其特征在于，
    所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均包括序列号。
  15. 根据权利要求11-14任意一项所述的方法，其特征在于，
    所述第一基准帧尾报文的切换参考信息、所述第二基准帧尾报文的切换参考信息、所述第一切换报文的切换参考信息和所述第二切换报文的切换参考信息均包括时间戳。
  16. 根据权利要求14所述的传输设备，其特征在于，
    所述处理单元用于根据所述第一基准帧尾报文的序列号确定所述第一视频流的第一切换报文的序列号包括：用于根据所述第一基准帧尾报文的序列号、以及所述第一视频流和所述第二视频流之间的同步关系确定所述第一视频流的第一对齐帧尾报文的序列号，并将所述第一对齐帧尾报文的序列号作为所述第一切换报文的序列号，或将用于封装所述第一对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第一切换报文的序列号；
    所述处理单元用于根据所述第二基准帧尾报文的序列号确定所述第二视频流的第二切换报文的序列号包括：用于根据所述第二基准帧尾报文的序列号、以及所述同步关系确定所述第二视频流的第二对齐帧尾报文的序列号，并将所述第二对齐帧尾报文的序列号作为所述第二切换报文的序列号，或将用于封装所述第二对齐帧尾报文中的视频数据所属视频帧的下一视频帧的消隐区数据的数据报文的序列号确定为所述第二切换报文的序列号，所述第一对齐帧尾报文和所述第二对齐帧尾报文为在时间上对应的帧尾报文。
  17. 根据权利要求16所述的传输设备，其特征在于，所述处理单元具体用于：
    在确定所述同步关系为所述第一视频流早于所述第二视频流时，将所述第一基准帧尾报文的序列号与m*M1的和确定为所述第一对齐帧尾报文的序列号，并将所述第二基准帧尾报文的序列号与(m+1)*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，m为自然数；
    在确定所述同步关系为所述第一视频流晚于所述第二视频流时，将所述第一基准帧尾报文的序列号与(n+1)*M1的和确定为所述第一对齐帧尾报文的序列号，将所述第二基准帧尾报文的序列号与n*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，n为自然数；或
    在确定所述同步关系为所述第一视频流和所述第二视频流同步时，将所述第一基准帧尾报文的序列号与k*M1的和确定为所述第一对齐帧尾报文的序列号，将所述第二基准帧尾报文的序列号与k*M2的和确定为所述第二对齐帧尾报文的序列号，其中，M1为封装所述第一视频流的一个视频帧所需的数据报文的数量，M2为封装所述第二视频流的一个视频帧所需的数据报文的数量，k为自然数。
  18. 根据权利要求16或17所述的传输设备，其特征在于，所述处理单元具体根据所述第一基准帧尾报文的时间戳和所述第二基准帧尾报文的时间戳确定所述第一视频流和所述第二视 频流的同步关系。
  19. 根据权利要求11-18任意一项所述的传输设备，其特征在于，所述通信单元通过所述目标出端口发出的第二视频流的数据报文中的第一字段的值与通过所述目标出端口发出的所述第一视频流的数据报文中的第一字段的值是相同的，所述第一字段为组播地址字段或载荷类型字段。
  20. 根据权利要求11-19任意一项所述的传输设备，其特征在于，所述通信单元通过所述目标出端口发出的所述第二视频流的数据报文的第二字段的值与通过所述目标出端口发出的所述第一视频流的数据报文中的第二字段的值是连续的，所述第二字段为序列号字段、帧号字段或时间戳字段。
  21. 一种实现视频流切换的系统，其特征在于，所述系统包括：控制器和如权利要求11-20任意一项所述的传输设备；
    所述控制器，用于向所述传输设备发送切换指令。
  22. 一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1-10任意一项所述的方法。
PCT/CN2019/101241 2018-08-22 2019-08-18 一种实现视频流切换的方法、装置和系统 WO2020038309A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
KR1020237010789A KR102671249B1 (ko) 2018-08-22 2019-08-18 비디오 스트림을 스위칭하기 위한 방법, 장치, 및 시스템
JP2021509807A JP7224439B2 (ja) 2018-08-22 2019-08-18 ビデオ・ストリーム切り換えを実装するための方法、装置およびシステム
KR1020217006921A KR20210041052A (ko) 2018-08-22 2019-08-18 비디오 스트림을 스위칭하기 위한 방법, 장치, 및 시스템
EP19851740.1A EP3836552A4 (en) 2018-08-22 2019-08-18 VIDEO STREAM SWITCHING PROCESS, APPARATUS AND SYSTEM
US17/180,568 US11483495B2 (en) 2018-08-22 2021-02-19 Method, apparatus, and system for implementing video stream switching

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810962907.4 2018-08-22
CN201810962907.4A CN110858925B (zh) 2018-08-22 2018-08-22 一种实现视频流切换的方法、设备、系统和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/180,568 Continuation US11483495B2 (en) 2018-08-22 2021-02-19 Method, apparatus, and system for implementing video stream switching

Publications (1)

Publication Number Publication Date
WO2020038309A1 true WO2020038309A1 (zh) 2020-02-27

Family

ID=69592280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/101241 WO2020038309A1 (zh) 2018-08-22 2019-08-18 一种实现视频流切换的方法、装置和系统

Country Status (6)

Country Link
US (1) US11483495B2 (zh)
EP (1) EP3836552A4 (zh)
JP (1) JP7224439B2 (zh)
KR (2) KR102671249B1 (zh)
CN (2) CN110858925B (zh)
WO (1) WO2020038309A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114338456B (zh) * 2020-09-25 2024-03-22 浙江宇视科技有限公司 图像采集设备状态确定方法、装置、电子设备及介质
CN113794942B (zh) * 2021-09-09 2022-12-02 北京字节跳动网络技术有限公司 自由视角视频的视角切换方法、装置、系统、设备和介质
CN115314643B (zh) * 2022-07-28 2023-12-15 中央广播电视总台 一种实现净切换的方法、系统、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090097496A1 (en) * 2006-05-02 2009-04-16 Kazunori Nakamura Video Signal Switching System
CN102341859A (zh) * 2009-03-05 2012-02-01 诺基亚公司 来自多个内容源的内容的同步
CN102833584A (zh) * 2012-05-22 2012-12-19 中华电信股份有限公司 多视角互动电视的影音串流方法
CN103907358A (zh) * 2011-06-22 2014-07-02 无线电广播技术研究所有限公司 用于切换实时媒体流的设备和方法
CN105554532A (zh) * 2016-02-15 2016-05-04 北京中传数广技术有限公司 一种直播电视精准广告、信息投放的方法及系统
CN107888909A (zh) * 2017-11-29 2018-04-06 中国传媒大学 一种ip视频净切换系统的测量方法

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4474310B2 (ja) * 2005-03-24 2010-06-02 パナソニック株式会社 デバイス切替システム、送信元および配信先デバイス
CN1331359C (zh) 2005-06-28 2007-08-08 清华大学 交互式多视点视频系统中视频流的传输方法
US20080021971A1 (en) * 2006-07-21 2008-01-24 Halgas Joseph F System and Method for Electronic Messaging Notification Using End-User Display Devices
CN100559877C (zh) 2007-04-27 2009-11-11 北京大学 一种支持多视点视频合成的网络流媒体播放器及方法
JP2009100411A (ja) * 2007-10-19 2009-05-07 Hitachi Ltd 映像配信システム、映像中継装置、及び映像中継方法
US20090106807A1 (en) * 2007-10-19 2009-04-23 Hitachi, Ltd. Video Distribution System for Switching Video Streams
CN101316357B (zh) * 2008-06-30 2011-09-14 华为技术有限公司 一种频道切换的方法和终端
CN102119519A (zh) 2008-08-12 2011-07-06 爱立信(中国)通信有限公司 在通信系统中的快速内容切换
CN101753973B (zh) * 2008-12-12 2013-01-02 华为技术有限公司 一种频道切换方法、装置和系统
JP5428734B2 (ja) * 2009-10-14 2014-02-26 ソニー株式会社 ネットワーク機器、情報処理装置、ストリーム切替方法、情報処理方法、プログラムおよびコンテンツ配信システム
CN101917390B (zh) 2009-12-17 2013-03-06 新奥特（北京）视频技术有限公司 一种多功能网络播放器
EP2343845A3 (en) * 2009-12-31 2011-10-19 Thomson Licensing Precise compensation of video propagation duration
CN101854533B (zh) * 2010-06-10 2012-05-23 华为技术有限公司 频道切换方法、装置及系统
EP2448265A1 (en) 2010-10-26 2012-05-02 Google, Inc. Lip synchronization in a video conference
US20130301830A1 (en) 2012-05-08 2013-11-14 Hagai Bar-El Device, system, and method of secure entry and handling of passwords
US9860458B2 (en) * 2013-06-19 2018-01-02 Electronics And Telecommunications Research Institute Method, apparatus, and system for switching transport stream
GB2516316A (en) * 2013-07-19 2015-01-21 Sony Corp Video network
WO2016084592A1 (ja) * 2014-11-28 2016-06-02 ソニー株式会社 送信装置、送信方法、受信装置および受信方法
CN107431845B (zh) 2015-03-31 2021-03-26 松下电器（美国）知识产权公司 发送方法、接收方法、发送装置以及接收装置
US10038651B2 (en) * 2015-09-05 2018-07-31 Nevion Europe As Asynchronous switching system and method
EP3185564A1 (en) * 2015-12-22 2017-06-28 Harmonic Inc. Video stream splicing of groups of pictures (gop)
CN106937155B (zh) * 2015-12-29 2020-06-02 北京华为数字技术有限公司 接入设备、因特网协议电视iptv系统和频道切换方法
CN106162235B (zh) 2016-08-17 2018-06-01 北京百度网讯科技有限公司 用于切换视频流的方法和装置
CN107770538B (zh) 2016-08-23 2020-09-11 华为技术有限公司 一种检测场景切换帧的方法、装置和系统
US10412441B2 (en) * 2016-12-06 2019-09-10 Rgb Spectrum Systems, methods, and devices for high-bandwidth digital content synchronization
CN107086963B (zh) * 2017-03-10 2018-10-09 中国传媒大学 一种基于sdn的目的端精准视频切换方法
CN107135169B (zh) * 2017-04-25 2018-08-28 中国传媒大学 一种基于sdn交换机的视频净切换方法
CN107018379B (zh) 2017-04-25 2019-07-23 北京东土科技股份有限公司 一种视频流的传输方法和装置
CN110891182B (zh) * 2018-09-11 2022-04-12 华为技术有限公司 一种实现视频流切换的方法、装置和系统
CN111629158B (zh) * 2019-02-28 2021-08-03 华为技术有限公司 一种音频流和视频流同步切换方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090097496A1 (en) * 2006-05-02 2009-04-16 Kazunori Nakamura Video Signal Switching System
CN102341859A (zh) * 2009-03-05 2012-02-01 诺基亚公司 来自多个内容源的内容的同步
CN103907358A (zh) * 2011-06-22 2014-07-02 无线电广播技术研究所有限公司 用于切换实时媒体流的设备和方法
CN102833584A (zh) * 2012-05-22 2012-12-19 中华电信股份有限公司 多视角互动电视的影音串流方法
CN105554532A (zh) * 2016-02-15 2016-05-04 北京中传数广技术有限公司 一种直播电视精准广告、信息投放的方法及系统
CN107888909A (zh) * 2017-11-29 2018-04-06 中国传媒大学 一种ip视频净切换系统的测量方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3836552A4 *

Also Published As

Publication number Publication date
KR102671249B1 (ko) 2024-05-31
EP3836552A4 (en) 2021-08-18
KR20230047512A (ko) 2023-04-07
US11483495B2 (en) 2022-10-25
CN110858925B (zh) 2021-10-15
CN110858925A (zh) 2020-03-03
JP2021535658A (ja) 2021-12-16
EP3836552A1 (en) 2021-06-16
US20210176408A1 (en) 2021-06-10
KR20210041052A (ko) 2021-04-14
CN114143606A (zh) 2022-03-04
JP7224439B2 (ja) 2023-02-17

Similar Documents

Publication Publication Date Title
WO2020052456A1 (zh) 一种实现视频流切换的方法、装置和系统
US11483495B2 (en) Method, apparatus, and system for implementing video stream switching
US11184627B2 (en) Video transcoding system, method, apparatus, and storage medium
JP5788473B2 (ja) 端末の出力を同期させる方法およびシステム
US20210352345A1 (en) Video distribution synchronization
WO2020173165A1 (zh) 一种音频流和视频流同步切换方法及装置
US20120117265A1 (en) Method and communication system for implementing stream services, and relevant device
US20060161676A1 (en) Apparatus for IP streaming capable of smoothing multimedia stream
US9647951B2 (en) Media stream rate reconstruction system and method
CN113645491A (zh) 一种多个直播播放端实现实时同步播放的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19851740

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021509807

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019851740

Country of ref document: EP

Effective date: 20210310