WO2023160361A1 - Rtc数据的处理方法以及装置 - Google Patents

Rtc数据的处理方法以及装置 Download PDF

Info

Publication number
WO2023160361A1
WO2023160361A1 PCT/CN2023/074514 CN2023074514W WO2023160361A1 WO 2023160361 A1 WO2023160361 A1 WO 2023160361A1 CN 2023074514 W CN2023074514 W CN 2023074514W WO 2023160361 A1 WO2023160361 A1 WO 2023160361A1
Authority
WO
WIPO (PCT)
Prior art keywords
data stream
server
media data
media
user terminal
Prior art date
Application number
PCT/CN2023/074514
Other languages
English (en)
French (fr)
Inventor
卢日
肖凯
陈鹏
Original Assignee
阿里巴巴（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴（中国）有限公司 filed Critical 阿里巴巴（中国）有限公司
Publication of WO2023160361A1 publication Critical patent/WO2023160361A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1069—Session establishment or de-establishment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS

Definitions

  • the present application relates to the field of network technologies, and in particular to a method and device for processing RTC data.
  • the embodiment of the present application provides a method and device for processing RTC data, in order to shorten the time of the first frame, and further increase the broadcasting speed of the client.
  • an embodiment of the present application provides a method for processing RTC data, including: a user terminal sends an RTC session request to a first server, and the RTC session request carries streaming information of a target media data stream; On the command channel, receive the RTC session response sent by the first server and the first data stream in the target media data stream, the RTC session response carries the interface information of the media service unit, and the interface information of the media service unit is used to establish A media data channel, where the media data channel is used to transmit a second data stream, and the second data stream includes data streams in the target media data stream except the first data stream; the user terminal plays the first data stream.
  • the embodiment of the present application provides an RTC data transmission method, including: the first server receives the RTC session request sent by the user terminal, and the RTC session request carries the streaming information of the target media data stream; On the signaling channel, send an RTC session response and the first data stream in the target media data stream to the user terminal, where the RTC session response carries a media service unit Interface information of the media service unit, the interface information of the media service unit is used to establish a media data channel, and the media data channel is used to transmit the second data stream, and the second data stream includes the target media data stream except the first data stream data flow.
  • an embodiment of the present application provides an apparatus for processing RTC data, including: a signaling unit configured to send an RTC session request to a first server, where the RTC session request carries streaming information of a target media data stream;
  • the signaling unit is also used to receive the RTC session response sent by the first server and the first data stream in the target media data stream on the signaling channel, the RTC session response carries the interface information of the media service unit, and the media
  • the interface information of the service unit is used to establish a media data channel, and the media data channel is used to transmit a second data stream, and the second data stream includes a data stream in the target media data stream except the first data stream; the media unit , for playing the first data stream.
  • an embodiment of the present application provides an apparatus for processing RTC data, including: a signaling service unit configured to receive an RTC session request sent by a user terminal, where the RTC session request carries streaming information of a target media data stream; The signaling service unit is further configured to send an RTC session response and the first data stream in the target media data stream to the user terminal on a signaling channel, the RTC session response carries interface information of the media service unit, and the media The interface information of the service unit is used to establish a media data channel, and the media data channel is used to transmit a second data stream, and the second data stream includes a data stream in the target media data stream except the first data stream.
  • an embodiment of the present application provides an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor Execute the method provided in the first aspect or the second aspect.
  • the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the implementation as provided in the first aspect or the second aspect is realized.
  • an embodiment of the present application provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, implement the first aspect or the second aspect or the provided method.
  • the first server sends the first data stream containing the target media data stream on the signaling channel, so that the first data stream in the target media data stream A data stream is sent to the user terminal in advance without waiting for the establishment of the media data channel, which shortens the time for the first frame of the media data stream to be played, thereby increasing the start-up speed of the client.
  • Fig. 1 is a schematic diagram of a data transmission system provided by the present application.
  • FIG. 2 is a schematic diagram of an interaction flow of a method for processing RTC data provided in an embodiment of the present application
  • FIG. 3 is a schematic diagram of an interaction flow of a method for processing RTC data provided in an embodiment of the present application
  • FIG. 4 is a schematic block diagram of an RTC data processing device provided in an embodiment of the present application.
  • FIG. 5 is a schematic block diagram of an RTC data processing device provided in an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a terminal device provided in an exemplary embodiment of the present application.
  • Fig. 7 is a schematic structural diagram of a server provided in an exemplary embodiment of the present application.
  • This application is applicable to any RTC audio/video data transmission scenario, such as Internet telephony, conference call, video chat, live broadcast, etc., and is especially suitable for low-latency live broadcast (Real-time Streaming, RTS) scenarios.
  • RTS Real-time Streaming
  • the audio/video data transmitted in this application can be a kind of data stream (data stream) (hereinafter referred to as media data stream), which can be understood as a group of ordered, with a starting point and an end point.
  • data stream data stream
  • media data stream a kind of data stream (hereinafter referred to as media data stream), which can be understood as a group of ordered, with a starting point and an end point.
  • FIG. 1 is a schematic diagram of a data transmission system provided by the present application.
  • the data transmission system 100 includes a playback client 110 , a server 120 and an anchor client 130 .
  • the server 120 communicates with the player client 110 and the anchor client 130 respectively.
  • the anchor client 130 sends the recorded data stream to the server 120.
  • This process is also called the streaming process.
  • the anchor client 130 can continuously send real-time data streams to the server 120 during audio/video recording; End 110 acquires the media data stream stored by server 120 from server 120.
  • This process is also referred to as the streaming process.
  • the media data stream may be a part of the data stream being recorded by anchor client 130 that has been sent to server 120. data. Based on this, the audio/video recorded in real time in the anchor client 130 is transmitted to the playback client 110 through the server 120, and rendered and played in the playback client 110 to realize live broadcast.
  • the above playback client 110, server 120 and anchor client 130 can all be deployed with web real-time communication (Web Real-Time Communication, WebRTC) to support web browsers for real-time audio/video data transmission and realize the above RTS.
  • Web Real-Time Communication Web Real-Time Communication
  • the above-mentioned playback client 110 and anchor client 130 can all be implemented as any terminal equipment, or be called a user terminal, such as an ordinary computer PC, mobile phone (mobile phone), tablet computer (pad), smart wearable device, with wireless transceiver function Computers, virtual reality (virtual reality, VR) terminal equipment, augmented reality (augmented reality, AR) terminal equipment, wireless terminals in industrial control (industrial control), wireless terminals in self driving (self driving), remote surgery Wireless terminals in remote medical surgery, wireless terminals in smart grid, wireless terminals in transportation safety, wireless terminals in smart city, smart home wireless terminals, etc.
  • a user terminal such as an ordinary computer PC, mobile phone (mobile phone), tablet computer (pad), smart wearable device, with wireless transceiver function Computers, virtual reality (virtual reality, VR) terminal equipment, augmented reality (augmented reality, AR) terminal equipment, wireless terminals in industrial control (industrial control), wireless terminals in self driving (self driving), remote surgery Wireless terminals in remote medical surgery, wireless terminals in smart grid, wireless terminals in
  • the above-mentioned server 120 may be implemented as a common server, a server cluster, or a cloud server or a server cluster.
  • the server 120 can be used to implement signaling service, media service, Network Address Translation (Network Address Translation, NAT) session traversal application (Session Traversal Utilities for NAT, STUN) service and NAT relay traversal (Traversal Using Relays around NAT) , TURN) part or all of the service.
  • the above-mentioned services can be implemented as independent servers, for example, it can include part or all of the signaling server 121, media server 122, and STUN/TURN server 123 shown in FIG.
  • the TURN server can generally implement STUN service and TURN service; in some other embodiments, some or all of the above services may be integrated in the same server in the form of service units, for example, the first server hereinafter may include a signaling service unit and a media service unit.
  • the signaling service can be used to realize the signaling interaction between the playback client 110 and the server 120;
  • the media service can be used to receive the media data stream sent by the anchor client 130, and send 110 provides media data flow;
  • STUN service is used to detect whether there is NAT around the playback client 110;
  • TURN service is used to penetrate the NAT around the playback client 110, and realize the establishment of a media data channel between the playback client 110 and the server 120 .
  • a media data channel is usually established between the playback client 110 and the server 120, for example, a WebRTC media data channel is established to implement WebRTC standard access to streaming playback.
  • a media data channel may include a signaling phase and an Interactive Connectivity Establishment (ICE) phase.
  • ICE Interactive Connectivity Establishment
  • the playback client 110 initiates a stream pull request to the server 120, and the playback client 110 and the service
  • the terminals 120 negotiate the metadata of the media data stream to be transmitted, and establish a media data channel based on STUN and/or TURN in the ICE phase, and then transmit the media data stream in the media data channel.
  • the player Client 110 is from initiating the stream pull request to obtaining the first audio/video frame (the same as the first frame data in the following) of the media data stream (the same as the target media data stream in the following), and performing the first audio/video frame Playing takes a long time, that is, the first frame takes a long time, which reduces the start-up speed of the playback client.
  • the first audio/video frame the same as the first frame data in the following
  • the media data stream the same as the target media data stream in the following
  • the embodiment of the present application considers pulling the first data stream in the target media data in the above-mentioned signaling phase, so as to shorten the time of the first frame, and further increase the speed of starting the broadcast of the client.
  • the first frame data is the first audio/video frame of the target media data stream received by the client after requesting to pull the target media data stream
  • the first frame data may not be the first audio/video frame of the target media data stream frame.
  • the target media data stream is multiple audio/video frames continuously recorded by the anchor client
  • the first frame data is the target media data stream that has been pushed from the playback client after the server receives the RTC session request sent by the playback client.
  • the audio/video frame acquired in the preset time period at the current moment the audio/video frame may be the nth audio/video frame in the target media data stream, and n is a positive integer.
  • the audio/video frame is used as the first frame data, and the data stream after the first frame data is continuously acquired.
  • the user terminal may be, for example, the playback client 110 in FIG. 1
  • the first server may be, for example, the server 120 in FIG. 1 or a server in the server 120
  • the first server may be the signaling server 121 in FIG. 1 , or a server deployed with a signaling service.
  • the first server is also deployed with a media service unit.
  • the media service unit is deployed on the second server, such as the media server 122 in FIG. 1 .
  • the first server and the user terminal also have data interaction with the second server respectively.
  • the first server and the user terminal also have data interaction with a third server, for example, the third server may be the STUN/TURN server 123 in FIG. 1 .
  • FIG. 2 is a schematic diagram of an interaction flow of a method for processing RTC data provided by an embodiment of the present application. As shown in Figure 2, the method includes:
  • the user terminal sends an RTC session request to the first server, and the RTC session request carries streaming information of a target media data stream; correspondingly, the first server receives the RTC session request sent by the user terminal.
  • the first server sends the RTC session response and the first data stream in the target media data stream to the user terminal on the signaling channel
  • the RTC session response carries the interface information of the media service unit
  • the media service unit The interface information is used to establish a media data channel
  • the media data channel is used to transmit a second data stream
  • the second data stream includes data streams other than the first data stream in the target media data stream; correspondingly, the user terminal Receive the RTC session response sent by the first server and the first data stream in the target media data stream on the command channel.
  • the user terminal plays the first data stream.
  • the above-mentioned RTC session request carries at least stream-pull information for obtaining the target media data flow
  • the stream-pull information includes, for example, a Uniform Resource Identifier (Uniform Resource Identifier, URL), domain name, Internet Protocol (Internet Protocol, IP) of the target media data stream. ) at least one of the addresses.
  • the first server can determine the target media data stream required by the user terminal.
  • target media data stream is the media data stream that needs to be pulled by the user terminal, and does not specifically refer to a certain type of media data stream.
  • the first server in response to the RTC session request, sends an RTC session response to the user terminal through a signaling channel, and the RTC session response carries the interface information of the media service unit, so that the user terminal and the server deploying the media service unit A media data channel can be established between them, and then the target media data can be sent to the user terminal.
  • the interface information of the media service unit includes but not limited to the IP address and/or port of the media service unit.
  • the media service unit can be deployed on the first server or the second server.
  • the interface information of the media service unit can be the IP address and/or port of the first server; similarly, when the media service unit is deployed on the second server, the interface information of the media service unit It may be the IP address and/or port of the second server.
  • the first server also transmits the first data stream in the target media data stream through the signaling channel, so that the first server does not need to wait for the establishment of the media data channel Completed, that is, sending the first data stream of the target media data to the user terminal.
  • the first server may obtain the first data stream in the target media data stream stored by itself based on the pull stream information, for example, the first server pulls the first data stream from a media service unit deployed in the first server; or, the first server may The first data stream in the target media data stream is acquired from the second server based on the streaming information, for example, the first server sends the streaming information to the second server and receives the first data stream sent by the second server.
  • the timing of the first data stream in the target media data stream is earlier than the timing of the second data stream in the target media data stream.
  • the first data stream includes a section of data stream starting from the first frame data in the target media data stream
  • the second data stream includes a section of data stream starting from the frame data at the end position of the first data stream in the target media data.
  • the frame data at the end position of the first data stream and the frame data at the start position of the second data stream are adjacent frame data in the target media data stream, that is, the first data stream and the second data stream have continuity
  • the present application does not exclude the situation that the first data stream and the second data stream are discontinuous or overlapped.
  • the embodiment of the present application does not limit the duration of the first data stream.
  • the first data stream may include a data stream with a preset duration starting from the first frame data, and the preset duration may be positively correlated with the duration of establishing a media data channel; as another example, the first data stream includes In the target media data stream, from the first frame of data to when the media data channel is established, the data stream has been sent on the signaling channel.
  • the above-mentioned S220 can be specifically implemented as: The data stream starting from the first frame data in the target media data stream is sent to the user terminal until the establishment of the media data channel is completed.
  • the first data stream transmitted through the signaling channel and other data streams in the target media data stream switched to the media data channel can be better connected, reducing the number of playing cards. Ton phenomenon.
  • the first data stream may include all data streams in the target media data stream.
  • the first server may send the target media data stream in a signaling channel when the total duration (or data size) of the target media data stream is less than a preset value.
  • both the signaling channel and the media data channel are logical transmission channels, which are related to the stage of transmission and the content of transmission.
  • the signaling channel is used to represent the channel for signaling interaction between the user terminal and the first server in the signaling phase;
  • the media data channel is used to represent the channel for transmitting media data between the user terminal and the first server (or second server) in the streaming phase , such as a WebRTC-based media data channel.
  • the user terminal may send the RTC session request to the first server through a signaling channel.
  • Both the RTC session request in S210 and the session response in S220 may be signaling. It should be understood that signaling is used to coordinate communication, for example, a WebRTC application can establish a session through signaling interaction.
  • the RTC session request may be implemented as a session description protocol (Session Description Protocol, SDP) request (offer), and the RTC session response may be implemented as an SDP response (answer).
  • SDP offer and SDP answer can be used for two session entities (such as user terminal and first server) Negotiations between the media.
  • the RTC session request is also used to request the first server and the user terminal to negotiate the metadata of the media data transmitted on the media data channel.
  • metadata is defined as: data describing data, descriptive information on data and information resources, and metadata of media data refers to data describing media data.
  • the metadata includes, for example, at least one of codec settings, media format, transmission bandwidth, etc. of the media data. The negotiation process will be described in Figure 3 below.
  • the user terminal renders the received first data stream, and first displays the rendered picture on the display interface of the user terminal; if the first data stream is audio data stream, the user terminal plays the first data stream, and optionally, the user terminal displays an audio playback control interface when playing the first data stream.
  • the first server sends the first data stream containing the target media data stream on the signaling channel, so that the first data stream in the target media data stream
  • the data stream is sent to the user terminal in advance without waiting for the establishment of the media data channel, which shortens the time for the first frame of media data stream playback, thereby increasing the speed of starting the broadcast at the client.
  • the media data channel is conducive to providing the transmission quality of the media data and the real-time performance of the media data stream transmission.
  • the template media data can be switched to the media data channel for transmission, so as to further improve the quality and real-time performance of data transmission.
  • the method also includes some or all of the processes in S240 to S260 as shown in Figure 2:
  • the user terminal When playing the first data stream, the user terminal establishes a media data channel between the user terminal and the media service unit according to the interface information and the STUN of the media service unit.
  • the first server sends the second data stream to the user terminal on the media data channel through the media service unit; correspondingly, the user terminal receives the second data stream sent by the media service unit on the media data channel.
  • the user terminal plays the second data stream.
  • the user terminal may start the process of establishing the media data channel during the process of playing the first data stream, for example, during the process of rendering the first data stream.
  • the above S240 and the above S230 may be executed synchronously.
  • the user terminal may use the interface information of the media service unit as a candidate address, and perform NAT detection through the STUN service unit (which may be implemented as a STUN/TURN server).
  • STUN service unit which may be implemented as a STUN/TURN server.
  • the TURN service unit (which can be implemented as a TURN server) establishes a media data channel with the media service unit, in other words, establishes a media data channel between the user terminal and the media service unit through the relay service provided by TURN. Further description will be made in the embodiment described in FIG. 3 below.
  • the first server switches the data streams in the target media data that are not sent through the signaling channel to the media data channel for transmission.
  • the switching process can be understood as that the first server sends the data streams according to the transmission protocol of the media data channel, For example, data streams in the target media data that are not sent through the signaling channel are encapsulated according to the Real-time Transport Protocol (RTP) before being sent.
  • RTP Real-time Transport Protocol
  • the first server sends the data stream of the target media data not sent through the signaling channel to the TURN server, and then the TURN server forwards it to the user terminal.
  • the second data includes part or all of other data streams in the target media data stream except the first data stream; or the second data may also include data streams other than the target media data stream, such as the first
  • the data stream includes all data streams in the target media data stream, so the target media data stream does not need to be transmitted in the media data channel. In this case, the second data does not include the data stream in the target media data stream.
  • the user terminal plays the second data stream, and its implementation is similar to that of playing the first data stream, which will not be repeated here.
  • the user terminal including a media unit and signaling unit, the first server including a signaling service unit and a media service unit, and the third server including a STUN/TURN service unit as examples.
  • the user terminal may not distinguish between the media unit and the signaling unit, and for example, the media service unit may be deployed on the second server.
  • FIG. 3 is a schematic diagram of an interaction flow of a method for processing RTC data provided by an embodiment of the present application. As shown in Figure 3, the method includes part or all of the following processes from S301 to S311:
  • the signaling unit sends an RTC session request to the signaling service unit, and the RTC session request carries stream pulling information of the target media data stream;
  • the signaling service unit sends streaming information to the media service unit
  • the signaling service unit performs signaling negotiation
  • the media service unit sends the first data stream to the signaling service unit
  • the signaling service unit sends the RTC session response and the first data stream on the signaling channel;
  • the signaling unit sends the first data stream to the media unit
  • the media unit plays the first data stream
  • the signaling unit sends a STUN binding (binfing) offer to the STUN/TURN service unit;
  • the STUN/TURN service unit sends a STUN binding response to the signaling unit
  • the media service unit sends the second data stream to the media unit on the media data channel
  • the media unit plays the second data stream.
  • FIG. 3 Some of the above steps are similar to the corresponding steps in FIG. 2 and have the same or similar implementation methods, the difference is that the execution subject in FIG. 3 is embodied as a unit in the user terminal or the first server, for example, S301, S305, S307, S310 and S311. Therefore, the implementation manner thereof will not be described in detail.
  • the signaling service unit sends streaming information to the media service unit to obtain the first data stream sent by the media service unit; in S303, signaling negotiation is performed based on the RTC session request; and then the signaling service unit according to the negotiation An RTC session response is generated as a result, and the RTC session response and the acquired first data stream are transmitted to the signaling unit through the signaling channel.
  • the signaling negotiation of S303 mainly includes the negotiation of the metadata of the media data.
  • the metadata of the media data may include at least one of codec/decoder settings, media format, transmission bandwidth, etc. of the media data.
  • the first server such as a signaling service unit
  • the first server can determine the metadata of the media data supported by the user terminal Metadata of the media data transmitted on the media data channel.
  • the metadata of the media data transmitted on the media data channel determined by the first server on the one hand, should be the metadata of the media data supported by the user terminal, and on the other hand, should be the metadata that the media data provided by the media service unit can satisfy .
  • the RTC session response may indicate the negotiation The media format of the subsequent media data is AVI.
  • the media format supported by the user terminal indicated by the RTC session request includes AVI and MPEG
  • the first server determines that the format of the requested target media data stream for the RTC session request is MOV;
  • the metadata of the supported media data is used to transmit the media data, or the first server has the ability to convert MOV to AVI, then the first server indicates that the media format of the negotiated media data is AVI through the RTC session response.
  • the above S302 and S304 are the internal implementation process of the first server. If the signaling service unit is deployed on the first server, the media service unit If the body service unit is deployed on the second server, the above S302 and S304 are the interaction process between the first server and the second server.
  • the above S308 and S309 can realize the establishment of the media data channel.
  • ICE establishment may include a NAT detection phase and a hole punching phase.
  • the above S308 and S309 may be used to implement NAT detection in the NAT detection phase.
  • the STUN binding offer carries the IP address and port of the user terminal
  • the STUN/TURN service After the unit receives the STUN binding offer, it obtains the IP address and port of the sender, and compares the IP address and port of the sender with the IP address and port carried in the STUN binding offer. If they are consistent, it indicates that there is no NAT device in front of the user terminal. If not, it indicates that there is a NAT device in front of the user terminal.
  • a relay service can be provided through the TURN service unit to establish a media data channel between the media unit and the media service unit.
  • the media service unit switches the target media data from the signaling channel to the media data channel to continue sending. This process has been described in the embodiment shown in FIG. 2 and will not be repeated here.
  • FIG. 4 is a schematic block diagram of an apparatus for processing RTC data provided by an embodiment of the present application.
  • the apparatus 400 for processing RTC data may include a signaling unit 410 and a media unit 420 .
  • the RTC data processing device 400 may correspond to the user terminal in the above method embodiment, for example, it may be an implementation of the aforementioned playback client, or a component (such as a chip or chip system) configured in the playback client wait).
  • the signaling unit 410 can be used to send an RTC session request to the first server, and the RTC session request carries the streaming information of the target media data stream; the transceiver unit 410 is also used to receive the first The RTC session response sent by the server and the first data stream in the target media data stream, the RTC session response carries the interface information of the media service unit, the interface information of the media service unit is used to establish a media data channel, and the media data channel It is used to transmit the second data stream; the media unit 420 can be used to play the first data stream.
  • the timing of the first data stream in the target media data stream is earlier than the timing of the second data stream in the target media data stream.
  • the signaling unit 410 is further configured to, when playing the first data stream, according to the media
  • the interface information of the body service unit and the network address translation session pass through the application program STUN to establish a media data channel between the user terminal and the media service unit; the media unit 420 is also used to receive the media service unit on the media data channel.
  • the second data stream sent; the media unit 420 is also used to play the second data stream.
  • the first data stream includes a data stream of a preset duration starting from the first frame of data in the target media data stream.
  • the first data stream includes the data stream that has been sent on the signaling channel from the first frame data in the target media data stream to when the media data channel is established; the signaling unit 410 specifically uses On the signaling channel, continue to receive the data stream starting from the first frame data in the target media data stream sent by the first server until the media data channel is established.
  • the signaling unit 410 is specifically configured to send the RTC session request to the signaling service unit of the first server.
  • the media service unit is included in the first server or the second server, and the first server and the second server are independent of each other.
  • FIG. 5 is a schematic block diagram of an apparatus for processing RTC data provided by an embodiment of the present application.
  • the apparatus 500 for processing RTC data may include a signaling service unit 510 and a media service unit 520 .
  • the RTC data processing apparatus 500 may correspond to the first server in the above method embodiments, for example, may be the first server, or a component (such as a chip or a chip system, etc.) configured in the first server.
  • the signaling service unit 510 can be used to receive the RTC session request sent by the user terminal, and the RTC session request carries the streaming information of the target media data stream; the signaling service unit 510 is also used to send The user terminal sends an RTC session response and the first data stream in the target media data stream, the RTC session response carries interface information of a media service unit, the interface information of the media service unit is used to establish a media data channel, and the media data The channel is used to transmit the second data stream.
  • the timing of the first data stream in the target media data stream is earlier than the timing of the second data stream in the target media data stream.
  • the media service unit 520 sends a message to the user terminal on the media data channel The end sends the second data stream.
  • the first data stream includes a data stream of a preset duration starting from the first frame of data in the target media data stream.
  • the first data stream includes the data stream that has been sent on the signaling channel from the first frame of data in the target media data stream to when the media data channel is established; the media service unit 510 specifically uses On the signaling channel, continuously send the data stream starting from the first frame data in the target media data stream to the user terminal until the media data channel is established.
  • the signaling service unit 510 is specifically configured to receive the RTC session request sent by the user terminal; the signaling service unit 510 is also configured to obtain the first data from the media service unit 520 according to the streaming information flow.
  • Fig. 6 is a schematic structural diagram of a terminal device provided in an exemplary embodiment of the present application.
  • the terminal device may be implemented as the user terminal in the above method embodiments.
  • the terminal device 600 includes: a processor 610 and a transceiver 620 .
  • the terminal device 600 further includes a memory 630 .
  • the processor 610, the transceiver 620 and the memory 630 can communicate with each other through an internal connection path, and transmit control and/or data signals. Call and run the computer program to control the transceiver 620 to send and receive signals.
  • the terminal device 600 may further include an antenna 640, configured to transmit the uplink data or uplink control signaling output by the transceiver 620 through wireless signals.
  • the processor 610 and the memory 630 may be combined into a processing device, and the processor 610 is configured to execute the program codes stored in the memory 630 to realize the above functions.
  • the memory 630 may also be integrated in the processor 610 , or be independent of the processor 610 .
  • the above-mentioned transceiver 620 may include a receiver (or called a receiver, a receiving circuit) and a transmitter (or called a transmitter, a transmitting circuit). Among them, the receiver is used to receive signals, and the transmitter is used to transmit signals.
  • the terminal device 600 may further include a power supply 650, configured to provide power to various devices or circuits in the terminal device 600.
  • a power supply 650 configured to provide power to various devices or circuits in the terminal device 600.
  • the terminal device 600 may also include one or more of an input unit 660, a display unit 670, an audio circuit 680, a camera 690, and a sensor 700.
  • the audio The circuitry may also include a speaker 680a, a microphone 680b, and the like.
  • the terminal device 600 shown in FIG. 6 can implement the first Each process of the terminal or the second terminal.
  • the operations and/or functions of the various modules in the terminal device 600 are respectively for implementing the corresponding processes in the foregoing method embodiments.
  • Fig. 7 is a schematic structural diagram of a server provided in an exemplary embodiment of the present application.
  • the server 800 may be an implementation of the first server in the above method embodiment.
  • the cloud server 800 includes: a memory 810 and a processor 820 .
  • the memory 810 is used to store computer programs, and can be configured to store other various data to support operations on the cloud server.
  • the storage 810 may be an object storage (Object Storage Service, OSS).
  • the processor 820 coupled to the memory 810, is configured to execute the computer program in the memory 810, so as to implement the method implemented by the first server in the method embodiment above.
  • the server when the server is implemented as a cloud server, it also includes: a firewall 830 , a load balancer 840 , a communication component 850 , a power supply component 860 and other components.
  • FIG. 7 only schematically shows some components, which does not mean that the server only includes the components shown in FIG. 7 .
  • server 800 shown in FIG. 7 can implement various processes related to the first server in the above method embodiments.
  • the operations and/or functions of the various modules in the server 800 are respectively for realizing the corresponding processes in the foregoing method embodiments.
  • Both the terminal device shown in FIG. 6 and the server shown in FIG. 7 may be electronic devices.
  • the present application also provides a processing device, including at least one processor, and the at least one processor is used to execute the computer program stored in the memory, so that the processing device executes the method performed by the user terminal or the first server in the above method embodiment. Methods.
  • the embodiment of the present application also provides a processing device, including a processor and an input/output interface.
  • the input-output interface is coupled with the processor.
  • the input and output interface is used for inputting and/or outputting information.
  • the information includes at least one of instructions and data.
  • the processor is configured to execute a computer program, so that the processing device executes the method executed by the user terminal or the first server in the above method embodiments.
  • the embodiment of the present application also provides a processing device, including a processor and a memory.
  • the memory is used to store a computer program
  • the processor is used to call and run the computer program from the memory, so that the processing device executes the method performed by the user terminal or the first server in the above method embodiments.
  • the above processing device may be one or more chips.
  • the processing device may be a field programmable gate array (field programmable gate array, FPGA), an application specific integrated circuit (ASIC), or a system chip (system on chip, SoC). It can be a central processor unit (CPU), a network processor (network processor, NP), a digital signal processing circuit (digital signal processor, DSP), or a microcontroller (micro controller unit) , MCU), can also be a programmable controller (programmable logic device, PLD) or other integrated chips.
  • CPU central processor unit
  • NP network processor
  • DSP digital signal processor
  • microcontroller micro controller unit
  • PLD programmable logic device
  • each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.
  • the steps of the methods disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware. To avoid repetition, no detailed description is given here.
  • the processor in the embodiment of the present application may be an integrated circuit chip, which has a signal processing capability.
  • each step of the above-mentioned method embodiments may be completed by an integrated logic circuit of hardware in a processor or instructions in the form of software.
  • the above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components .
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA field-programmable gate array
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
  • the memory in the embodiments of the present application may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories.
  • the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile The memory may be random access memory (RAM), which acts as an external cache.
  • RAM random access memory
  • static RAM static random access memory
  • dynamic RAM dynamic random access memory
  • DRAM dynamic random access memory
  • synchronous dynamic random access memory synchronous DRAM, SDRAM
  • double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM
  • enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
  • serial link DRAM SLDRAM
  • direct memory bus random access memory direct rambus RAM, DR RAM
  • the present application also provides a computer program product, the computer program product including: computer program code, when the computer program code is run on the computer, the computer is made to execute the method in the above method embodiment The method performed by the first terminal, the second terminal device or the cloud server.
  • the present application also provides a computer-readable storage medium, the computer-readable storage medium stores program codes, and when the program codes are run on a computer, the computer is made to execute the above-mentioned method embodiments The method performed by the first terminal, the second terminal device or the cloud server.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Information Transfer Between Computers (AREA)
  • Computer And Data Communications (AREA)

Abstract

本申请提供一种RTC数据的处理方法以及装置。该方法包括：用户终端向第一服务器发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息，用户终端在信令通道上，接收第一服务器发送的RTC会话响应和目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括目标媒体数据流中除第一数据流之外的数据流，进一步地，用户终端播放该第一数据流。缩短了媒体数据流播放的首帧时间，进而提升了客户端的启播速度。

Description

RTC数据的处理方法以及装置
本申请要求于2022年02月25日提交中国专利局、申请号为202210179933.6、申请名称为“RTC数据的处理方法以及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及网络技术领域，尤其涉及一种RTC数据的处理方法以及装置。
背景技术
在实时通信(Real-Time Communication，RTC)的音/视频数据传输场景中，客户端从服务端拉取媒体数据的拉流过程中，为了降低数据传输的时延，通常在客户端和服务端之间建立媒体数据通道。然而，由于建立媒体数据通道的过程较为复杂，导致客户端从发起拉流请求到接收到媒体数据的首个音/视频帧的时间较长，降低了媒体数据的启播速度。
发明内容
本申请实施例提供一种RTC数据的处理方法以及装置，以期缩短首帧时间，进而提升客户端的启播速度。
第一方面，本申请实施例提供一种RTC数据的处理方法，包括：用户终端向第一服务器发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该用户终端在信令通道上，接收该第一服务器发送的RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括该目标媒体数据流中除该第一数据流之外的数据流；该用户终端播放该第一数据流。
第二方面，本申请实施例提供一种RTC数据传输方法，包括：第一服务器接收用户终端发送的RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该第一服务器在信令通道上，向该用户终端发送RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元 的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括该目标媒体数据流中除该第一数据流之外的数据流。
第三方面，本申请实施例提供一种RTC数据的处理装置，包括：信令单元，用于向第一服务器发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该信令单元还用于在信令通道上，接收该第一服务器发送的RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括该目标媒体数据流中除该第一数据流之外的数据流；媒体单元，用于播放该第一数据流。
第四方面，本申请实施例提供一种RTC数据的处理装置，包括：信令服务单元，用于接收用户终端发送的RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该信令服务单元还用于在信令通道上，向该用户终端发送RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括该目标媒体数据流中除该第一数据流之外的数据流。
第五方面，本申请实施例提供一种电子设备，包括：至少一个处理器和存储器；该存储器存储计算机执行指令；该至少一个处理器执行该存储器存储的计算机执行指令，使得该至少一个处理器执行如第一方面、或第二方面提供的方法。
第六方面，本申请实施例提供一种计算机可读存储介质，该计算机可读存储介质中存储有计算机执行指令，当处理器执行该计算机执行指令时，实现如第一方面或第二方面提供的方法。
第七方面，本申请实施例提供一种计算机程序产品，包括计算机指令，该计算机指令被处理器执行时实现第一方面或第二方面或提供的方法。
在本申请实施例中，在用户终端从第一服务器拉流的过程中，第一服务器通过在信令通道上发送包含目标媒体数据流中的第一数据流，使得目标媒体数据流中的第一数据流提前下发至用户终端，而无需等等媒体数据通道的建立，缩短了媒体数据流播放的首帧时间，进而提升了客户端的启播速度。
附图说明
图1为本申请提供的一种数据传输系统的示意图；
图2为本申请实施例提供的一种RTC数据的处理方法的交互流程示意图；
图3为本申请实施例提供的一种RTC数据的处理方法的交互流程示意图；
图4为本申请实施例提供的一种RTC数据的处理装置的示意性框图；
图5为本申请实施例提供的一种RTC数据的处理装置的示意性框图；
图6为本申请示例性实施例提供的一种终端设备的结构示意图；
图7为本申请示例性实施例提供的一种服务器的结构示意图。
具体实施方式
本申请适用于任意一种RTC的音/视频数据传输场景，例如网络电话、电话会议、视频聊天、直播等等，尤其适用于低延时直播(Real-time Streaming，RTS)场景。
需要说明的是，本申请中传输的音/视频数据均可以是一种数据流(data stream)(下文中称作媒体数据流)，其可以理解为一组有序的，有起点和终点的字节的数据序列。
下面主要以低延时直播为例对本申请进行说明，但并不应理解为对本申请的任何限定。
图1为本申请提供的一种数据传输系统的示意图。如图1所示，该数据传输系统100包括播放客户端110、服务端120和主播客户端130。其中，服务端120分别与播放客户端110和主播客户端130进行通信连接。主播客户端130将录制的数据流发送至服务端120，该过程或称作推流过程，主播客户端130可以在音/视频录制的过程中持续向服务端120发送实时的数据流；播放客户端110从服务端120获取服务端120存储的媒体数据流，该过程或称作拉流过程，该媒体数据流可以是主播客户端130正在录制的数据流中已发送至服务端120的部分流数据。基于此，通过服务端120向播放客户端110传输主播客户端130中实时录制的音/视频，并在播放客户端110中进行渲染播放，以实现直播。
上述播放客户端110、服务端120和主播客户端130中均可以部署有网页即时通信(Web Real-Time Communication，WebRTC)，以支持网页浏览器进行实时音/视频数据传输，实现上述RTS。
上述播放客户端110和主播客户端130均可以实现为任意终端设备，或称作用户终端，例如普通计算机PC、手机(mobile phone)、平板电脑(pad)、智能可穿戴设备、带无线收发功能的电脑、虚拟现实(virtual reality，VR)终端设备、增强现实(augmented reality，AR)终端设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端等。
上述服务端120可以实现为普通服务器、服务器集群，或者云端服务器、服务器集群。服务端120可以用于实现信令服务、媒体服务、网络地址转换(Network Address Translation，NAT)会话穿越应用程序(Session Traversal Utilities for NAT，STUN)服务和NAT的中继穿越(Traversal Using Relays around NAT，TURN)服务中的部分或者全部。在一些实施例中，上述各服务可以分别实现为独立的服务器，例如可以包括图1所示的信令服务器121、媒体服务器122、STUN/TURN服务器123中的部分或者全部，TURN服务器一般可以实现STUN服务和TURN服务；在另一些实施例中，上述部分或者全部服务可以以服务单元的形式集成于同一服务器中，例如下文中的第一服务器可以包括信令服务单元和媒体服务单元。
本申请实施例中，信令服务可以用于实现播放客户端110与服务端120之间的信令交互；媒体服务可以用于实现接收主播客户端130发送的媒体数据流，以及向播放客户端110提供媒体数据流；STUN服务用于检测播放客户端110周围是否存在NAT；TURN服务用于穿透播放客户端110周围的NAT，实现在播放客户端110与服务端120之间建立媒体数据通道。
在上述拉流过程中，为了降低数据传输的时延，通常在播放客户端110和服务端120之间建立媒体数据通道，例如建立WebRTC的媒体数据通道，以实现WebRTC标准接入拉流播放。然而，建立媒体数据通道的过程较为复杂。例如，建立媒体数据通道可以包括信令阶段和交互式连接建立(Interactive Connectivity Establishment，ICE)阶段，在信令阶段中，播放客户端110向服务端120发起拉流请求，播放客户端110和服务端120之间针对所需传输的媒体数据流的元数据进行协商，在ICE阶段基于STUN和/或TURN，建立媒体数据通道，进而在媒体数据通道中传输媒体数据流。此种情况下，播放客 户端110从发起拉流请求，到获取媒体数据流(同下文中的目标媒体数据流)的首个音/视频帧(同下文中的首帧数据)，并对首个音/视频帧进行播放，消耗的时间较长，也即首帧时间较长，降低了播放客户端的启播速度。当然，在其他RTC的会话场景中也同样存在首帧时间长，启播速度慢的问题。
针对上述技术问题，本申请实施例考虑在上述信令阶段中对目标媒体数据中的第一数据流进行拉流，以期缩短首帧时间，进而提升客户端的启播速度。
需要说明的是，首帧数据是客户端请求拉取目标媒体数据流后接收到的目标媒体数据流的首个音/视频帧，该首帧数据可以不是目标媒体数据流的首个音/视频帧。例如目标媒体数据流是主播客户端持续录制的多个音/视频帧，而首帧数据是服务端接收到播放客户端发送的RTC会话请求后，从播放客户端已推流的目标媒体数据流中获取的处于当前时刻的预设时间段内的音/视频帧，该音/视频帧可以是目标媒体数据流中的第n个音/视频帧，n为正整数。进而将该音/视频帧作为首帧数据，持续获取首帧数据之后的数据流。
下面结合附图对本申请实施例提供的RTC数据的处理方法做详细说明。
应理解，下文仅为便于理解和说明，以用户终端和第一服务器之间的交互为例详细说明本申请实施例所提供的方法。该用户终端例如可以是图1中的播放客户端110、第一服务器例如可以是图1中的服务端120或服务端120中的一个服务器。示例性的，第一服务器可以是图1中的信令服务器121，或者部署有信令服务的服务器。
在一些实施例中，第一服务器还部署有媒体服务单元。
在另一些实施例中，媒体服务单元部署于第二服务器，例如是图1中的媒体服务器122，此种情况下，第一服务器和用户终端还分别与第二服务器存在数据交互。
在又一些实施例中，第一服务器和用户终端还与第三服务器存在数据交互，第三服务器例如可以是图1中的STUN/TURN服务器123。
图2为本申请实施例提供的一种RTC数据的处理方法的交互流程示意图。如图2所示，该方法包括：
S210，用户终端向第一服务器发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；相应的，第一服务器接收用户终端发送的RTC会话请求。
S220，第一服务器在信令通道上，向所述用户终端发送RTC会话响应和目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流，该第二数据流包括目标媒体数据流中除第一数据流之外的数据流；相应的，用户终端在信令通道上，接收第一服务器发送的RTC会话响应和目标媒体数据流中的第一数据流。
S230，用户终端播放该第一数据流。
上述RTC会话请求至少携带有用于获取目标媒体数据流的拉流信息，该拉流信息例如包括目标媒体数据流的统一资源标识符(Uniform Resource Identifier，URL)、域名、互联网协议(Internet Protocol，IP)地址中的至少之一。第一服务器基于该拉流信息可以确定用户终端所需的目标媒体数据流。
需要说明的是，目标媒体数据流为用户终端所需拉取的媒体数据流，并不特指某一类媒体数据流。
一般来说，第一服务器响应于该RTC会话请求，通过信令通道向用户终端发送RTC会话响应，该RTC会话响应携带有媒体服务单元的接口信息，使用户终端和部署媒体服务单元的服务器之间可以建立媒体数据通道，进而向用户终端发送该目标媒体数据。其中，媒体服务单元的接口信息包括但不限于媒体服务单元的IP地址和/或端口。
如前所述，媒体服务单元可以部署于第一服务器或第二服务器。当媒体服务单元部署于第一服务器时，媒体服务单元的接口信息可以是第一服务器的IP地址和/或端口；类似的，当媒体服务单元部署于第二服务器时，媒体服务单元的接口信息可以是第二服务器的IP地址和/或端口。
进一步地，本申请实施例为了能够缩短首帧时间，在上述S220中，第一服务器还通过信令通道传输目标媒体数据流中的第一数据流，使第一服务器不需要等待媒体数据通道建立完成，即向用户终端发送目标媒体数据的第一数据流。
第一服务器可以基于拉流信息获取自身存储的目标媒体数据流中的第一数据流，例如第一服务器从第一服务器中部署的媒体服务单元拉取第一数据流；或者，第一服务器可以基于拉流信息从第二服务器获取目标媒体数据流中的第一数据流，例如第一服务器向第二服务器发送拉流信息并接收第二服务器发送的第一数据流。
在一些实施例中，第一数据流在目标媒体数据流中的时序早于第二数据流在目标媒体数据流中的时序。例如，第一数据流包括目标媒体数据流中由首帧数据开始的一段数据流，第二数据流包括目标媒体数据中由第一数据流结束位置的帧数据开始的一段数据流。一般来说，第一数据流结束位置的帧数据与第二数据流开始位置的帧数据在目标媒体数据流中为相邻的帧数据，也即第一数据流和第二数据流具有连续性，当然，本申请并不排除第一数据流与第二数据流不连续或者存在重叠的情况。
本申请实施例对第一数据流的时长不做限定。作为一种示例，第一数据流可以包括由首帧数据开始的预设时长的数据流，该预设时长可以与建立媒体数据通道的时长正相关；作为另一种示例，第一数据流包括目标媒体数据流中由首帧数据开始至媒体数据通道建立完成时，已在信令通道发送的数据流，此种情况下，上述S220具体可以实现为，第一服务器在信令通道上，持续向用户终端发送目标媒体数据流中由首帧数据开始的数据流，直至媒体数据通道建立完成。上述第二种示例相对于第一种示例来说，通过信令通道传输的第一数据流和切换至媒体数据通道传输的目标媒体数据流中的其他数据流能够更好的衔接，减少播放卡顿的现象。
可选的，第一数据流可以包括目标媒体数据流中的全部数据流。例如，第一服务器可以在目标媒体数据流的总时长(或者说数据大小)小于预设值时，在信令通道中发送该目标媒体数据流。
可以理解的是，信令通道和媒体数据通道均为逻辑传输通道，其与传输的阶段以及传输的内容相关。信令通道用于表征用户终端和第一服务器在信令阶段进行信令交互的通道；媒体数据通道用于表征用户终端和第一服务器(或第二服务器)在拉流阶段传输媒体数据的通道，例如基于WebRTC的媒体数据通道。
可选的，用户终端可以通过信令通道向第一服务器发送RTC会话请求。
在上述S210中的RTC会话请求和S220中的会话响应均可以是信令。应理解，信令用于协调通信，例如通过信令交互可以使WebRTC应用程序能够建立一个会话。
例如，RTC会话请求可以实现为会话描述协议(Session Description Protocol，SDP)请求(offer)，RTC会话响应可以实现为SDP响应(answer)。SDP offer和SDP answer可以用于两个会话实体(如用户终端和第一服务器) 之间的媒体协商。
基于此，为了实现信令阶段的协调通信，RTC会话请求还用于请求第一服务器和用户终端协商媒体数据通道上传输的媒体数据的元数据。应理解，元数据被定义为：描述数据的数据，对数据及信息资源的描述性信息，媒体数据的元数据为描述媒体数据的数据。该元数据例如包括媒体数据的编/解码器设置、媒体格式、传输带宽等中的至少之一。在如下图3中将对协商过程进行说明。
在上述S230中，若第一数据流是视频数据流，用户终端将接收到的第一数据流进行渲染，并先用户终端的显示界面中显示渲染得到的画面；若第一数据流是音频数据流，用户终端播放该第一数据流，可选的，用户终端在播放第一数据流时显示音频播放控制界面。
本申请实施例中，在用户终端从第一服务器拉流的过程中，第一服务器通过在信令通道上发送包含目标媒体数据流中的第一数据流，使得目标媒体数据流中的第一数据流提前下发至用户终端，而无需等等媒体数据通道的建立，缩短了媒体数据流播放的首帧时间，进而提升了客户端的启播速度。
一般来说，媒体数据通道有利于提供媒体数据的传输质量，以及媒体数据流传输的实时性。当媒体数据通道建立完成后，可以将模板媒体数据切换至媒体数据通道进行传输，以进一步提高数据传输的质量和实时性。基于此，该方法还包括如图2所示的S240至S260中的部分或者全部过程：
S240，用户终端在播放第一数据流时，根据媒体服务单元的接口信息和STUN，建立用户终端与媒体服务单元之间的媒体数据通道。
S250，第一服务器通过媒体服务单元在媒体数据通道上，向用户终端发送第二数据流；相应的，用户终端在媒体数据通道上，接收媒体服务单元发送的第二数据流。
S260，用户终端播放第二数据流。
为了加快媒体数据通道建立的速度，用户终端可以在播放第一数据流的过程中，例如在对第一数据流进行渲染的过程中，启动建立媒体数据通道的过程。换言之，上述S240和上述S230可以同步执行。
上述S240中，用户终端可以将媒体服务单元的接口信息作为候选地址，并通过STUN服务单元(可以实现为STUN/TURN服务器)进行NAT检测。在检测通过时与媒体服务单元建立媒体数据通道；在NAT检测失败时，基于 TURN服务单元(可以实现为TURN服务器)与媒体服务单元建立媒体数据通道，换言之，通过TURN提供的中继服务，建立用户终端与媒体服务单元之间的媒体数据通道。下面图3所述实施例中将作进一步说明。
上述S250中，第一服务器将目标媒体数据中未通过信令通道发送的数据流切换至媒体数据通道上发送，该切换过程可以理解为，第一服务器按照媒体数据通道的传输协议发送数据流，例如将目标媒体数据中未通过信令通道发送的数据流按照实时传输协议(Real-time Transport Protocol，RTP)进行数据封装后发送。当然，在需要TURN服务器提供中继服务的情况下，第一服务器将目标媒体数据中未通过信令通道发送的数据流发送至TURN服务器，再由TURN服务器转发至用户终端。
可选的，第二数据包括目标媒体数据流中除第一数据流之外的其他数据流中的部分或者全部；或者第二数据还可以包括目标媒体数据流之外的数据流，例如第一数据流包括目标媒体数据流中的全部数据流，则不需要在媒体数据通道中传输目标媒体数据流，此种情况下，第二数据中不包括目标媒体数据流中的数据流。
上述S260中，用户终端播放第二数据流，其实现方式与播放第一数据流类似，此处不再赘述。
下面结合图3所示，以用户终端包括媒体单元和信令单元、第一服务器包括信令服务单元和媒体服务单元、第三服务器包括STUN/TURN服务单元为例，进行示例性的说明。但不应理解为对本申请的限定，例如用户终端中可以不区分媒体单元和信令单元，又例如媒体服务单元可以部署于第二服务器。
图3为本申请实施例提供的一种RTC数据的处理方法的交互流程示意图。如图3所示，该方法包括如下S301至S311中的部分或者全部过程：
S301，信令单元向信令服务单元发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；
S302，信令服务单元向媒体服务单元发送拉流信息；
S303，信令服务单元进行信令协商；
S304，媒体服务单元向信令服务单元发送第一数据流；
S305，信令服务单元在信令通道上，发送RTC会话响应和第一数据流；
S306，信令单元向媒体单元发送第一数据流；
S307，媒体单元播放第一数据流；
S308，信令单元向STUN/TURN服务单元发送STUN绑定(binfing)offer；
S309，STUN/TURN服务单元向信令单元发送STUN binding response；
S310，媒体服务单元在媒体数据通道上，向媒体单元发送第二数据流；
S311，媒体单元播放第二数据流。
上述部分步骤，与图2中对应的步骤类似，具有相同或者相似的实现方式，区别在于图3中的执行主体体现为用户终端或第一服务器中的单元，例如，S301、S305、S307、S310和S311。因此，不再对其实现方式进行赘述。
在上述S302和S304中，信令服务单元向媒体服务单元发送拉流信息以获取媒体服务单元发送的第一数据流；在S303中基于RTC会话请求进行信令协商；进而信令服务单元根据协商的结果生成RTC会话响应，并将RTC会话响应与获取到的第一数据流一起通过信令通道传输给信令单元。
其中，S303的信令协商主要包括对媒体数据的元数据的协商。如前所述，媒体数据的元数据可以包括媒体数据的编/解码器设置、媒体格式、传输带宽等中的至少之一。为了实现媒体数据的元数据的协商，RTC会话请求中应携带有用户终端支持的媒体数据的元数据，第一服务器(例如信令服务单元)可以针对用户终端支持的媒体数据的元数据，确定媒体数据通道上传输的媒体数据的元数据。第一服务器确定的媒体数据通道上传输的媒体数据的元数据，一方面应为用户终端所支持的媒体数据的元数据，另一方面应为媒体服务单元提供的媒体数据所能满足的元数据。
举例而言，若RTC会话请求指示的用户终端支持的媒体格式包括AVI、MPEG、MOV，第一服务器确定RTC会话请求所请求的目标媒体数据流的格式为AVI，则可以通过RTC会话响应指示协商后的媒体数据的媒体格式为AVI。若RTC会话请求指示的用户终端支持的媒体格式包括AVI、MPEG，第一服务器确定RTC会话请求所请求的目标媒体数据流的格式为MOV；则第一服务器通过RTC会话响应指示无法基于该用户终端所支持的媒体数据的元数据进行媒体数据的传输，或者，第一服务器具备将MOV转换为AVI的能力，则第一服务器通过通过RTC会话响应指示协商后的媒体数据的媒体格式为AVI。
若信令服务单元和媒体服务单元均部署于第一服务器，则上述S302和S304为第一服务器内部的实现过程，若信令服务单元部署于第一服务器、媒 体服务单元部署于第二服务器，则上述S302和S304为第一服务器与第二服务器之间的交互过程。
上述S308和S309可以实现媒体数据通道的建立。示例性的，ICE建立可以包括NAT检测阶段和打洞阶段，上述S308和S309可用于实现NAT检测阶段的NAT检测，例如，STUN binding offer中携带有用户终端的IP地址和端口，STUN/TURN服务单元接收到STUN binding offer后，获取发送方的IP地址和端口，将发送方的IP地址和端口与STUN binding offer中携带IP地址和端口进行比较，如果一致，表明用户终端前不存在NAT设备，如果不一致，表明用户终端前存在NAT设备。在用户终端前存在NAT设备的情况下，可以通过TURN服务单元提供中继服务，以建立媒体单元和媒体服务单元之间的媒体数据通道。
在媒体数据通道建立完成后，媒体服务单元将目标媒体数据由信令通道切换至媒体数据通道上继续发送，该过程已在图2所示实施例中说明，此处不再赘述。
需要说明的是，本文中的“第一”、“第二”等描述，是用于区分不同的数据、设备等，不代表先后顺序，也不限定“第一”和“第二”是不同的类型。
图4为本申请实施例提供的一种RTC数据的处理装置的示意性框图。如图4所示，该RTC数据的处理装置400可以包括信令单元410和媒体单元420。
可选的，该RTC数据的处理装置400可对应于上文方法实施例中的用户终端，例如可以为前述播放客户端的一种实现，或者配置于播放客户端中的部件(如芯片或芯片系统等)。
其中，信令单元410可以用于向第一服务器发送RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该收发单元410还用于在信令通道上，接收该第一服务器发送的RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流；媒体单元420可以用于播放该第一数据流。
在一些实施例中，第一数据流在目标媒体数据流中的时序早于第二数据流在目标媒体数据流中的时序。
在一些实施例中，信令单元410还用于在播放该第一数据流时，根据该媒 体服务单元的接口信息和网络地址转换会话穿越应用程序STUN，建立该用户终端与该媒体服务单元之间的媒体数据通道；媒体单元420还用于在该媒体数据通道上，接收该媒体服务单元发送的第二数据流；媒体单元420还用于播放该第二数据流。
在一些实施例中，该第一数据流包括该目标媒体数据流中由该首帧数据开始的预设时长的数据流。
在一些实施例中，该第一数据流包括该目标媒体数据流中由该首帧数据开始至该媒体数据通道建立完成时，已在该信令通道发送的数据流；信令单元410具体用于在该信令通道上，持续接收该第一服务器发送该目标媒体数据流中由该首帧数据开始的数据流，直至该媒体数据通道建立完成。
在一些实施例中，信令单元410具体用于向该第一服务器的信令服务单元发送该RTC会话请求。
在一些实施例中，该媒体服务单元包括于该第一服务器或第二服务器，该第一服务器与该第二服务器相互独立。
应理解，各单元执行上述相应步骤的具体过程在上述方法实施例中已经详细说明，为了简洁，在此不再赘述。
图5为本申请实施例提供的一种RTC数据的处理装置的示意性框图。如图5所示，该RTC数据的处理装置500可以包括信令服务单元510和媒体服务单元520。
可选的，该RTC数据的处理装置500可对应于上文方法实施例中的第一服务器，例如可以为第一服务器，或者配置于第一服务器中的部件(如芯片或芯片系统等)。
其中，信令服务单元510可以用于接收用户终端发送的RTC会话请求，该RTC会话请求携带有目标媒体数据流的拉流信息；该信令服务单元510还用于在信令通道上，向该用户终端发送RTC会话响应和该目标媒体数据流中的第一数据流，该RTC会话响应携带有媒体服务单元的接口信息，该媒体服务单元的接口信息用于建立媒体数据通道，该媒体数据通道用于传输第二数据流。
在一些实施例中，该第一数据流在目标媒体数据流中的时序早于第二数据流在目标媒体数据流中的时序。
在一些实施例中，该媒体服务单元520在该媒体数据通道上，向该用户终 端发送第二数据流。
在一些实施例中，该第一数据流包括该目标媒体数据流中由该首帧数据开始的预设时长的数据流。
在一些实施例中，该第一数据流包括该目标媒体数据流中由该首帧数据开始至该媒体数据通道建立完成时，已在该信令通道发送的数据流；媒体服务单元510具体用于在该信令通道上，持续向该用户终端发送该目标媒体数据流中由该首帧数据开始的数据流，直至该媒体数据通道建立完成。
在一些实施例中，信令服务单元510具体用于接收该用户终端发送的该RTC会话请求；该信令服务单元510还用于根据该拉流信息，从媒体服务单元520获取该第一数据流。
应理解，各单元执行上述相应步骤的具体过程在上述方法实施例中已经详细说明，为了简洁，在此不再赘述。
图6为本申请示例性实施例提供的一种终端设备的结构示意图。该终端设备可以实现为上文方法实施例中的用户终端。如图6所示，该终端设备600包括：处理器610和收发器620。可选地，该终端设备600还包括存储器630。其中，处理器610、收发器620和存储器630之间可以通过内部连接通路互相通信，传递控制和/或数据信号，该存储器630用于存储计算机程序，该处理器610用于从该存储器630中调用并运行该计算机程序，以控制该收发器620收发信号。可选地，终端设备600还可以包括天线640，用于将收发器620输出的上行数据或上行控制信令通过无线信号发送出去。
上述处理器610可以和存储器630可以合成一个处理装置，处理器610用于执行存储器630中存储的程序代码来实现上述功能。具体实现时，该存储器630也可以集成在处理器610中，或者独立于处理器610。
上述收发器620可以包括接收器(或称接收机、接收电路)和发射器(或称发射机、发射电路)。其中，接收器用于接收信号，发射器用于发射信号。
可选地，上述终端设备600还可以包括电源650，用于给终端设备600中的各种器件或电路提供电源。
除此之外，为了使得该终端设备的功能更加完善，该终端设备600还可以包括输入单元660、显示单元670、音频电路680、摄像头690和传感器700等中的一个或多个，所述音频电路还可以包括扬声器680a、麦克风680b等。
应理解，图6所示的终端设备600能够实现上文方法实施例中涉及第一 终端或第二终端的各个过程。终端设备600中的各个模块的操作和/或功能，分别为了实现上述方法实施例中的相应流程。具体可参见上述方法实施例中的描述，为避免重复，此处适当省略详细描述。
图7为本申请示例性实施例提供的一种服务器的结构示意图。该服务器800可以为上文方法实施例中第一服务器的一种实现。如图7所示，该云服务器800包括：存储器810和处理器820。
存储器810，用于存储计算机程序，并可被配置为存储其它各种数据以支持在云服务器上的操作。该存储器810可以是对象存储(Object Storage Service，OSS)。
处理器820，与存储器810耦合，用于执行存储器810中的计算机程序，以用于实现上文方法实施例中由第一服务器实现的方法。
进一步，如图7所示，该服务器实现为云服务器时还包括：防火墙830、负载均衡器840、通信组件850、电源组件860等其它组件。图7中仅示意性给出部分组件，并不意味着服务器只包括图7所示组件。
应理解，图7所示的服务器800能够实现上文方法实施例中涉及第一服务器的各个过程。服务器800中的各个模块的操作和/或功能，分别为了实现上述方法实施例中的相应流程。具体可参见上述方法实施例中的描述，为避免重复，此处适当省略详细描述。
图6所示的终端设备和图7所示的服务器均可以是一种电子设备。
本申请还提供了一种处理装置，包括至少一个处理器，所述至少一个处理器用于执行存储器中存储的计算机程序，以使得所述处理装置执行上述方法实施例中用户终端或第一服务器执行的方法。
本申请实施例还提供了一种处理装置，包括处理器和输入输出接口。所述输入输出接口与所述处理器耦合。所述输入输出接口用于输入和/或输出信息。所述信息包括指令和数据中的至少一项。所述处理器用于执行计算机程序，以使得所述处理装置执行上述方法实施例中用户终端或第一服务器执行的方法。
本申请实施例还提供了一种处理装置，包括处理器和存储器。所述存储器用于存储计算机程序，所述处理器用于从所述存储器调用并运行所述计算机程序，以使得所述处理装置执行上述方法实施例中用户终端或第一服务器执行的方法。
应理解，上述处理装置可以是一个或多个芯片。例如，该处理装置可以是现场可编程门阵列(field programmable gate array，FPGA)，可以是专用集成芯片(application specific integrated circuit，ASIC)，还可以是系统芯片(system on chip，SoC)，还可以是中央处理器(central processor unit，CPU)，还可以是网络处理器(network processor，NP)，还可以是数字信号处理电路(digital signal processor，DSP)，还可以是微控制器(micro controller unit，MCU)，还可以是可编程控制器(programmable logic device，PLD)或其他集成芯片。
在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
应注意，本申请实施例中的处理器可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
可以理解，本申请实施例中的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性 存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
根据本申请实施例提供的方法，本申请还提供一种计算机程序产品，该计算机程序产品包括：计算机程序代码，当该计算机程序代码在计算机上运行时，使得该计算机执行上述方法实施例中的第一终端、第二终端设备或云服务端执行的方法。
根据本申请实施例提供的方法，本申请还提供一种计算机可读存储介质，该计算机可读存储介质存储有程序代码，当该程序代码在计算机上运行时，使得该计算机执行上述方法实施例中的第一终端、第二终端设备或云服务端执行的方法。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (14)

  1. 一种实时通信RTC数据的处理方法，其特征在于，包括：
    用户终端向第一服务器发送RTC会话请求，所述RTC会话请求携带有目标媒体数据流的拉流信息；
    所述用户终端在信令通道上，接收所述第一服务器发送的RTC会话响应和所述目标媒体数据流中的第一数据流，所述RTC会话响应携带有媒体服务单元的接口信息，所述媒体服务单元的接口信息用于建立媒体数据通道，所述媒体数据通道用于传输第二数据流，所述第二数据流包括所述目标媒体数据流中除所述第一数据流之外的数据流；
    所述用户终端播放所述第一数据流。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述用户终端在播放所述第一数据流时，根据所述媒体服务单元的接口信息和网络地址转换会话穿越应用程序STUN，建立所述用户终端与所述媒体服务单元之间的媒体数据通道；
    所述用户终端在所述媒体数据通道上，接收所述媒体服务单元发送的所述第二数据流；
    所述用户终端播放所述第二数据流。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第一数据流包括所述目标媒体数据流中由首帧数据开始的预设时长的数据流。
  4. 根据权利要求1或2所述的方法，其特征在于，所述第一数据流包括所述目标媒体数据流中由首帧数据开始至所述媒体数据通道建立完成时，已在所述信令通道发送的数据流；
    所述用户终端在信令通道上，接收所述第一服务器发送的所述目标媒体数据流中的第一数据流，包括：
    所述用户终端在所述信令通道上，持续接收所述第一服务器发送所述目标媒体数据流中由所述首帧数据开始的数据流，直至所述媒体数据通道建立完成。
  5. 根据权利要求1或2所述的方法，其特征在于，所述第一数据流在所述目标媒体数据流中的时序早于所述第二数据流在所述目标媒体数据流中的时序。
  6. 根据权利要求1或2所述的方法，其特征在于，所述用户终端向第一服务器发送RTC会话请求，包括：
    所述用户终端向所述第一服务器的信令服务单元发送所述RTC会话请求。
  7. 根据权利要求1或2所述的方法，其特征在于，所述媒体服务单元包括于所述第一服务器或第二服务器，所述第一服务器与所述第二服务器相互独立。
  8. 一种RTC数据的处理方法，其特征在于，包括：
    第一服务器接收用户终端发送的RTC会话请求，所述RTC会话请求携带有目标媒体数据流的拉流信息；
    所述第一服务器在信令通道上，向所述用户终端发送RTC会话响应和所述目标媒体数据流中的第一数据流，所述RTC会话响应携带有媒体服务单元的接口信息，所述媒体服务单元的接口信息用于建立媒体数据通道，所述媒体数据通道用于传输第二数据流，所述第二数据流包括所述目标媒体数据流中除所述第一数据流之外的数据流。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    所述第一服务器通过所述媒体服务单元在所述媒体数据通道上，向所述用户终端发送所述第二数据流。
  10. 根据权利要求8或9所述的方法，其特征在于，所述第一数据流包括所述目标媒体数据流中由首帧数据开始的预设时长的数据流。
  11. 根据权利要求8或9所述的方法，其特征在于，所述第一数据流包括所述目标媒体数据流中由首帧数据开始至所述媒体数据通道建立完成时，已在所述信令通道发送的数据流；
    所述第一服务器在信令通道上，向所述用户终端发送所述目标媒体数据流中的第一数据流，包括：
    所述第一服务器在所述信令通道上，持续向所述用户终端发送所述目标媒体数据流中由所述首帧数据开始的数据流，直至所述媒体数据通道建立完成。
  12. 根据权利要求8或9所述的方法，其特征在于，所述第一服务器接收用户终端发送的RTC会话请求，包括：
    所述第一服务器通过信令服务单元接收所述用户终端发送的所述RTC会话请求；
    所述方法还包括：
    所述第一服务器通过所述信令服务单元根据所述拉流信息，从媒体服务单元获取所述第一数据流。
  13. 一种RTC数据的处理装置，其特征在于，包括：
    信令单元，用于向第一服务器发送RTC会话请求，所述RTC会话请求携带有目标媒体数据流的拉流信息；
    所述信令单元还用于在信令通道上，接收所述第一服务器发送的RTC会话响应和所述目标媒体数据流中的第一数据流，所述RTC会话响应携带有媒体服务单元的接口信息，所述媒体服务单元的接口信息用于建立媒体数据通道，所述媒体数据通道用于传输第二数据流，所述第二数据流包括所述目标媒体数据流中除所述第一数据流之外的数据流，所述第一数据流在所述目标媒体数据流中的时序早于所述第二数据流在所述目标媒体数据流中的时序；
    媒体单元，用于播放所述第一数据流。
  14. 一种RTC数据的处理装置，其特征在于，包括：
    信令服务单元，用于接收用户终端发送的RTC会话请求，所述RTC会话请求携带有目标媒体数据流的拉流信息；
    所述信令服务单元还用于在信令通道上，向所述用户终端发送RTC会话响应和所述目标媒体数据流中的第一数据流，所述RTC会话响应携带有媒体服务单元的接口信息，所述媒体服务单元的接口信息用于建立媒体数据通道，所述媒体数据通道用于传输第二数据流，所述第二数据流包括所述目标媒体数据流中除所述第一数据流之外的数据流，所述第一数据流在所述目标媒体数据流中的时序早于所示第二数据流在所述目标媒体数据流中的时序。
PCT/CN2023/074514 2022-02-25 2023-02-06 Rtc数据的处理方法以及装置 WO2023160361A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210179933.6A CN114553839B (zh) 2022-02-25 2022-02-25 Rtc数据的处理方法以及装置
CN202210179933.6 2022-02-25

Publications (1)

Publication Number Publication Date
WO2023160361A1 true WO2023160361A1 (zh) 2023-08-31

Family

ID=81678797

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/074514 WO2023160361A1 (zh) 2022-02-25 2023-02-06 Rtc数据的处理方法以及装置

Country Status (2)

Country Link
CN (1) CN114553839B (zh)
WO (1) WO2023160361A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114553839B (zh) * 2022-02-25 2024-03-15 阿里巴巴（中国）有限公司 Rtc数据的处理方法以及装置
CN115037979B (zh) * 2022-07-13 2023-09-01 北京字跳网络技术有限公司 投屏方法及相关设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101188734A (zh) * 2006-11-15 2008-05-28 中兴通讯股份有限公司 一种流媒体快速播放的方法
CN105791894A (zh) * 2014-12-24 2016-07-20 中兴通讯股份有限公司 一种频道码流处理方法、装置、终端及系统
US20180007430A1 (en) * 2008-08-13 2018-01-04 At&T Intellectual Property I, L.P. Peer-to-Peer Video Data Sharing
CN107948664A (zh) * 2017-11-20 2018-04-20 广州虎牙信息科技有限公司 直播间视频播放控制方法、装置及终端
CN111818361A (zh) * 2020-09-15 2020-10-23 平安国际智慧城市科技股份有限公司 控制流媒体业务交互的方法、web客户端设备及系统
CN114553839A (zh) * 2022-02-25 2022-05-27 阿里巴巴（中国）有限公司 Rtc数据的处理方法以及装置

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2860121A1 (fr) * 2003-09-23 2005-03-25 France Telecom Procede d'etablissement d'un transfert de donnees entre deux dispositifs de communication et dispositif associe
KR100953647B1 (ko) * 2006-12-13 2010-04-20 엘지전자 주식회사 무선 네트워크에서의 연결 제어 방법
CN102123511B (zh) * 2011-03-18 2013-08-28 中国电信股份有限公司 移动网络数据传输方法、系统及移动终端
US9088406B2 (en) * 2012-07-29 2015-07-21 Qualcomm Incorporated Frame sync across multiple channels
CN103634299B (zh) * 2013-11-14 2016-09-14 北京邮电大学 基于多连接的实时流媒体传输终端与方法
US9444792B2 (en) * 2014-10-21 2016-09-13 Oracle International Corporation Dynamic tunnel for real time data communication
CN105744209B (zh) * 2014-12-12 2019-01-25 中兴通讯股份有限公司 媒体数据流的传输方法和装置
CN109274634B (zh) * 2017-07-18 2021-06-11 腾讯科技（深圳）有限公司 多媒体通信方法及装置、存储介质
CN109802932B (zh) * 2017-11-17 2022-04-12 华为技术有限公司 一种直播通信的方法和装置
CN109218745B (zh) * 2018-10-31 2021-07-23 网宿科技股份有限公司 一种直播方法、服务器、客户端及可读存储介质
CN110278452A (zh) * 2019-06-24 2019-09-24 北京字节跳动网络技术有限公司 视频启动加速方法、装置、存储介质、终端和服务器
CN113037751B (zh) * 2021-03-09 2023-10-31 北京字节跳动网络技术有限公司 创建音视频接收流的方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101188734A (zh) * 2006-11-15 2008-05-28 中兴通讯股份有限公司 一种流媒体快速播放的方法
US20180007430A1 (en) * 2008-08-13 2018-01-04 At&T Intellectual Property I, L.P. Peer-to-Peer Video Data Sharing
CN105791894A (zh) * 2014-12-24 2016-07-20 中兴通讯股份有限公司 一种频道码流处理方法、装置、终端及系统
CN107948664A (zh) * 2017-11-20 2018-04-20 广州虎牙信息科技有限公司 直播间视频播放控制方法、装置及终端
CN111818361A (zh) * 2020-09-15 2020-10-23 平安国际智慧城市科技股份有限公司 控制流媒体业务交互的方法、web客户端设备及系统
CN114553839A (zh) * 2022-02-25 2022-05-27 阿里巴巴（中国）有限公司 Rtc数据的处理方法以及装置

Also Published As

Publication number Publication date
CN114553839A (zh) 2022-05-27
CN114553839B (zh) 2024-03-15

Similar Documents

Publication Publication Date Title
CN109327728B (zh) 一种一对多同屏方法、装置和系统、同屏设备及存储介质
WO2023160361A1 (zh) Rtc数据的处理方法以及装置
EP3562163B1 (en) Audio-video synthesis method and system
US10110393B2 (en) Protocol switching over multi-network interface
CN105763832B (zh) 一种视频互动、控制方法及装置
EP2933982B1 (en) Media stream transfer method and user equipment
CN105282601A (zh) 一种同屏共享的方法、装置及系统
US20190110091A1 (en) Method and device for synchronously performing an operation on contents
CN103348657B (zh) 流媒体播放方法、设备及系统
JP6430034B2 (ja) 低遅延生放送コンテンツ提供のためのプログラムを記録した記録媒体及び装置
CN111092898B (zh) 报文传输方法及相关设备
WO2021168649A1 (zh) 多功能接收设备和会议系统
KR102064284B1 (ko) 실시간 통신을 수행하는 장치, 시스템, 및 방법
WO2023071813A1 (zh) 一种数据传输方法、设备及存储介质和终端设备
CN107547517B (zh) 音视频节目录制方法和网络设备及计算机装置
CN112153457A (zh) 无线投屏连接方法、装置、计算机存储介质及电子设备
CN114710568B (zh) 音视频数据通信方法、设备及存储介质
US20190098351A1 (en) Method for managing the access right to an item of digital content
US20180267907A1 (en) Methods and apparatus for communication between mobile devices and accessory devices
CN115334059A (zh) 一种音视频互通方法、装置、设备及存储介质
EP4287586A1 (en) Media playback method and apparatus and electronic device
CN113992637B (zh) 音视频数据接收方法、装置、设备、系统和存储介质
WO2023000133A1 (zh) 互联网直播系统
CN113727183B (zh) 直播推流方法、装置、设备、存储介质及计算机程序产品
CN112738056B (zh) 编解码方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23758986

Country of ref document: EP

Kind code of ref document: A1