WO2016112639A1 - 流媒体数据传输方法、客户端和服务器 - Google Patents

流媒体数据传输方法、客户端和服务器 Download PDF

Info

Publication number
WO2016112639A1
WO2016112639A1 PCT/CN2015/081737 CN2015081737W WO2016112639A1 WO 2016112639 A1 WO2016112639 A1 WO 2016112639A1 CN 2015081737 W CN2015081737 W CN 2015081737W WO 2016112639 A1 WO2016112639 A1 WO 2016112639A1
Authority
WO
WIPO (PCT)
Prior art keywords
fragment
server
code rate
client
rate
Prior art date
Application number
PCT/CN2015/081737
Other languages
English (en)
French (fr)
Inventor
张楚雄
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to JP2016569038A priority Critical patent/JP6563424B2/ja
Priority to EP15837172.4A priority patent/EP3247121B1/en
Priority to US14/895,378 priority patent/US10516712B2/en
Priority to KR1020167031247A priority patent/KR101978738B1/ko
Publication of WO2016112639A1 publication Critical patent/WO2016112639A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26233—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving content or additional data duration or size, e.g. length of a movie, size of an executable file
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/611—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for multicast or broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/633—Control signals issued by server directed to the network components or client
    • H04N21/6332—Control signals issued by server directed to the network components or client directed to client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1812—Hybrid protocols; Hybrid automatic repeat request [HARQ]
    • H04L1/1819—Hybrid protocols; Hybrid automatic repeat request [HARQ] with retransmission of additional or different redundancy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1829—Arrangements specially adapted for the receiver end
    • H04L1/1835—Buffer management
    • H04L1/1845—Combining techniques, e.g. code combining

Definitions

  • the present disclosure relates to streaming media technologies, and in particular, to a streaming media data transmission apparatus, method and system.
  • the Dynamic Adaptive Streaming over HTTP (DASH) standard hereinafter referred to as the MPEG DASH standard, developed by the Moving Pictures Experts Group (MPEG) provides a standardized solution for transmitting streaming media data using Internet streaming media technology.
  • DASH Dynamic Adaptive Streaming over HTTP
  • MPEG Moving Pictures Experts Group
  • the hierarchical structure model of Media Presentation Description (MPD) defined by the MPEG DASH standard is shown in Figure 1.
  • a period is used to describe media content that can be played for a period of time, and media content described in an order of adjacent time periods is continuous in time.
  • a time period includes a plurality of adaptation sets, each of which describes media content adapted to a plurality of code rates, and each code rate corresponds to one representation (Representation). Presenting information describing the specific encapsulation format, code rate, codec parameters, and the like of the media content.
  • a Uniform Resoure Locator (URL) containing a plurality of segments is used to indicate a storage location of the corresponding slice, and the slice includes specific media content, that is, audio, video, subtitle, and complex. Use audio and video, etc.
  • an alternative solution for transmitting streaming media data over the Internet is to first establish a WebSocket two-way connection, transmit the control information of the MPEG DASH standard through a WebSocket text frame, and transmit the fragments through the WebSocket binary frame.
  • the process of the client displaying the media content under the framework shown in FIG. 1 refers to FIG. 2.
  • the process of the client displaying the media content under the framework shown in FIG. 1 includes the following steps.
  • the client sends an OpenMedia message to the server.
  • the client passes the cancellation
  • the URL carries the URL address of the MPD, and indicates to the server the media to be played.
  • step S202 the server sends a MediaInfo message to the client to inform the client of the media information, such as whether the media is ready.
  • step S203 the client sends a StartStream message to the server to request the server to deliver the media.
  • step S204 the server sends a StreamInfo message to the client to inform the client of the media stream information.
  • step S205 the server starts to deliver the media slice.
  • the server may carry a plurality of presentations for indicating whether the server will deliver the same content and different code rates in the StreamInfo message sent to the client by using the step S204 in FIG. Indicates the message multipleRepresentation. If the indication information indicates that the server sends multiple presentations of the same content and different code rates, the server sends multiple fragments of the same content and different code rates to the client simultaneously when the fragment is sent, that is, the multiple The pieces that contain the same content and different code rates are presented.
  • the OpenMedia message, the MediaInfo message, the StartStream message, and the StreamInfo message involved in the above process may be regarded as messages extended by the MPEG DASH standard on the WebSocket protocol and transmitted through the text frame of the WebSocket.
  • the client does not know whether the fragment sent by the server is high bit rate or low bit rate.
  • a known implementation manner includes: after receiving a fragment, the client determines whether to receive a higher code rate fragment within a preset waiting time; if receiving a higher rate rate fragment, Then, the slice of the higher code rate is decoded and displayed; if the slice of the higher code rate is not received, the previously received slice is decoded and displayed.
  • the client after the client receives the fragment of the high code rate, the client does not know that the same content sent by the subsequent server and the fragment of the different code rate are higher.
  • the rate of fragmentation, or the lower rate rate of the fragment will still wait for the above-mentioned preset waiting time, increasing the delay of the client decoding.
  • the present disclosure provides a streaming media data transmission apparatus, method and system for solving the problem that a client waits for a higher code rate fragmentation and prolongs decoding.
  • a client including: a first processing module, Determining whether the server delivers multiple fragments of the same content and different code rates; and the second processing module is configured to determine, by the first processing module, that the server delivers multiple fragments of the same content and different code rates. In the case, it is determined whether the server preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low bit rate.
  • a server including: a processing module, configured to determine to preferentially send a high-rate-rate fragment in a case where a plurality of fragments of the same content and different code rates are delivered to a client Or sending the low bit rate fragment preferentially; the transceiver module sends the first indication information for indicating whether to send the high code rate fragment preferentially to the client.
  • a streaming media data transmission method including: determining whether a server delivers multiple fragments of the same content and different code rates; if the server delivers the same content and different code rates If the plurality of fragments are used, it is determined whether the server preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low bit rate.
  • another streaming media data transmission method including: determining to preferentially transmit a high code rate fragment when a plurality of fragments of the same content and different code rates are delivered to a client Or sending the low bit rate fragment preferentially; sending the first indication information for indicating whether to send the high code rate fragment preferentially to the client.
  • a streaming media data transmission system including a client and a server, wherein the client is configured to determine whether the server delivers the same content, multiple points of different code rates. If the server delivers multiple fragments of the same content and different code rates, it is determined whether the server preferentially transmits the high-rate rate fragment or the low-rate rate fragment; the server is used to Send media fragments to the client.
  • the client can determine whether the server preferentially transmits the high code rate fragment or the low code rate fragment, so that the client can determine that the server preferentially transmits the high code rate.
  • the server In the case of a slice, there is no need to wait for the server to deliver a higher bit rate fragment, and directly decode the received slice, which shortens the delay of the client decoding.
  • Figure 1 is a model diagram of an MPD hierarchy defined by the MPEG DASH standard
  • FIG. 2 is a schematic flowchart of a process of displaying media content by a client under the framework shown in FIG. 1;
  • FIG. 3 is a schematic structural diagram of a streaming media data transmission system according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a frame structure of a WebSocket
  • FIG. 5 is a schematic diagram of a code rate switching manner
  • FIG. 6 is a flowchart of processing of streaming media data transmission of a client according to Embodiment 1 of the present disclosure
  • FIG. 7A is a schematic diagram of a first structure of a client according to an embodiment of the present disclosure.
  • FIG. 7B is a schematic diagram of a second structure of a client according to an embodiment of the present disclosure.
  • FIG. 7C is a schematic diagram of a third structure of a client according to an embodiment of the present disclosure.
  • FIG. 7D is a schematic structural diagram of a fourth terminal of a client according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of a server according to an embodiment of the present disclosure.
  • FIG. 9 is a flowchart of a method for transmitting streaming media data according to an embodiment of the present disclosure.
  • FIG. 10 is a flowchart of another streaming media data transmission method according to an embodiment of the present disclosure.
  • the embodiments of the present disclosure provide a streaming media data transmission apparatus and method, which are used to solve the problem that a client waits for a higher code rate fragmentation and is extended during decoding.
  • FIG. 3 is a schematic structural diagram of a streaming media data transmission system according to an embodiment of the present disclosure. As shown in FIG. 3, the system includes a client 301 and a server 302.
  • the client 301 determines whether the server 302 delivers multiple fragments of the same content and different code rates. In the case that the determining server 302 delivers multiple fragments of the same content and different code rates, the client 301 further determines whether the server 302 preferentially transmits the high-rate rate fragment or preferentially transmits the low-rate rate fragment.
  • the server 302 delivers the media to the client 301.
  • FIG. 3 for simplicity of illustration, only one client 301 and one server 302 are shown. In fact, one server 302 can deliver media to multiple clients 301, and one client 301 can also be directed to multiple servers 302. Request to deliver the media.
  • the client 301 establishes a WebSocket bidirectional connection with the server 302, and performs streaming media transmission based on the established WebSocket bidirectional connection.
  • the client 301 can request the server 302 to deliver the media through the step S203 in FIG. 2, that is, the client 301 sends a StartStream message to the server 302 to request the media stream.
  • the client 301 can send the media after requesting the server 302 and at the server 302. Before the fragment is sent to the client 301, it is determined whether the server 302 preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low bit rate. Alternatively, the client 301 may also determine whether the server 302 preferentially transmits the high code rate fragment or the low rate rate fragment in the process of sending the media by the server 302.
  • the server 302 can indicate to the client 301 whether the server 302 delivers multiple fragments of the same content and different code rates by using the multipleRepresentation key in the StreamInfo message sent to the client 301.
  • the client 301 is instructed that the server 302 delivers multiple fragments of the same content and different code rates.
  • the client 301 determines in the process of sending the media by the server 302 that the server 302 preferentially transmits the high-rate rate fragment or preferentially transmits the low-rate rate fragment, the client 301 can Or the second method determines whether the server 302 preferentially transmits the high code rate fragment or preferentially transmits the low code rate fragment.
  • the client 301 determines whether the server 302 preferentially transmits the high code rate fragment or the priority transmission low rate rate fragment before the server 302 sends each fragment
  • the client The terminal 301 can determine whether the server 302 preferentially transmits the high-rate rate fragment or the low-rate rate fragment by preferentially performing the following manner.
  • the code rate of the first fragment sent by the server 302 is a preset rate limit. For example, the client 301 knows in advance the server 302 sends the packet to the client 301. If the highest code rate of each slice is set to the preset code rate upper limit, the client 301 determines that the server 302 preferentially transmits the high code rate fragment; if the client 301 is under the request server 302 After the media is received, the code rate of the first fragment sent by the server 302 is a preset lower rate. For example, the client 301 knows in advance the lowest bit rate of each fragment sent by the server 302 to the client 301. The minimum bit rate is set to a preset lower code rate), and the client 301 determines that the server 302 preferentially transmits the low bit rate slice.
  • the client 301 receives the high-rate fragment sent by the server 302 after receiving the media, the client 301 receives the low-rate fragment sent by the server 302, and determines that the server 302 preferentially transmits the high-code.
  • the fragmentation of the rate if the fragment of the low bit rate sent by the server 302 is received first, and then the fragment of the high rate rate sent by the server 302 is received, it is determined that the server 302 preferentially transmits the fragment of the low bit rate.
  • the server 302 waits for the receiving.
  • the next fragment if the code rate of the next fragment received is lower than the code rate of the first fragment, it is determined that the server 302 preferentially transmits the fragment of the high code rate; if the next fragment is received
  • the code rate is higher than the code rate of the first slice, and it is determined that the server 302 preferentially transmits the slice of the low code rate.
  • the second mode is determined by the client 301 according to the first indication information sent by the server 302.
  • the first indication information is used to indicate whether the server 302 preferentially transmits the fragment of the high code rate.
  • the StreamInfo message shown in Table 1 may be modified to provide the first indication information in the StreamInfo message.
  • the key "bitratePriority” may be added to the StreamInfo message shown in Table 1, and the first indication information may be provided by the key "bitratePriority".
  • bitratePriority is located after the key "multipleRepresentation”.
  • the position is not limited to the position shown in Table 2, for example, it may be placed at the end of the entire message.
  • bitratePriority in Table 2 is a Boolean type.
  • the value of "true” indicates that the server 302 preferentially transmits the fragment of the high bit rate.
  • the value of "false” indicates that the server 302 preferentially transmits the fragment of the low bit rate.
  • other types may also be used, such as a numeric type.
  • a value of 1 indicates that the server 302 preferentially transmits a high-rate rate fragment.
  • a value of 0 indicates that the server 302 preferentially transmits a low-rate fragment.
  • the specific implementation is not limited to the above example, as long as the client 301 can determine whether the server 302 preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low bit rate.
  • the client 301 may also determine that the receiving is high. Whether the reception of the high rate slice is completed within the first time threshold after the slice of the code rate. If the receiving of the fragment of the high code rate is not completed within the first time threshold, the client 301 The high rate rate fragment can be discarded and begin to receive low bit rate fragments.
  • the client 301 may start timing after starting to receive the high code rate fragment.
  • the timing may be started after determining that the server 302 preferentially transmits the high code rate fragment.
  • the client 301 sends the media after requesting the server 302 and before the server 302 sends the fragments to the client 301, it is determined whether the server 302 preferentially transmits the high-rate fragment or preferentially transmits the low-rate. After fragmentation, the client 301 can also start timing after requesting the server 302 to deliver the media.
  • the fragment transmission efficiency between the client 301 and the server 302 is improved. If the client 301 completes the receiving of the high-rate rate fragment within the first time threshold, the client The terminal 301 sends the second indication information to the server 302, where the second indication information is used to notify the server 302 that the low code rate fragment is no longer needed.
  • the client 301 can determine whether to complete the receiving of the fragment according to the indication information in the WebSocket frame.
  • the frame structure of WebSocket is shown in Figure 4.
  • the operation code occupies 4 bits. The following is the correspondence between the value of the operation code and the meaning of the operation code:
  • end tag 1 opcode ⁇ 0
  • end tag 0, opcode ⁇ 0
  • the client 301 After receiving the text frame containing the StreamInfo message sent by the server 302, the client 301 starts receiving the fragment sent by the server 302. If a fragment is too large, the fragment may be divided into multiple segments, and the multiple segments may be respectively transmitted in multiple WebSocket binary frames, specifically, one segment is transmitted in a WebSocket binary frame. segment.
  • Server 302 sends a slice through one or more binary frames.
  • the client 301 determines that the current fragment is sent in segments (ie, one slice is sent through multiple binary frames), The binary frame contains the first segment of the slice and subsequently needs to receive the binary frame containing the remaining segments of the slice.
  • the client 301 continuously receives subsequent binary frames until the end of the received binary frame is marked with "0” and the operation code is "0", and it is determined that the reception of the fragment has been completed.
  • the client 301 sends the second indication information to the server 302, where the second indication information is used to notify the server 302 that there is no need to Send a low bit rate fragment.
  • the second indication information can be sent by the newly defined message by defining the following new message ReceiveReport, as shown in Table 3.
  • the message may contain the key-value pairs listed in Table 4 below.
  • the client 301 If the client 301 completes the receiving of the high code rate fragment within the first time threshold, the client 301 sends the ReceiveReport message to the server, and the ReceiveStatus of the message is True. After receiving the message, the server 302 will not send the low bit rate fragment to the client 301.
  • the client 301 If the client 301 does not complete the receiving of the high-rate fragment within the first time threshold, the client 301 sends the ReceiveReport message to the server 302, and the ReceiveStatus of the message is False. The server 302 will terminate the transmission of the high code rate fragments and begin transmitting the low code rate fragments.
  • the client 301 continues to receive the high-rate fragment after starting to receive the low-rate fragment.
  • the client 301 determines whether there are multiple high rate rate fragments. If there are multiple high rate rate fragments, the client 301 selects a high code rate fragment from the plurality of high code rate fragments existing according to the network bandwidth with the server 302 and starts receiving.
  • the client 301 After starting to receive the high code rate fragment, the client 301 determines whether to complete the reception of the high code rate fragment within the second time threshold after starting to receive the high code rate fragment. If the client 301 determines that the reception of the fragment of the high code rate is not completed within the second time threshold after the start of receiving the fragment of the high code rate, the client 301 decodes the fragmentation of the received low bit rate. .
  • the client 301 may also start timing after determining that the server 302 preferentially transmits the low bit rate fragment.
  • the client 301 sends the media after requesting the server 302 and before the server 302 sends the fragments to the client 301, it is determined whether the server 302 preferentially transmits the high-rate fragment or priority. After the low bit rate fragment is sent, the client 301 can also start timing after requesting the server 302 to deliver the media.
  • the client 301 after receiving the low code rate fragment, the client 301 does not decode the received low code rate fragment but waits to receive the high code rate fragment. If the client 301 finishes receiving the fragment of the high code rate within the second time threshold after the server 302 is requested to send the media, the high code rate fragment is decoded, and the previously received low code is discarded. The fragmentation of the rate; if the reception of the fragment of the high code rate is not completed within the second time threshold after the requesting server 302 sends the media, the fragment of the low bit rate received is decoded to ensure the score The reliability of the chip reception.
  • the first time threshold and the second time threshold may be determined in various manners including the following manners:
  • the first method is a preset duration, for example, pre-configured according to a network topology between the client 301 and the server 302.
  • the client 301 divides the number of fragment bytes included in the URL of the fragment in the MPD and the value of the bandwidth attribute of the fragment to which the fragment belongs, and calculates the length of time required to transmit the fragment, according to the calculation.
  • the obtained duration determines a first time threshold and a second time threshold;
  • the third method is determined according to the playing time of the segment.
  • the first time threshold and/or the second time threshold are set as the play duration of the slice, that is, the value of the duration attribute of the presentation to which the slice belongs, or set to half of the value;
  • the fourth mode is adjusted in real time according to the network situation between the client 301 and the server 302 obtained in real time (for example, network congestion, content acquisition duration, decoding duration, cache media playable time, etc.).
  • the attributes of the presentation to which the slice described in the MPD belongs can be found in Table 5 below.
  • the client 301 can switch the code rate by the manner shown in FIG. 5:
  • the client 301 finds in the MPD the presentation to which the slice currently being played belongs (herein referred to as "presentation 1" for clarity of description), according to the value of the startNumber attribute in the presentation 1 and the sequence number of the SegmentURL element corresponding to the slice. (Remarked as N), calculate that the serial number of the current fragment is startNumber+N; next find the presentations that belong to the same adaptation set and contain the same content and have different code rates, ready to switch past (denoted as "presentation” 2"), and find the fragment with the sequence number startNumber+N+1 (denoted as fragment N+1) in the presentation 2; subsequently, the client 301 requests the server 302 to present the fragment N+1 in the presentation 2.
  • presentation 1 the presentation to which the slice currently being played belongs
  • the streaming media transmission system and the rate switching scheme provided by the embodiments of the present disclosure are described in detail above.
  • the scheme of streaming media data transmission is introduced from the perspective of mutual cooperation between the client and the server, this does not mean that the client and the server must cooperate with each other.
  • the client and the server are implemented separately, they are respectively solved.
  • the problems that exist on the client side and the server side are only a combination of the two, which will give better technical results.
  • the following describes the client, the server, and the two types of streaming media data transmission methods provided by the embodiments of the present disclosure.
  • the principle of solving the technical problem is similar to the streaming media data transmission system provided by the embodiment of the present disclosure, and the implementation may refer to the system. The implementation, repetitions will not be repeated.
  • step S600 the client 301 requests the server 302 to deliver the media.
  • step S601 the client 301 receives the StreamInfo message sent by the server 302.
  • step S602 the client 301 determines whether the server 302 delivers multiple fragments of the same content and different code rates according to the indication information multipleRepresentation in the StreamInfo message. If the multipleRepresentation value is false, the client 301 determines that the server 302 only delivers a slice of the code rate, and then performs step S609; if the multipleRepresentation value is true, the client 301 determines that the server 302 delivers the same content and is different. A plurality of slices of the code rate are next performed in step S603.
  • step S603 the client 301 according to the indication information in the StreamInfo message
  • the bitratePriority determines whether the server 302 preferentially delivers the high code rate fragment. If the bitratePriority value is true, the client 301 determines that the server 302 preferentially delivers the high code rate fragment, and then performs step S611; if the bitratePriority value is false, the client 301 determines that the server 302 preferentially delivers the low code rate.
  • the fragmentation is followed by step S604.
  • step S604 the client 301 starts receiving fragments of low code rate.
  • step S605 the client 301 starts receiving fragments of a high code rate.
  • step S606 the client 301 determines whether the reception of the fragmentation of the high code rate is completed within the second time threshold after the requesting server 302 delivers the medium. If yes, go to step S615; otherwise, go to step S607.
  • step S607 the fragment of the received low code rate is decoded.
  • step S608 the slice is played.
  • the client 301 determines that the server 302 delivers only one code rate fragment according to the indication information multipleRepresentation in the StreamInfo message, the client 301 receives the media fragment delivered by the server 302 in step S609.
  • step S610 the received slice is decoded; next, step S608 is performed.
  • step S603 When the client 301 determines in step S603 that the server 302 preferentially delivers the fragment of the high code rate based on the indication information bitratePriority in the StreamInfo message, the client 301 starts to receive the fragment of the high code rate in step S611.
  • step S612 the client 301 determines whether the reception of the fragmentation of the high code rate is completed within the first time threshold after the requesting server 302 delivers the medium. If yes, go to step S615; otherwise, go to step S613.
  • step S613 the client 301 relinquishes receiving the fragment of the high code rate.
  • step S614 the client 301 starts receiving the fragment of the low code rate; next step S607 is performed.
  • the client 301 decodes the fragment of the received high code rate.
  • FIG. 7A is a schematic diagram of a first structure of a client according to an embodiment of the present disclosure. As shown in FIG. 7A, the client includes: a first processing module 701 and a second processing module 702.
  • the first processing module 701 determines whether the server delivers multiple fragments of the same content and different code rates.
  • the first processing module 702 determines whether the server preferentially transmits the fragment of the high code rate or preferentially transmits the low code rate. sheet.
  • the client may further include a first transceiver module 703, which is used to When the second processing module 702 determines that the server preferentially transmits the fragment of the high code rate, it starts to receive the fragment of the high code rate.
  • a first transceiver module 703 which is used to When the second processing module 702 determines that the server preferentially transmits the fragment of the high code rate, it starts to receive the fragment of the high code rate.
  • the second processing module 702 is further configured to: after the first transceiver module 703 starts to receive the high code rate fragment, determine whether the first transceiver module 703 is starting to receive the high. The reception of the fragment of the high code rate is completed within a first time threshold after the fragmentation of the code rate.
  • the second processing module 702 controls the first transceiver module 703 to abandon receiving the high code rate fragment, and Start receiving low-rate segments.
  • the second processing module 702 is further configured to: if the first transceiver module 703 completes the receiving of the high code rate fragment within the first time threshold, control the first transceiver module 703 to send the second indication information to the server.
  • the second indication information is used to notify the server that there is no need to send a low bit rate fragment.
  • the client may further include: a second transceiver module 704, configured to start receiving a low code rate when the second processing module 702 determines that the server preferentially transmits the low code rate fragment. Fragmentation, and after receiving the low bit rate fragment, continue to receive the high bit rate fragment.
  • a second transceiver module 704 configured to start receiving a low code rate when the second processing module 702 determines that the server preferentially transmits the low code rate fragment. Fragmentation, and after receiving the low bit rate fragment, continue to receive the high bit rate fragment.
  • the second transceiver module 704 does not directly decode the received fragment of the low code rate, but determines whether the pair is determined according to the control of the second processing module 702.
  • the received low bit rate slice is decoded.
  • the second processing module 702 is further configured to: after the second transceiver module 704 starts receiving the fragment of the high code rate, determine whether the second transceiver module 704 is beginning to receive the The reception of the fragment of the high code rate is completed within a second time threshold after the fragmentation of the high code rate.
  • the second processing module 702 controls the second transceiver module 704 to decode the fragmentation of the received low code rate.
  • the second processing module 702 is further configured to: if the second transceiver module 704 does not complete the receiving of the high code rate fragment within the second time threshold, control the second transceiver module 704 to abandon the high code rate. The reception of the fragments.
  • the client provided by the embodiment of the present disclosure may further have the structure shown in FIG. 7D, including a first processing module 701, a second processing module 702, a first transceiver module 703, and a second transceiver module 704, where
  • the processing of a processing module 701 and the second processing module 702 can refer to FIG. 7A above.
  • the second processing module 702 is specifically configured to: according to the first indication information sent by the server, used to indicate whether the server preferentially sends the fragment of the high code rate, It is judged whether the server preferentially transmits the fragment of the high bit rate or preferentially transmits the fragment of the low bit rate.
  • the first indication information is located in a StreamInfo message.
  • FIG. 8 is a schematic structural diagram of a server according to an embodiment of the present disclosure. As shown in FIG. 8, the server includes a processing module 801 and a transceiver module 802.
  • the processing module 801 determines whether to send the high code rate fragment preferentially when sending the same content and multiple fragments of different code rates to the client, or to preferentially send the low code rate fragment.
  • the transceiver module 802 sends the first indication information for indicating whether to send the high code rate fragment preferentially to the client.
  • the transceiver module 802 is further configured to: after sending the first indication information to the client, receive the second indication information sent by the client, where the second indication information is used to indicate that the server does not need to send the low code rate to the client again. And the fragmentation of the low bit rate is stopped to the client according to the second indication information.
  • FIG. 9 is a flowchart of a method for transmitting streaming media data according to an embodiment of the present disclosure.
  • the streaming media data transmission method shown in FIG. 9 can be applied to a client.
  • step S901 it is determined whether the server delivers multiple fragments of the same content and different code rates.
  • step S902 if the server delivers multiple fragments of the same content and different code rates, it is determined whether the server preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low code rate.
  • step S902 if it is determined in step S902 that the server preferentially transmits the fragment of the high code rate, the fragmentation of the high code rate is started.
  • the streaming media data transmission method further includes: determining whether the high code rate is completed within a first time threshold after starting to receive the high code rate fragment. Receiving the fragmentation; if it is judged that the reception of the fragment of the high code rate is not completed within the first time threshold, the fragment receiving the high code rate is discarded, and the fragmentation of the low code rate is started.
  • the streaming media data transmission method further includes: if it is determined that the receiving of the high code rate fragment is completed within the first time threshold, sending the second indication information, the second indication information to the server Used to inform the server that there is no need to send a low bit rate fragment.
  • step S902 if it is determined in step S902 that the server preferentially transmits the fragment of the low code rate, after receiving the fragment of the low code rate, the fragmentation of receiving the high code rate is continued.
  • the streaming media data transmission method further includes: not directly decoding the received low code rate fragment, but according to the high code rate fragmentation.
  • the receiving condition is used to determine whether to decode the received low bit rate fragment.
  • the streaming media data transmission method further includes: determining whether the high is completed within a second time threshold after starting to receive the high code rate fragment. Receive of the slice of the code rate; if the reception of the slice of the high code rate is not completed within the second time threshold, the slice of the low code rate received is decoded.
  • the streaming media data transmission method further includes: if the receiving of the high-rate rate fragment is not completed within the second time threshold, discarding the receiving of the high-rate rate fragment.
  • step S902 it is determined that the server preferentially transmits the fragment of the high code rate or the fragment of the low bit rate preferentially, and the method further includes: receiving, by the receiving server, the fragment indicating whether the server preferentially transmits the high bit rate.
  • the first indication information is determined, according to the received first indication information, whether the server preferentially transmits the fragment of the high code rate or preferentially transmits the fragment of the low code rate.
  • the first indication information is located in the StreamInfo message.
  • FIG. 10 is a flowchart of another streaming media data transmission method according to an embodiment of the present disclosure.
  • the streaming media data transmission method shown in FIG. 10 can be applied to a server.
  • step S1001 it is determined that when a plurality of fragments of the same content and different code rates are delivered to the client, the fragment of the high code rate is preferentially transmitted, or the fragment of the low bit rate is preferentially transmitted.
  • step S1002 the first indication information for indicating whether to preferentially transmit the fragment of the high code rate is sent to the client.
  • the streaming media data transmission method further includes: receiving second indication information sent by the client, where the second indication information is used to indicate that the server does not need to The terminal sends the fragment of the low bit rate; and stops sending the low bit rate fragment to the client according to the second indication information.
  • the embodiments of the present disclosure provide a streaming media data transmission apparatus, method, and system.
  • the client can determine whether the server preferentially transmits the high code rate fragment when sending the same content and multiple fragments of different code rates, or preferentially sends the low code rate fragment, and then determines that the server preferentially transmits the high code rate.
  • sharding there is no need to wait for the server to deliver a higher rate shard, but directly to the received shard Decoding reduces the delay of client decoding.
  • the client determines whether the server preferentially transmits the high-rate rate fragment or the low-rate rate fragment according to the fragmentation rate of the code rate sent by the server, or the client determines the server according to the first indication information sent by the server. Priority is given to sending high-rate shards or preferentially sending low-rate shards, providing two specific ways of client-side judgment.
  • the server is instructed to stop sending the low code rate fragment, which improves the efficiency of the server sending the fragment and saves the network transmission resource.
  • the client cannot complete the receiving of the high code rate fragment within the preset time threshold, the low bit rate fragment is received and decoded, thereby ensuring the reliability of the streaming media data transmission and the media playing. Coherence improves the user experience.
  • embodiments of the present disclosure can be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. Instructions are provided for implementation in the flowchart The steps of a process or a plurality of processes and/or block diagrams of a function specified in a block or blocks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Computer And Data Communications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Information Transfer Between Computers (AREA)
  • Communication Control (AREA)

Abstract

一种流媒体数据传输方法、客户端、以及服务器，用以解决客户端等待更高码率的分片，解码时延长的问题。本公开实施例提供的一种客户端包括：第一处理模块，用于判断服务器是否下发相同内容、不同码率的多个分片；第二处理模块，用于在所述第一处理模块判断出所述服务器下发相同内容、不同码率的多个分片的情况下，判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。由于客户端在判断出服务器优先发送高码率的分片时，无需再等待服务器下发更高码率的分片，而直接对接收的分片解码，因而缩短了客户端解码的时延。

Description

流媒体数据传输方法、客户端和服务器 技术领域
本公开涉及流媒体技术，尤其涉及一种流媒体数据传输装置、方法和系统。
背景技术
在采用传统的流媒体技术的情况下，传输的流媒体数据需要经过防火墙，需要专业的媒体服务器来支持流媒体技术，而且传统的流媒体技术的实现方式较为复杂。目前，已经出现了通过互联网传输流媒体数据的互联网流媒体技术，其不对现有互联网体系提出额外的要求，可通过对媒体文件的存储、信息描述方式进行修改，使得通过现有的HTTP协议传输流媒体数据。
移动图像专家组(Moving Pictures Experts Group，MPEG)制定的动态自适应流媒体(Dynamic Adaptive Streaming over HTTP，DASH)标准，简称MPEG DASH标准，提供了采用互联网流媒体技术传输流媒体数据的标准化方案。
MPEG DASH标准定义的媒体展现描述(Media Presentation Description，MPD)的层次结构模型如图1所示。
该层次结构模型中，时段(Period)用于描述可播放一段时间的媒体内容，次序相邻的时段描述的媒体内容在时间上是连续的。一个时段包含多个适配集合(Adaptation Set)，每个适配集合描述适配多个码率的媒体内容，每个码率对应一个呈现(Representation)。呈现描述媒体内容的具体封装格式、码率、编解码参数等信息。一个呈现包含多个分片(Segment)的统一资源定位符(Uniform Resoure Locator，URL)，用于指示对应的分片的存储位置，分片包含具体的媒体内容，即音频、视频、字幕、复用的音频和视频等。
基于上述MPEG DASH标准，通过互联网传输流媒体数据的可选方案是：首先建立WebSocket双向连接，通过WebSocket文本帧传输MPEG DASH标准的控制信息，通过WebSocket二进制帧传输分片。
图1所示的框架下客户端展现媒体内容的过程参见图2，在图1所示的框架下客户端展现媒体内容的过程包括如下步骤。
在步骤S201：客户端向服务器发送OpenMedia消息。客户端通过在该消 息中携带MPD的URL地址，向服务器指示需要播放的媒体。
在步骤S202：服务器向客户端发送MediaInfo消息，以告知客户端媒体的信息，比如：媒体是否已经就绪。
在步骤S203：客户端向服务器发送StartStream消息，以请求服务器下发媒体。
在步骤S204：服务器向客户端发送StreamInfo消息，以告知客户端媒体流的信息。
在步骤S205：服务器开始下发媒体分片。
客户端向服务器请求启动播放媒体后，服务器可通过图2中的步骤S204，在向客户端发送的StreamInfo消息中，携带用于指示服务器是否会下发相同内容、不同码率的多个呈现的指示信息multipleRepresentation。若该指示信息指示服务器会下发相同内容、不同码率的多个呈现，则服务器在发送分片时，会向客户端同时发送多个相同内容、不同码率的分片，即所述多个呈现包含的相同内容、不同码率的分片。
上述过程涉及的OpenMedia消息、MediaInfo消息、StartStream消息和StreamInfo消息可视为针对MPEG DASH标准在WebSocket协议上扩展的、并通过WebSocket的文本帧传输的消息。
但是，客户端并不知道服务器先发送的分片是高码率的还是低码率的。
已知的一种实现方式包括：客户端在收到一个分片后，在预设的等待时长内，判断是否收到更高码率的分片；若收到更高码率的分片，则对该更高码率的分片解码并展现；若未收到更高码率的分片，则对先前收到的分片进行解码并展现。
对于服务器先发送高码率的分片的情况，上述实现方式中，客户端收到高码率的分片后，由于不知道后续服务器发送的相同内容、不同码率的分片是更高码率的分片，还是更低码率的分片，则仍会等待上述预设的等待时长，增加了客户端解码的时延。
发明内容
本公开提供一种流媒体数据传输装置、方法和系统，用以解决客户端等待更高码率的分片，解码时延长的问题。
根据本公开的第一方面，提供了一种客户端，包括：第一处理模块，用 于判断服务器是否下发相同内容、不同码率的多个分片；第二处理模块，用于在所述第一处理模块判断出所述服务器下发相同内容、不同码率的多个分片的情况下，判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。
根据本公开的第二方面，提供了一种服务器，包括：处理模块，用于确定在向客户端下发相同内容、不同码率的多个分片的情况下优先发送高码率的分片、还是优先发送低码率的分片；收发模块，将用于指示是否优先发送高码率的分片的第一指示信息发给所述客户端。
根据本公开的第三方面，提供了一种流媒体数据传输方法，包括：判断服务器是否下发相同内容、不同码率的多个分片；若所述服务器下发相同内容、不同码率的多个分片，则判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。
根据本公开的第四方面，提供了另一种流媒体数据传输方法，包括：确定在向客户端下发相同内容、不同码率的多个分片的情况下优先发送高码率的分片、还是优先发送低码率的分片；将用于指示是否优先发送高码率的分片的第一指示信息发给所述客户端。
根据本公开的第五方面，提供了一种流媒体数据传输系统，包括客户端和服务器，其中，所述客户端，用于判断所述服务器是否下发相同内容、不同码率的多个分片，若所述服务器下发相同内容、不同码率的多个分片，则判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片；所述服务器，用于向所述客户端下发媒体分片。
综上，在本公开实施例中，客户端能够判断服务器优先发送高码率的分片、还是优先发送低码率的分片，因此可实现客户端在判断出服务器优先发送高码率的分片时，无需再等待服务器下发更高码率的分片，而直接对接收的分片解码，缩短了客户端解码的时延。
附图说明
图1为MPEG DASH标准定义的MPD层次结构的模型图；
图2为在图1所示的框架下客户端展现媒体内容的过程的示意性流程图；
图3为本公开实施例提供的流媒体数据传输系统的结构示意图；
图4为WebSocket的帧结构的示意图；
图5为码率切换方式示意图；
图6为本公开实施例一的客户端的流媒体数据传输的处理的流程图；
图7A为本公开实施例提供的客户端的第一种结构示意图；
图7B为本公开实施例提供的客户端的第二种结构示意图；
图7C为本公开实施例提供的客户端的第三种结构示意图；
图7D为本公开实施例提供的客户端的第四种结构示意图；
图8为本公开实施例提供的服务器的结构示意图；
图9为本公开实施例提供的一种流媒体数据传输方法的流程图；
图10为本公开实施例提供的另一种流媒体数据传输方法的流程图。
具体实施方式
本公开实施例提供一种流媒体数据传输装置和方法，用以解决客户端等待更高码率的分片，解码时延长的问题。
下面，结合附图对本公开实施例进行详细说明。首先，介绍本公开实施例提供的流媒体数据传输系统；然后分别介绍本公开实施例提供的客户端和服务器；最后介绍本公开实施例提供的两种流媒体数据传输方法。
图3为本公开实施例提供的流媒体数据传输系统的结构示意图。如图3所示，该系统包括：客户端301和服务器302。
客户端301判断服务器302是否下发相同内容、不同码率的多个分片。在判断服务器302下发相同内容、不同码率的多个分片的情况下，客户端301还进一步判断服务器302优先发送高码率的分片，还是优先发送低码率的分片
服务器302向客户端301下发媒体。
图3中，为了简单示意，仅示出了一个客户端301和一个服务器302，实际上，一个服务器302可向多个客户端301下发媒体，一个客户端301也可以分别向多个服务器302请求下发媒体。
本公开实施例中，可选地，客户端301与服务器302建立了WebSocket双向连接，基于建立的该WebSocket双向连接进行流媒体传输。客户端301可通过图2中步骤S203，请求服务器302下发媒体，即客户端301向服务器302发送StartStream消息，请求媒体流。
例如，客户端301可以在请求服务器302下发媒体之后并且在服务器302 向客户端301下发各分片之前，判断服务器302优先发送高码率的分片，还是优先发送低码率的分片。替代地，客户端301也可以在服务器302下发媒体的过程中，判断服务器302优先发送高码率的分片、还是优先发送低码率的分片。
具体地，服务器302可通过在向客户端301发送的StreamInfo消息中的multipleRepresentation键，向客户端301指示：服务器302是否下发相同内容、不同码率的多个分片。当该键取值为“true”时，向客户端301指示：服务器302下发相同内容、不同码率的多个分片。
目前，StreamInfo消息的格式如表1所示。
表1
Figure PCTCN2015081737-appb-000001
具体地，在客户端301在服务器302下发媒体的过程中判断服务器302优先发送高码率的分片、还是优先发送低码率的分片的情况下，客户端301可通过下述方式一或方式二判断服务器302优先发送高码率的分片、还是优先发送低码率的分片。在客户端301在服务器302下发各分片之前判断服务器302优先发送高码率的分片、还是优先发送低码率的分片的情况下，客户 端301可通过下述方式二判断服务器302优先发送高码率的分片、还是优先发送低码率的分片。
方式一、客户端301自身实现判断
若客户端301在请求服务器302下发媒体后收到服务器302发送的第一个分片的码率为预设的码率上限(比如：客户端301预先知道服务器302向客户端301下发的各分片的最高码率，则将该最高码率设置为预设的码率上限)，则客户端301判断出服务器302优先发送高码率的分片；若客户端301在请求服务器302下发媒体后收到服务器302发送的第一个分片的码率为预设的码率下限(比如：客户端301预先知道服务器302向客户端301下发的各分片的最低码率，则将该最低码率设置为预设的码率下限)，则客户端301判断出服务器302优先发送低码率的分片。
若客户端301在请求服务器302下发媒体后，先收到服务器302发送的高码率的分片，再收到服务器302发送的低码率的分片，则判断出服务器302优先发送高码率的分片；若先收到服务器302发送的低码率的分片，再收到服务器302发送的高码率的分片，则判断出服务器302优先发送低码率的分片。
具体地，若客户端301在请求服务器302下发媒体后收到服务器302发送的第一个分片的码率小于预设的码率上限且大于预设的码率下限，则服务器302等待接收下一个分片，若收到的下一个分片的码率比第一个分片的码率低，则判断出服务器302优先发送高码率的分片；若收到的下一个分片的码率比第一个分片的码率高，则判断出服务器302优先发送低码率的分片。
客户端301自身判断的方式有很多，以上仅为举例。
方式二、客户端301根据服务器302发送的第一指示信息判断
该第一指示信息用于指示服务器302是否优先发送高码率的分片。可选地，若客户端301与服务器302建立了WebSocket双向连接，基于建立的该WebSocket双向连接进行流媒体传输，可以修改表1所示的StreamInfo消息以便在StreamInfo消息中提供该第一指示信息。例如，如下面的表2所示，可以在表1所示的StreamInfo消息中增加键“bitratePriority”，并通过该键“bitratePriority”来提供该第一指示信息。例如，在表2中，“bitratePriority”位于键“multipleRepresentation”之后，实际实现时，位置不限于表2中所示的位置，比如，也可置于在整个消息的最后。
表2中“bitratePriority”的值为Boolean类型，取值为“true”表示服务器302优先发送高码率的分片，取值为“false”表示服务器302优先发送低码率的分片。在实际实现时，也可为其他类型，比如：数值型，取值为1时表示服务器302优先发送高码率的分片，取值为0时表示服务器302优先发送低码率的分片。具体实现方式不限于上述举例，只要能使客户端301判断出服务器302优先发送高码率的分片、还是优先发送低码率的分片即可。
表2
Figure PCTCN2015081737-appb-000002
可选地，为了实现客户端301可靠接收分片，在判断出服务器302优先发送高码率的分片时，在开始接收高码率的分片之后，客户端301还可判断在开始接收高码率的分片之后的第一时间阈值内是否完成对该高速率的分片的接收。若在该第一时间阈值内未完成对该高码率的分片的接收，客户端301 可以放弃接收该高码率的分片，并开始接收低码率的分片。
可选地，所述客户端301可以从开始接收高码率的分片之后开始计时，替代地，也可以在判断出服务器302优先发送高码率的分片之后开始计时。
可选地，若客户端301在请求服务器302下发媒体之后并且在服务器302向客户端301下发各分片之前，判断服务器302优先发送高码率的分片、还是优先发送低码率的分片，则客户端301还可以从请求服务器302下发媒体之后开始计时。
可选地，为了节约网络传输资源，提高客户端301与服务器302之间的分片传输效率，若客户端301在上述第一时间阈值内完成对该高码率的分片的接收，则客户端301向服务器302发送第二指示信息，该第二指示信息用于通知服务器302无需再发送低码率的分片。
可选地，若客户端301与服务器302建立了WebSocket双向连接，基于建立的该WebSocket双向连接进行流媒体传输，则客户端301可根据WebSocket帧中的指示信息判断是否完成对分片的接收。
WebSocket的帧结构如图4所示。其中，操作码占4比特(bit)，下面为操作码的取值与操作码表示的含义之间的对应关系：
0：与结束标记一起，表示分段帧的中间片段
1：文本帧
2：二进制帧
3～7：保留
8：关闭连接
9：Ping命令
A：Pong命令
B～F：保留
若要传输的数据过大，一个WebSocket二进制帧无法封装，则WebSocket协议规定，可以将待传输的数据分段：
对于未分段的帧，结束标记＝1操作码≠0
对于分段的帧：
第1个分段：结束标记＝0，操作码≠0
第2至第(N-1)个分段：结束标记＝0，操作码＝0
第N个分段，即最后一个分段：结束标记＝1，操作码＝0
客户端301在收到服务器302发送的包含StreamInfo消息的文本帧之后，开始接收服务器302发送的分片。若一个分片过大，则可将该分片划分为多个分段，所述多个分段可以分别置于多个WebSocket二进制帧中传输，具体地，在一个WebSocket二进制帧中传输一个分段。
服务器302将一个分片通过一个或多个二进制帧发送。客户端301在收到的二进制帧的结束标记为“0”，操作码不为“0”时，则确定当前分片是分段发送的(即一个分片通过多个二进制帧发送)，该二进制帧包含该分片的第一分段，且后续还需要接收包含该分片的其余分段的二进制帧。客户端301不断接收后续的二进制帧，直到收到的二进制帧的结束标记为“0”，操作码为“0”，则确定已完成对该分片的接收。
如前所述，若客户端301在第一时间阈值内完成对高码率的分片的接收，则客户端301向服务器302发送第二指示信息，第二指示信息用于通知服务器302无需再发送低码率的分片。可通过定义下列新的消息ReceiveReport，如表3所示，通过该新定义的消息发送该第二指示信息。
表3
Figure PCTCN2015081737-appb-000003
可选地，该消息可包含如下表4中列出的键值对。
表4
Figure PCTCN2015081737-appb-000004
Figure PCTCN2015081737-appb-000005
若在上述第一时间阈值内，客户端301完成对高码率的分片的接收，则客户端301向服务器发送上述ReceiveReport消息，该消息的ReceiveStatus为True。服务器302收到该消息之后，将不再向客户端301发送低码率的分片。
若在上述第一时间阈值内，客户端301未完成对高码率的分片的接收，则客户端301将向服务器302发送上述ReceiveReport消息，该消息的ReceiveStatus为False。服务器302将终止发送高码率的分片，并开始发送低码率的分片。
以上，具体描述了客户端301在确定服务器302优先发送高码率的分片的情况下的处理方案。下面，介绍客户端301在确定服务器302优先发送低码率的分片的情况下的处理方案。
若判断出服务器302优先发送低码率的分片，则客户端301在开始接收低码率的分片之后，继续接收高码率的分片。
可选地，客户端301判断是否存在多个高码率的分片。若存在多个高码率的分片，则客户端301根据与服务器302之间的网络带宽，从存在的多个高码率的分片中选择一个高码率的分片并开始接收。
客户端301在开始接收高码率的分片之后，判断是否在开始接收该高码率的分片之后的第二时间阈值内完成对该高码率的分片的接收。若客户端301判断在开始接收该高码率的分片之后的第二时间阈值内未完成对该高码率的分片的接收，则客户端301对接收完成的低码率的分片解码。
可选地，客户端301也可在判断出服务器302优先发送低码率的分片之后开始计时。
可选地，如前所述，若客户端301在请求服务器302下发媒体之后并且在服务器302向客户端301下发各分片之前，判断服务器302优先发送高码率的分片、还是优先发送低码率的分片，则客户端301还可以从请求服务器302下发媒体之后开始计时。
该可选方案中，客户端301在先接收到低码率的分片后，先不对所接收到的低码率的分片进行解码，而等待接收高码率的分片。若客户端301在请求服务器302下发媒体之后的第二时间阈值内，完成对该高码率的分片的接收，则对该高码率的分片进行解码，丢弃先前接收到的低码率的分片；若在请求服务器302下发媒体之后的第二时间阈值内未完成对该高码率的分片的接收，则对接收完成的低码率的分片进行解码，以保证分片接收的可靠性。
上述第一时间阈值、第二时间阈值，可通过包括下列方式在内的多种方式确定：
方式一、为预设的时长，比如：根据客户端301与服务器302之间的网络拓扑预先配置；
方式二、客户端301将MPD中分片的URL中包含的分片字节数和分片所属的呈现的bandwidth属性的取值相除，计算得到的传输该分片所需的时长，根据计算得到的该时长确定第一时间阈值和第二时间阈值；
方式三、根据分片的播放时长确定。比如：将该第一时间阈值和/或第二时间阈值设置为分片的播放时长，即分片所属的呈现的duration属性的值，或设置为该值的一半等；
方式四、根据实时获取的客户端301与服务器302之间的网络情况(比如：网络拥塞情况、内容获取时长、解码时长、缓存媒体可播放时间等)实时调整。
MPD中描述的分片所属的呈现的属性可参见下面的表5。
表5
Figure PCTCN2015081737-appb-000006
Figure PCTCN2015081737-appb-000007
本公开实施例中，客户端301可通过图5所示的方式进行码率的切换：
客户端301在MPD中找到当前正在播放的分片所属的呈现(为描述清楚起见，这里记为“呈现1”)，根据呈现1中的startNumber属性的值和分片对应的SegmentURL元素的排列序号(记为N)，计算得出当前分片的序号为startNumber+N；接下来找到属于同一个适配集合的、包含相同内容且具有不同码率的、准备切换过去的呈现(记为“呈现2”)，并在呈现2中找到序号为startNumber+N+1的分片(记为分片N+1)；随后，客户端301向服务器302请求呈现2中的分片N+1。
以上，详细介绍了本公开实施例提供的流媒体传输系统和码率切换的方案。尽管从客户端和服务器相互配合的角度，介绍了流媒体数据传输的方案，但这并不意味着客户端和服务器必须相互配合，实际上，当客户端和服务器分开实施时，也分别解决了在客户端侧和服务器侧所存在的问题，只是二者结合使用时，会获得更好的技术效果。
下面，分别介绍本公开实施例提供的客户端、服务器和两种流媒体数据传输方法，由于其解决技术问题的原理与本公开实施例提供的流媒体数据传输系统类似，其实施可参考该系统的实施，重复之处不再赘述。
下面将参考图6描述本公开实施例一的客户端301的流媒体数据传输的处理示例。
在步骤S600：客户端301请求服务器302下发媒体。
在步骤S601：客户端301接收服务器302发送的StreamInfo消息。
在步骤S602：客户端301根据StreamInfo消息中的指示信息multipleRepresentation，判断服务器302是否下发多个相同内容、不同码率的多个分片。若multipleRepresentation取值为false，则客户端301确定服务器302仅下发一个码率的分片，接下来执行步骤S609；若multipleRepresentation取值为true，则客户端301确定服务器302下发相同内容、不同码率的多个分片，接下来执行步骤S603。
在步骤S603：客户端301根据StreamInfo消息中的指示信息 bitratePriority，判断服务器302是否优先下发高码率的分片。若bitratePriority取值为true，则客户端301确定服务器302优先下发高码率的分片，接下来执行步骤S611；若bitratePriority取值为false，则客户端301确定服务器302优先下发低码率的分片，接下来执行步骤S604。
在步骤S604：客户端301开始接收低码率的分片。
在步骤S605：客户端301开始接收高码率的分片。
在步骤S606：客户端301判断在请求服务器302下发媒体之后的第二时间阈值内是否完成对高码率的分片的接收。若是，则执行步骤S615；否则执行步骤S607。
在步骤S607：对接收完成的低码率的分片进行解码。
在步骤S608：播放分片。
在客户端301在步骤S602根据StreamInfo消息中的指示信息multipleRepresentation，确定服务器302仅下发一个码率的分片的情况下，在步骤S609，客户端301接收服务器302下发的媒体分片。
然后，在步骤S610，对接收完成的分片进行解码；接下来执行步骤S608。
在客户端301在步骤S603根据StreamInfo消息中的指示信息bitratePriority，判断服务器302优先下发高码率的分片的情况下，在步骤S611：客户端301开始接收高码率的分片。
然后，在步骤S612：客户端301判断在请求服务器302下发媒体之后的第一时间阈值内是否完成对高码率的分片的接收。若是，则执行步骤S615；否则执行步骤S613。
在步骤S613：客户端301放弃接收高码率的分片。
在步骤S614：客户端301开始接收低码率的分片；接下来执行步骤S607。
在步骤S615：客户端301对接收完成的高码率的分片进行解码。
图7A为本公开实施例提供的客户端的第一种结构示意图。如图7A所示，该客户端包括：第一处理模块701以及第二处理模块702。
第一处理模块701判断服务器是否下发相同内容、不同码率的多个分片。
第二处理模块702在第一处理模块701判断出服务器下发相同内容、不同码率的多个分片的情况下，判断服务器优先发送高码率的分片、还是优先发送低码率的分片。
可选地，如图7B所示，该客户端还可包括第一收发模块703，其用于在 第二处理模块702判断出服务器优先发送高码率的分片时，开始接收高码率的分片。
可选地，对于图7B所示的客户端，第二处理模块702还用于：在第一收发模块703开始接收高码率的分片之后，判断第一收发模块703是否在开始接收该高码率的分片之后的第一时间阈值内完成对该高码率的分片的接收。
若第一收发模块703在第一时间阈值内未完成对该高码率的分片的接收，则所述第二处理模块702控制第一收发模块703放弃接收该高码率的分片，并开始接收低码率的分片。
进一步地，第二处理模块702还用于：若第一收发模块703在第一时间阈值内完成对该高码率的分片的接收，则控制第一收发模块703向服务器发送第二指示信息，第二指示信息用于通知服务器无需再发送低码率的分片。
可选地，如图7C所示，该客户端还可包括：第二收发模块704，其用于在第二处理模块702判断出服务器优先发送低码率的分片时，开始接收低码率的分片，并且在接收到低码率的分片之后，继续开始接收高码率的分片。
可选地，第二收发模块704在接收到该低码率的分片之后，不直接对接收的该低码率的分片进行解码，而是根据第二处理模块702的控制，确定是否对接收的该低码率的分片进行解码。
可选地，对于图7C所示的客户端，第二处理模块702还用于：在第二收发模块704开始接收该高码率的分片之后，判断第二收发模块704是否在开始接收该高码率的分片之后的第二时间阈值内完成对该高码率的分片的接收。
若第二收发模块704在第二时间阈值内未完成对该高码率的分片的接收，则第二处理模块702控制第二收发模块704对接收完成的低码率的分片解码。
进一步地，第二处理模块702还用于：若第二收发模块704在第二时间阈值内未完成对该高码率的分片的接收，则控制第二收发模块704放弃对该高码率的分片的接收。
可选地，本公开实施例提供的客户端还可具有图7D所示的结构，包括第一处理模块701、第二处理模块702、第一收发模块703和第二收发模块704，其中，第一处理模块701和第二处理模块702的处理可参考上述图7A～ 图7C所示的客户端中相同模块的处理；第一收发模块703与第二处理模块702之间的交互，可参考上述图7B所示的客户端中的处理，第二收发模块704与第二处理模块702之间的交互，可参考上述图7C所示的客户端中的处理，这里不再赘述。
可选地，对于图7A～图7D任一所示的客户端，第二处理模块702具体用于：根据服务器发送的用于指示服务器是否优先发送高码率的分片的第一指示信息，判断服务器优先发送高码率的分片、还是优先发送低码率的分片。
可选地，该第一指示信息位于StreamInfo消息中。
图8为本公开实施例提供的服务器的结构示意图。如图8所示，该服务器包括：处理模块801以及收发模块802。
处理模块801确定在向客户端下发相同内容、不同码率的多个分片时优先发送高码率的分片，还是优先发送低码率的分片。
收发模块802将用于指示是否优先发送高码率的分片的第一指示信息发给客户端。
可选地，收发模块802还用于：在将第一指示信息发给客户端之后，接收客户端发送的第二指示信息，第二指示信息用于指示服务器无需再向客户端发送低码率的分片；并且根据第二指示信息停止向客户端发送低码率的分片。
图9为本公开实施例提供的一种流媒体数据传输方法的流程图。如图9所示的所述流媒体数据传输方法可以应用于客户端。
在步骤S901：判断服务器是否下发相同内容、不同码率的多个分片。
在步骤S902：若所述服务器下发相同内容、不同码率的多个分片，则判断服务器优先发送高码率的分片，还是优先发送低码率的分片。
可选地，若步骤S902判断出服务器优先发送高码率的分片，则开始接收高码率的分片。
可选地，在开始接收高码率的分片之后，所述流媒体数据传输方法还包括：判断是否在开始接收该高码率的分片之后的第一时间阈值内完成对该高码率的分片的接收；若判断在第一时间阈值内未完成对该高码率的分片的接收，则放弃接收该高码率的分片，并开始接收低码率的分片。
可选地，所述流媒体数据传输方法还包括：若判断在第一时间阈值内完成对该高码率的分片的接收，则向服务器发送第二指示信息，第二指示信息 用于通知服务器无需再发送低码率的分片。
可选地，若步骤S902中判断出服务器优先发送低码率的分片，则在接收到低码率的分片之后，继续开始接收高码率的分片。
可选地，在接收到该低码率的分片之后，所述流媒体数据传输方法还包括：不直接对接收的该低码率的分片解码，而是根据高码率的分片的接收情况来判断是否对接收的该低码率的分片进行解码。
可选地，在开始接收该高码率的分片之后，所述流媒体数据传输方法还包括：判断是否在开始接收该高码率的分片之后的第二时间阈值内，完成对该高码率的分片的接收；若在第二时间阈值内未完成对该高码率的分片的接收，则对接收完成的低码率的分片进行解码。
可选地，所述流媒体数据传输方法还包括：若在第二时间阈值内未完成对该高码率的分片的接收，则放弃对该高码率的分片的接收。
可选地，在步骤S902判断服务器优先发送高码率的分片、还是优先发送低码率的分片，可以具体包括：接收服务器发送的用于指示服务器是否优先发送高码率的分片的第一指示信息；根据接收的第一指示信息，判断服务器优先发送高码率的分片、还是优先发送低码率的分片。
可选地，第一指示信息位于StreamInfo消息中。
图10为本公开实施例提供的另一种流媒体数据传输方法的流程图。如图10所示的流媒体数据传输方法可以应用于服务器。
在步骤S1001：确定在向客户端下发相同内容、不同码率的多个分片时，优先发送高码率的分片，还是优先发送低码率的分片。
在步骤S1002：将用于指示是否优先发送高码率的分片的第一指示信息发给客户端。
可选地，在步骤S1002将第一指示信息发给客户端之后，所述流媒体数据传输方法还包括：接收客户端发送的第二指示信息，第二指示信息用于指示服务器无需再向客户端发送低码率的分片；根据第二指示信息停止向客户端发送低码率的分片。
综上，本公开实施例提供一种流媒体数据传输装置、方法和系统。其中，客户端能够判断服务器在下发相同内容、不同码率的多个分片时优先发送高码率的分片、还是优先发送低码率的分片，则在判断出服务器优先发送高码率的分片时，无需再等待服务器下发更高码率的分片，而直接对接收的分片 解码，缩短了客户端解码的时延。
进一步地，客户端根据服务器发送的各码率的分片判断服务器优先发送高码率的分片、还是优先发送低码率的分片，或者客户端根据服务器下发的第一指示信息判断服务器优先发送高码率的分片、还是优先发送低码率的分片，提供了两种客户端判断的具体方式。
此外，若客户端在预设的时间阈值内能够完成对高码率的分片的接收，则指示服务器停止发送低码率的分片，提高了服务器发送分片的效率，节省了网络传输资源。
另一方面，若客户端在预设的时间阈值内无法完成对高码率的分片的接收，则接收低码率的分片并解码，保证了流媒体数据传输的可靠性和媒体播放的连贯性，提高了用户体验。
本领域内的技术人员应明白，本公开的实施例可提供为方法、系统、或计算机程序产品。因此，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本公开是参照根据本公开实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图 一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本公开的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本公开范围的所有变更和修改。
显然，本领域的技术人员可以对本公开进行各种改动和变型而不脱离本公开的精神和范围。这样，倘若本公开的这些修改和变型属于本公开权利要求及其等同技术的范围之内，则本公开也意图包含这些改动和变型在内。
本申请要求2015年1月16日提交的申请号为201510024568.1且发明名称为“一种流媒体数据传输装置、方法和系统”的中国优先申请的优先权，通过引用将其全部内容并入于此。

Claims (19)

  1. 一种客户端，包括：
    第一处理模块，用于判断服务器是否下发相同内容、不同码率的多个分片；
    第二处理模块，用于在所述第一处理模块判断出所述服务器下发相同内容、不同码率的多个分片的情况下，判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。
  2. 如权利要求1所述的客户端，还包括：
    第一收发模块，用于在所述第二处理模块判断出所述服务器优先发送高码率的分片的情况下，开始接收高码率的分片。
  3. 如权利要求2所述的客户端，其中，所述第二处理模块还用于：
    在所述第一收发模块开始接收高码率的分片之后，判断所述第一收发模块是否在开始接收该高码率的分片之后的第一时间阈值内完成对该高码率的分片的接收；
    若所述第一收发模块在所述第一时间阈值内未完成对该高码率的分片的接收，则控制所述第一收发模块放弃接收该高码率的分片，并开始接收低码率的分片。
  4. 如权利要求3所述的客户端，其中，所述第二处理模块还用于：
    若所述第一收发模块在所述第一时间阈值内完成对该高码率的分片的接收，则控制所述第一收发模块向所述服务器发送第二指示信息，所述第二指示信息用于通知所述服务器无需再发送低码率的分片。
  5. 如权利要求1所述的客户端，还包括：
    第二收发模块，用于在所述第二处理模块判断出所述服务器优先发送低码率的分片的情况下，开始接收低码率的分片，并且在接收到低码率的分片之后，继续开始接收高码率的分片。
  6. 如权利要求5所述的客户端，其中，所述第二处理模块还用于：在所述第二收发模块开始接收该高码率的分片之后，
    判断所述第二收发模块是否在开始接收该高码率的分片之后的第二时间阈值内完成对该高码率的分片的接收；
    若所述第二收发模块在所述第二时间阈值内未完成对该高码率的分片的 接收，则控制所述第二收发模块对接收完成的低码率的分片进行解码。
  7. 如权利要求6所述的客户端，其中，所述第二处理模块还用于：若所述第二收发模块在所述第二时间阈值内，未完成对该高码率的分片的接收，则控制所述第二收发模块放弃对该高码率的分片的接收。
  8. 如权利要求1～7任一项所述的客户端，其特征在于，
    所述第二处理模块具体用于：根据所述服务器发送的用于指示所述服务器是否优先发送高码率的分片的第一指示信息，判断所述服务器优先发送高码率的分片，还是优先发送低码率的分片。
  9. 一种服务器，包括：
    处理模块，用于确定在向客户端下发相同内容、不同码率的多个分片时，优先发送高码率的分片、还是优先发送低码率的分片；
    收发模块，将用于指示是否优先发送高码率的分片的第一指示信息发给所述客户端。
  10. 如权利要求9所述的服务器，其中，所述收发模块还用于：
    在将所述第一指示信息发给所述客户端之后，接收所述客户端发送的第二指示信息，所述第二指示信息用于指示所述服务器无需再向所述客户端发送低码率的分片；
    根据所述第二指示信息停止向所述客户端发送低码率的分片。
  11. 一种流媒体数据传输方法，包括：
    判断服务器是否下发相同内容、不同码率的多个分片；
    若所述服务器下发相同内容、不同码率的多个分片，则判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。
  12. 如权利要求11所述的方法，还包括：
    若所述服务器优先发送高码率的分片，则开始接收高码率的分片。
    判断是否在开始接收该高码率的分片之后的第一时间阈值内完成对该高码率的分片的接收；
    若在所述第一时间阈值内未完成对该高码率的分片的接收，则放弃接收该高码率的分片，并开始接收低码率的分片。
  13. 如权利要求12所述的方法，还包括：若在所述第一时间阈值内完成对该高码率的分片的接收，则向所述服务器发送第二指示信息，所述第二指示信息用于通知所述服务器无需再发送低码率的分片。
  14. 如权利要求11所述的方法，还包括：
    若所述服务器优先发送低码率的分片，则开始接收低码率的分片，并且在接收到低码率的分片之后，继续开始接收高码率的分片。
  15. 如权利要求14所述的方法，还包括：在开始接收该高码率的分片之后，
    判断是否在开始接收该高码率的分片之后的第二时间阈值内完成对该高码率的分片的接收；
    若在所述第二时间阈值内未完成对该高码率的分片的接收，则对接收完成的低码率的分片进行解码。
  16. 如权利要求15所述的方法，还包括：
    若在所述第二时间阈值内未完成对该高码率的分片的接收，则放弃对该高码率的分片的接收。
  17. 如权利要求11～16任一项所述的方法，其中，判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片，包括：
    接收所述服务器发送的用于指示所述服务器是否优先发送高码率的分片的第一指示信息；
    根据接收的所述第一指示信息，判断所述服务器优先发送高码率的分片、还是优先发送低码率的分片。
  18. 一种流媒体数据传输方法，包括：
    确定在向客户端下发相同内容、不同码率的多个分片时，优先发送高码率的分片，还是优先发送低码率的分片；
    将用于指示是否优先发送高码率的分片的第一指示信息发给所述客户端。
  19. 如权利要求18所述的方法，在将所述第一指示信息发给所述客户端之后，还包括：
    接收所述客户端发送的第二指示信息，所述第二指示信息用于指示所述服务器无需再向所述客户端发送低码率的分片；
    根据所述第二指示信息停止向所述客户端发送低码率的分片。
PCT/CN2015/081737 2015-01-16 2015-06-18 流媒体数据传输方法、客户端和服务器 WO2016112639A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2016569038A JP6563424B2 (ja) 2015-01-16 2015-06-18 ストリームメディアデータ伝送方法、クライアント及びサーバ
EP15837172.4A EP3247121B1 (en) 2015-01-16 2015-06-18 Streaming media data transmission method, client and server
US14/895,378 US10516712B2 (en) 2015-01-16 2015-06-18 Streaming media data transmission method, client and server
KR1020167031247A KR101978738B1 (ko) 2015-01-16 2015-06-18 스트리밍 미디어 데이터 전송 방법, 클라이언트 및 서버

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510024568.1A CN104581229B (zh) 2015-01-16 2015-01-16 一种流媒体数据传输装置、方法和系统
CN201510024568.1 2015-01-16

Publications (1)

Publication Number Publication Date
WO2016112639A1 true WO2016112639A1 (zh) 2016-07-21

Family

ID=53096280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/081737 WO2016112639A1 (zh) 2015-01-16 2015-06-18 流媒体数据传输方法、客户端和服务器

Country Status (6)

Country Link
US (1) US10516712B2 (zh)
EP (1) EP3247121B1 (zh)
JP (1) JP6563424B2 (zh)
KR (1) KR101978738B1 (zh)
CN (1) CN104581229B (zh)
WO (1) WO2016112639A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108768952A (zh) * 2018-05-03 2018-11-06 深圳市网心科技有限公司 一种流媒体传输方法、客户端、服务器和存储介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104581229B (zh) 2015-01-16 2018-08-03 京东方科技集团股份有限公司 一种流媒体数据传输装置、方法和系统
CN107566855B (zh) * 2016-06-30 2020-11-10 华为技术有限公司 频道快速切换的方法、服务器和机顶盒
CN111083094B (zh) * 2018-10-22 2022-06-07 中国移动通信有限公司研究院 一种流媒体的码流切换方法及装置、计算机存储介质
CN109672887B (zh) * 2019-03-06 2021-04-09 北京奇艺世纪科技有限公司 一种视频编码方法及装置
US11076188B1 (en) * 2019-12-09 2021-07-27 Twitch Interactive, Inc. Size comparison-based segment cancellation
CN111600937A (zh) * 2020-04-28 2020-08-28 上海翌旭网络科技有限公司 基于自适应码率的流协议文件传输方法及系统
US11153581B1 (en) 2020-05-19 2021-10-19 Twitch Interactive, Inc. Intra-segment video upswitching with dual decoding
CN114071241A (zh) * 2020-08-06 2022-02-18 成都鼎桥通信技术有限公司 视频播放方法、系统、电子设备及存储介质
US11770592B2 (en) 2021-06-28 2023-09-26 Synamedia Limited Intrasegment adjustment of video transmission rate
CN113473183B (zh) * 2021-06-29 2023-05-05 华夏城视网络电视股份有限公司 一种应用于融合媒体的动静态媒体流批处理方法
CN115002519A (zh) * 2022-05-31 2022-09-02 北京势也网络技术有限公司 一种在低带宽网络下播放8k全景视频文件的方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101227590A (zh) * 2007-01-19 2008-07-23 北京风行在线技术有限公司 基于p2p协议的媒体文件点播控制方法及装置
CN101471919A (zh) * 2007-12-29 2009-07-01 突触计算机系统（上海）有限公司 基于点对点传输协议的设备中用于下载分片的方法及装置
CN102307302A (zh) * 2011-07-06 2012-01-04 杭州华三通信技术有限公司 一种保持视频图像连续性的方法和装置
CN102801690A (zh) * 2011-05-25 2012-11-28 华为技术有限公司 流媒体的处理方法、分发服务器、客户端及系统
CN104202618A (zh) * 2014-09-26 2014-12-10 三星电子（中国）研发中心 获取播放资源的方法、代理客户端、代理服务器和系统
CN104581229A (zh) * 2015-01-16 2015-04-29 京东方科技集团股份有限公司 一种流媒体数据传输装置、方法和系统

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100453505B1 (ko) 2002-04-03 2004-10-20 주식회사 케이티프리텔 무선 네트워크 클라이언트에서 스트리밍 데이터의 멀티비트 레이트 기능 제어 방법 및 그 장치
US7725557B2 (en) * 2002-06-24 2010-05-25 Microsoft Corporation Client-side caching of streaming media content
JP2007214675A (ja) * 2006-02-07 2007-08-23 Hitachi Ltd デジタル放送受信装置
US8230288B2 (en) * 2006-10-18 2012-07-24 Samsung Electronics Co., Ltd. Data transmission apparatus and method for applying an appropriate coding rate
KR100931344B1 (ko) 2007-08-31 2009-12-11 에스케이 텔레콤주식회사 Ｖｏｄ 스트리밍 서비스를 제공하는 방법과 그를 위한시스템, 서버 및 사용자 단말기
JP5245452B2 (ja) * 2008-02-26 2013-07-24 富士通株式会社 無線基地局、端末、および上位装置
WO2009155356A1 (en) * 2008-06-18 2009-12-23 Onion Networks, KK Traffic and cost containment for internet access by adapting the coding rate when distributing- media content
JP5347441B2 (ja) 2008-11-10 2013-11-20 日本電気株式会社 動画像処理装置
US9166746B2 (en) * 2009-03-24 2015-10-20 Samsung Electronics Co., Ltd. Operating method and apparatus according to data duplicate retransmission in mobile communication system
US8817684B2 (en) 2010-12-17 2014-08-26 Verizon Patent And Licensing Inc. Adaptive mobile multicasting for wireless networks
CN102143384B (zh) * 2010-12-31 2013-01-16 华为技术有限公司 一种媒体文件生成方法、装置及系统
US20120324122A1 (en) * 2011-06-20 2012-12-20 David Miles Method and apparatus for server-side adaptive streaming
US20130170561A1 (en) * 2011-07-05 2013-07-04 Nokia Corporation Method and apparatus for video coding and decoding
US9712891B2 (en) * 2011-11-01 2017-07-18 Nokia Technologies Oy Method and apparatus for selecting an access method for delivery of media
EP2899988A4 (en) * 2012-09-20 2016-03-09 Sony Corp RECEIVING DEVICE, SENDING / RECEIVING SYSTEM, RECEIVING METHOD AND PROGRAM
US9866886B2 (en) * 2012-10-23 2018-01-09 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for distributing a media content service
CN103248962B (zh) * 2013-04-23 2016-12-28 华为技术有限公司 获取流媒体数据的方法、设备及系统
CN103747283B (zh) * 2013-12-24 2017-02-01 中国科学院声学研究所 视频分片的下载方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101227590A (zh) * 2007-01-19 2008-07-23 北京风行在线技术有限公司 基于p2p协议的媒体文件点播控制方法及装置
CN101471919A (zh) * 2007-12-29 2009-07-01 突触计算机系统（上海）有限公司 基于点对点传输协议的设备中用于下载分片的方法及装置
CN102801690A (zh) * 2011-05-25 2012-11-28 华为技术有限公司 流媒体的处理方法、分发服务器、客户端及系统
CN102307302A (zh) * 2011-07-06 2012-01-04 杭州华三通信技术有限公司 一种保持视频图像连续性的方法和装置
CN104202618A (zh) * 2014-09-26 2014-12-10 三星电子（中国）研发中心 获取播放资源的方法、代理客户端、代理服务器和系统
CN104581229A (zh) * 2015-01-16 2015-04-29 京东方科技集团股份有限公司 一种流媒体数据传输装置、方法和系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3247121A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108768952A (zh) * 2018-05-03 2018-11-06 深圳市网心科技有限公司 一种流媒体传输方法、客户端、服务器和存储介质

Also Published As

Publication number Publication date
EP3247121B1 (en) 2024-02-14
EP3247121A4 (en) 2018-07-18
CN104581229B (zh) 2018-08-03
JP6563424B2 (ja) 2019-08-21
KR101978738B1 (ko) 2019-05-15
KR20160145657A (ko) 2016-12-20
EP3247121A1 (en) 2017-11-22
US10516712B2 (en) 2019-12-24
US20160212189A1 (en) 2016-07-21
CN104581229A (zh) 2015-04-29
JP2018509010A (ja) 2018-03-29

Similar Documents

Publication Publication Date Title
WO2016112639A1 (zh) 流媒体数据传输方法、客户端和服务器
US11706272B2 (en) Adaptive streaming with early client indication
CN108848060B (zh) 一种多媒体文件处理方法、处理系统及计算机可读存储介质
TWI623226B (zh) 用於儲存媒體片段之基於目錄限制之系統及方法
WO2016029804A1 (zh) 一种视频播放方法、媒体设备、播放设备以及多媒体系统
WO2016011823A1 (zh) 获取直播视频切片的方法、服务器及存储介质
US20140095593A1 (en) Method and apparatus for transmitting data file to client
WO2018133601A1 (zh) 一种流媒体传输方法、装置、服务器及终端
JP7496022B2 (ja) クライアント、サーバ、受信方法及び送信方法
WO2015169172A1 (zh) 网络视频播放的方法和装置
CN104168486A (zh) 基于云计算的虚拟机与客户端间视频重定向方法
JP2013509743A (ja) コンテンツストリームを個別化する方法およびシステム
WO2018001184A1 (zh) 频道快速切换的方法、服务器和机顶盒
WO2018103696A1 (zh) 媒体文件的播放方法、服务端、客户端及系统
WO2015192683A1 (zh) 一种基于码流自适应技术的内容分发方法、装置及系统
WO2013185514A1 (zh) 一种播放流媒体的系统和方法
WO2011054319A1 (zh) 一种在httpstreaming系统中实现分层请求内容的方法，装置和系统
WO2022262858A1 (zh) 图像传输方法、图像显示及处理设备、及图像传输系统
WO2016112641A1 (zh) 客户端、流媒体数据接收方法和流媒体数据传输系统
WO2016197955A1 (zh) 多媒体流组播方法和装置
WO2011143916A1 (zh) 媒体适配的方法和装置
CN108989905A (zh) 媒体流控制方法、装置、计算设备及存储介质
CN113905025B (zh) 一种传输流数据的方法、装置、介质及计算机设备
WO2022002070A1 (zh) 媒体流的自适应实时递送方法及服务器
WO2020216035A1 (zh) 媒体流的实时推送方法、实时接收方法、服务器及客户端

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 14895378

Country of ref document: US

REEP Request for entry into the european phase

Ref document number: 2015837172

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015837172

Country of ref document: EP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15837172

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20167031247

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2016569038

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE