WO2018171567A1

WO2018171567A1 - Method, server, and terminal for playing back media stream

Info

Publication number: WO2018171567A1
Application number: PCT/CN2018/079520
Authority: WO
Inventors: 杨生飞; 王赵淮; 王伟; 姜立科; 曹阳
Original assignee: 华为技术有限公司
Priority date: 2017-03-21
Filing date: 2018-03-20
Publication date: 2018-09-27
Also published as: CN108632681B; CN108632681A

Abstract

The present application relates to the field of communications. Disclosed are a method, server, and terminal for playing back a media stream. The method comprises: a terminal transmits an acquisition request carrying a fragment identifier of a video stream fragment, the video stream fragment being a video stream fragment that a video stream received by the terminal and transmitted by the server comprises; the server acquires the starting time of the video stream on the basis of the fragment identifier of the video stream fragment; the server acquires an audio stream corresponding to the video stream on the basis of the starting time; the server transmits the audio stream to the terminal; and the terminal receives the audio stream and plays back the video stream and the audio stream. The present application solves the problem in the prior art in which either audio cannot be played back by a terminal or the audio played is inconsistent with video picture and ensures the consistency between the audio and video picture played back by the terminal.

Description

Method, server and terminal for playing media stream

This application claims the priority of the Chinese Patent Application filed on March 21, 2017, the Chinese Patent Office, Application No. 201710172615.6, entitled "Methods, Servers and Terminals for Playing Media Streams", the entire contents of which are incorporated herein by reference. In the application.

Technical field

The present application relates to the field of communications, and in particular, to a method, a server, and a terminal for playing a media stream.

Background technique

In an Internet Protocol Television (IPTV) system, a user can watch a television program using a terminal such as a television, a mobile phone, or a tablet. When a user watches a television program of a certain channel, it is often necessary to switch to another channel to watch the television program of the other channel. For convenience of description, the other channel is referred to as a target channel, and the terminal can obtain the media stream of the target channel for playing by the following manner.

At present, there is a scene of adaptive code stream, in which the terminal can switch to the target channel, and the scene defines that the video stream of each channel includes two types of high-rate video stream and low-rate video stream, and fast channel switching. (Fast Channel Change, FCC) The server caches the high-rate video stream and the low-rate video stream of each channel sent by the video source, so that when the terminal switches to the target channel, the FCC server can select a video stream according to requirements. Provided to the terminal for playback. The handover process is as follows: after receiving the play request sent by the terminal for switching to the target channel, in order to enable the terminal to quickly play the TV program of the target channel, the FCC server first sends the low-rate video stream of the target channel to the terminal, and sends a segment. After the time, in order to improve the quality of the playback picture of the terminal, the high-rate video stream of the target channel is sent to the terminal and the terminal is notified to join the multicast group of the target channel. The terminal plays the high-rate video stream sent by the FCC server, and joins the multicast group of the target channel, and plays the audio stream and the video stream after receiving the audio stream and the video stream sent by the multicast group.

In the above scenario of the adaptive code stream, the FCC server sends the video stream to the terminal, and the audio stream is not involved, which may cause the terminal to play the sound or the played sound is inconsistent with the video picture.

Summary of the invention

In order to solve the problem in the prior art that the terminal may not play the sound or the played sound is inconsistent with the video picture, the embodiment of the present application provides a method, a server, and a terminal for playing the media stream. The technical solution is as follows:

A first aspect provides a method for playing a media stream, the method comprising: receiving, by a server, an acquisition request sent by a terminal, where the acquisition request carries a fragmentation identifier of a video stream fragment in a video stream that has been sent to the terminal; The server determines, according to the fragment identifier, a start time of the video stream that has been sent to the terminal, and obtains an audio stream corresponding to the video stream according to the start time; the server sends the audio stream to the terminal, so that the terminal plays the video. Stream and the audio stream. After receiving the acquisition request sent by the terminal, the server can determine the audio stream corresponding to the video stream that has been sent to the terminal according to the fragment identifier carried in the acquisition request, and the audio stream has the same start time as the video stream. Therefore, when the terminal simultaneously plays the video stream and the audio stream, the audio and video can be synchronized, and the problem that the terminal does not play the sound or the played sound is inconsistent with the video picture is solved in the related art.

In a possible implementation manner of the first aspect, the video stream sent by the server to the terminal includes description information of the at least one audio stream, so that the terminal selects, from the audio stream, the description information of the audio stream that matches the capability of the terminal, so that the terminal The obtaining request sent by the server further includes description information of the one audio stream; after receiving the obtaining request, the server determines, according to the fragment identifier in the obtaining request, a start time of the video stream that has been sent to the terminal; The description information of the one audio stream in the request is determined to determine an audio stream, and the audio stream corresponding to the video stream is obtained from the determined audio stream according to the start time. In this way, the terminal can acquire an audio stream that matches its own capabilities for playback according to its own capabilities.

In a possible implementation manner of the first aspect, the server sends a Real-time Transport Control Protocol (RTCP) packet to the terminal before receiving the acquisition request that is sent by the terminal and carrying the fragment identifier. The fragment timestamp field of the packet carries the fragment identifier. The server sends an RTCP packet to the terminal, where the fragment timestamp field of the RTCP packet carries the fragment identifier, so that the terminal sends an acquisition request carrying the fragment identifier to the server, where the acquisition request is used to request the server to acquire the terminal. The audio stream corresponding to the received video stream.

In a possible implementation manner of the first aspect, the video stream that the server has sent to the terminal is the first video stream of the first code rate, the method further includes: stopping, by the server, sending the first video stream to the terminal and When the terminal sends the second video stream of the second code rate, the first code rate is smaller than the second code rate, and the terminal sends a notification message, where the notification message carries the sequence number of the last data packet of the first video stream and the second video stream. The sequence number of the first packet. After receiving the last data packet of the first video stream, the terminal receives the first data packet of the second video stream, and the sequence number of the last data packet is not continuous with the sequence number of the first data packet. The terminal avoids the erroneous determination of the packet loss, stops playing the second data stream, and the server sends a notification message to the terminal, so that the terminal determines, according to the notification message, that the last data packet and the first data packet are consecutive data packets, so that the normal The first packet begins to play the second video stream.

In a possible implementation manner of the first aspect, the notification message sent by the server to the terminal is a server terminal notification (SCN) message, where the SCN message includes an old sequence number field and a new sequence number field, and the old sequence number The field carries the sequence number of the last data packet of the first video stream sent by the server to the terminal, and the new sequence number field carries the sequence number of the first data packet of the second video stream sent by the server to the terminal.

A second aspect provides a method for playing a media stream, the method comprising: the terminal sending an acquisition request to the server, where the acquisition request carries a fragmentation identifier of the video stream fragment in the received video stream, so that the server according to the Acquiring to send an audio stream corresponding to the video stream, the start time of the audio stream is the same as the start time of the video stream; the terminal receives the audio stream sent by the server, and plays the video stream and the audio stream it receives. Since the video stream and the start time of the audio stream are the same, the terminal can ensure the audio and video synchronization when playing the video stream and the audio stream at the same time, and solves the problem that the terminal does not play the sound or the played sound is inconsistent with the video picture in the related art. problem.

In a possible implementation manner of the second aspect, the terminal, the video stream received by the terminal, includes the description information of the at least one audio stream, and the terminal further selects, from the description information, the description information of the audio stream that matches the capability of the first channel. The acquisition request sent to the server further includes description information of the one audio stream to obtain an audio stream corresponding to its own capability from the server for playing.

In a possible implementation manner of the second aspect, the terminal receives the RTCP packet sent by the server, and obtains the fragment timestamp field of the RTCP packet to carry the terminal, before sending the acquisition request that carries the fragment identifier to the server. The fragment identifier of the video stream fragment in the received video stream, the fragment identifier is added to the acquisition request, and after receiving the acquisition request, the server sends the terminal identifier to the terminal according to the fragment identifier. The video stream corresponds to the audio stream.

In a possible implementation manner of the second aspect, the video stream received by the terminal is the first video stream of the first code rate, and the terminal further receives the second video stream of the second code rate sent by the server, the second code rate. The terminal further receives the notification message sent by the server, where the notification message carries two serial numbers, one of which is the serial number of the last data packet of the first video stream, and the other serial number is the second video stream. The sequence number of the first data packet; the second video stream it receives is played according to the notification message. After receiving the last data packet of the first video stream, the terminal receives the first data packet of the second video stream, and the sequence number of the last data packet is not continuous with the sequence number of the first data packet, to avoid the terminal. If the data packet is lost, the second data stream is stopped, and the server sends a notification message to the terminal, where the notification message includes the sequence number of the last data packet and the sequence number of the first data packet. The terminal determines, according to the notification message, that the last data packet and the first data packet are consecutive data packets, so that the second video stream is normally played from the first data packet.

In a possible implementation manner of the second aspect, the notification message sent by the server received by the terminal is an SCN message, where the SCN message includes an old sequence number field and a new sequence number field, and the old sequence number field carries the terminal received from the server. The sequence number of the last data packet of the first video stream, and the new sequence number field carries the sequence number of the first data packet of the second video stream received by the terminal from the server.

A third aspect provides an apparatus for playing a media stream, the apparatus comprising at least one unit, the at least one unit configured to implement the play media stream provided by the foregoing first aspect or any one of the possible implementation manners of the first aspect Methods.

A fourth aspect provides an apparatus for playing a media stream, the apparatus comprising at least one unit, the at least one unit configured to implement the play media stream provided by any one of the foregoing second aspect or the second aspect Methods.

In a fifth aspect, a server is provided, the server comprising: a processor and a network port, the processor being configured to implement the playback provided by any one of the first aspect or the first aspect of the first aspect by executing an instruction The method of media streaming.

In a sixth aspect, a terminal is provided, the terminal comprising: a processor and a network port, the processor being configured to implement the playback provided by any one of the foregoing second aspect or the second aspect by executing an instruction The method of media streaming.

The seventh aspect provides a system for playing a media stream, the system comprising the server provided by the third aspect or the fifth aspect, and the terminal provided by the fourth aspect or the sixth aspect.

In an eighth aspect, a computer readable medium storing instructions for implementing a playback media stream provided by the first aspect or any one of the possible implementations of the first aspect, or the computer The readable medium stores instructions for playing a media stream provided by any one of the possible implementations of the second aspect or the second aspect.

DRAWINGS

1 is a schematic structural diagram of an IPTV system according to an exemplary embodiment of the present invention;

FIG. 2 is a schematic diagram of video stream segmentation and audio stream segmentation according to an exemplary embodiment of the present invention; FIG.

FIG. 3 is a schematic structural diagram of a server according to an exemplary embodiment of the present invention; FIG.

FIG. 4 is a schematic structural diagram of a terminal according to an exemplary embodiment of the present invention; FIG.

FIG. 5 is a flowchart of a method for playing a media stream according to an exemplary embodiment of the present invention; FIG.

FIG. 6 is a schematic structural diagram of an RTCP packet according to an exemplary embodiment of the present invention;

FIG. 7 is a schematic structural diagram of an SCN message according to an exemplary embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a server according to another exemplary embodiment of the present invention; FIG.

FIG. 9 is a schematic structural diagram of a terminal according to another exemplary embodiment of the present invention.

detailed description

In order to make the objects, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

Please refer to FIG. 1 , which is a schematic structural diagram of an IPTV system according to an exemplary embodiment of the present invention. The IPTV system includes: a source server 110, an encoding server 120, a multicast server 130, an FCC server 140, and a terminal 150.

The source server 110 can be a server, or a server cluster consisting of several servers, or a cloud computing service center. The source server 110 includes a media stream of each of at least one channel, the media stream includes a second video stream and an audio stream corresponding to the second video stream, and the like, and may further include a subtitle stream or the like. The source server 110 is connected to the encoding server 120 via a wired network or a wireless network, and the media stream of each channel can be transmitted to the encoding server 120. Among them, the channel may include a television channel, a live channel, a carousel channel, and the like.

Generally, the media stream in the source server 110 is previously created by the technician and uploaded to the source server 110. When the technician creates the media stream, the second video stream and the at least one audio stream corresponding to the second video stream may be created, and the description information of each audio stream is set, and the created second video stream includes each path set. Description of the audio stream. The description information of the audio stream may include information such as a playing language, a stream identifier, and a code rate corresponding to the audio stream of the channel. For example, when the second video stream A is created, the technician creates two audio streams for the second video stream A, one of which is an audio stream in which the language is Chinese, and the other is The audio stream in which the language is English is played, and the created second video stream A includes description information of the two audio streams.

Among them, the second video stream usually produced by a technician has a higher code rate, so the resolution of the second video stream is higher. For convenience of explanation, the code rate of the second video stream is referred to as the second code rate.

The encoding server 120 receives the media stream of each channel transmitted by the source server 110. For the media stream of each channel, the encoding server 120 uses an Adaptive Bit Rate (ABR) encoding technique to down-convert the second video stream included in the media stream of the channel to generate at least one first video stream, the at least one The first code rate of each of the first video streams in the first video stream is different from each other, and is smaller than the second code rate of the second video stream. Therefore, each channel corresponds to one second video stream, at least one first video stream, and at least one audio stream. Since each first video stream is obtained by downsampling the second video stream, the resolution of the first video stream is smaller than the resolution of the second video stream.

Optionally, the encoding server 120 stores at least one preset first code rate, and the second video stream may be down-coded according to the stored at least one first code rate to obtain at least one first video stream.

The encoding server 120 is connected to the multicast server 130 through a wired network or a wireless network, and the encoding server 120 transmits the media stream of each channel to the multicast server 130, and the media stream of each channel transmitted to the multicast server 130 includes its source. In addition to the content of the second video stream and the at least one audio stream, the server 110 further includes at least one first video stream obtained by down-converting the second video stream.

The multicast server 130 can be a server, or a server cluster consisting of several servers, or a cloud computing service center. The multicast server 130 is connected to the FCC server 140 and the terminal 150 via a wired network or a wireless network, respectively.

The multicast server 130 receives the media stream of each channel sent by the encoding server 120, and forwards the received media stream of each channel to the FCC server 140 in real time. The multicast server 130 also maintains a multicast group for each channel, and transmits the media stream of the channel to the terminal 150 located in the multicast group.

The FCC server 140 can be a server, or a server cluster consisting of several servers, or a cloud computing service center. The FCC server 140 is connected to the terminal 150 through a wired network or a wireless network.

The terminal 150 may include a smart TV, a set top box, a smart phone, a tablet, a smart TV, a laptop portable computer, a desktop computer, etc., and transmits a channel identifier carrying the target channel to the FCC server 140 at startup or when switching channels. The play request requests the FCC server 140 to send the media stream of the target channel.

The FCC server 140 is configured to: after receiving the play request by the terminal 150, send, according to the channel identifier of the target channel carried by the play request, the first video stream corresponding to the target channel and the video stream included in the first video stream to the terminal 150. The fragmentation identifier of the fragment is received by the receiving terminal 150, and the audio stream corresponding to the first video stream is obtained according to the fragment identifier, and the audio stream is sent to the terminal 150.

The terminal 150 is configured to receive the first video stream sent by the FCC server 140 and the fragment identifier, and send an acquisition request carrying the fragment identifier to the FCC server 140, and the receiving FCC server 140 sends the audio stream corresponding to the first video stream. The first video stream and the audio stream are played. Since the first code rate of the first video stream is lower than the second code rate, the terminal 150 can quickly play the first video stream.

At some time after the start of the transmission of the audio stream, the FCC server 140 transmits a second video stream corresponding to the target channel to the terminal 150. Since the second code rate of the second video stream is greater than the first code rate of the first video stream, the quality of the video picture played by the terminal 150 can be improved.

The terminal 150 is further configured to receive the second video stream of the target channel sent by the FCC server 140 and play the second video stream, and also apply to join the multicast group corresponding to the target channel in the multicast server 130, and receive the multicast server. The second video stream of the target channel is sent by 130; when the second video stream sent by the FCC server 140 is the same as the second video stream sent by the multicast server 130, the second video stream sent by the multicast server 130 is played.

It should be noted that, when receiving the media stream of a certain channel sent by the source server 110, the encoding server 120 may further slice the second video stream included in the media stream, so that the second video stream includes multiple video streams. Fragmentation, which generates a fragmentation identifier for each video stream fragment. In addition, each of the first video streams obtained by the encoding server 120 down-converting the second video stream is also composed of video stream fragments.

Each video stream slice in the second video stream corresponds to one video stream slice in the first video stream. For the video stream segmentation of the second video stream, and the video stream segmentation corresponding to the one video stream segment in the first video stream, the start time, the video content, and the score of the two video stream segments The slice identifiers are all the same, but the resolutions of the two video stream fragments are different and the number of data packets included is different. The video stream fragments located in the second video stream include more data packets than the first video stream. The number of packets included in the video stream fragment. Each video stream segment located in the second video stream includes a first frame that is a key frame for decoding the video stream, and a first frame included in each video stream segment of the first video stream is also used for decoding. The key frame of the video stream. The data packet can be an RTP data packet.

Referring to FIG. 2, the following four video stream fragments S11, S12, S13, and S14 in the first video stream are used as an example. The video stream fragment S11 is corresponding to the video stream fragment in the second video stream. The fragment identifiers of S11 and S21 are both "1"; the video stream fragment S12 is corresponding to the video stream fragment in the second video stream, and the fragment identifiers of S12 and S22 are both "2"; the video stream is fragmented. S13, in the second video stream, the corresponding video stream fragment is S23, and the fragment identifiers of S13 and S23 are both “3”; and the corresponding video stream fragment of the video stream fragment S14 in the second video stream is S24. The slice identifier of S14 at S24 is "4".

Each video slice in the second video stream includes a greater number of data packets than the number of data packets included in the corresponding video stream segment of the video slice in a video stream. For example, referring to FIG. 2, the number of data packets included in the video stream segment S11 in the first video stream is 2, which is smaller than the number 4 of data packets included in the video stream segment S21.

In addition, the encoding server 120 may further slice the audio stream included in the media stream, and the audio stream includes a plurality of audio stream fragments, and generate a fragment identifier for the audio stream fragment. Each of the first video stream and the second video stream corresponds to an audio stream fragment in the audio stream, and the video stream fragment has the same start time as its corresponding audio stream fragment.

The encoding server 120 fragments the first video stream, the second video stream, and the audio stream included in the media stream, and then sends the media stream to the multicast server 130.

It should be noted that after receiving the media stream of a certain channel sent by the encoding server 120, the multicast server 130 may add a timestamp to each video stream segment in the first video stream included in the media stream. Each video stream slice in the second video stream adds a timestamp and each audio stream slice included in the audio stream adds a timestamp. For any video stream fragment in the first video stream, the video stream fragment and the video stream fragment are fragmented in a corresponding video stream in the second video stream, and the time stamps of the two video stream fragments are the same. The time stamp of the video stream fragment and the time stamp of the video stream fragment corresponding to the audio stream fragment in the audio stream are also the same.

Referring to FIG. 2, the audio stream segment corresponding to the video stream segment S11 and the video stream segment S21 is S31, and the timestamps corresponding to the three of S11, S21, and S31 are the same, so the video content of S11 and S21 is the same, in S31. The audio content corresponds to the video content.

The multicast server 130 adds timestamp information to each of the video stream segments in the first video stream, each of the second video streams, and each of the audio stream segments in the audio stream. The media stream is sent to the FCC server 140 and the terminal 150.

Please refer to FIG. 3 , which is a schematic structural diagram of a server provided by an exemplary embodiment of the present invention. The server may be the FCC server 140 in the embodiment shown in FIG. 1 , where the server includes: a processor 31 and a network interface 32. The cache 33 and the memory 34.

The processor 31 includes one or more processing cores, and the processor 31 executes various functional applications and data processing by running software programs and modules.

There may be multiple network interfaces 32, some of which are used to communicate with the multicast server 130, with a portion of the network interface 32 being used to communicate with the terminal 150.

The cache 33 is coupled to the processor 31 for buffering media streams of at least one channel received from the multicast server 130 and may be coupled to the network interface 32 via a bus.

The memory 34 is coupled to the processor 31. For example, the memory 34 can be coupled to the processor 31 via a bus; the memory 23 can be used to store software programs and modules.

The memory 34 can store an application module 35 required for at least one function, and the application module 35 can include a transmitting module 351, a processing module 352, a receiving module 353, and the like. The processor 31 executes the corresponding steps performed by the FCC server in FIG. 5 by running the transmitting module 351, the processing module 352, and the receiving module 353, as specifically described with respect to FIG.

The memory 34 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory ( English: electrically erasable programmable read-only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (English: programmable read only memory, PROM), only Read memory (English: read only memory image, ROM), magnetic memory, flash memory, disk or optical disk.

It will be understood by those skilled in the art that the structure of the server shown in FIG. 3 does not constitute a limitation to the server, and may include more or less components than those illustrated, or some components may be combined, or different component arrangements.

Please refer to FIG. 4, which is a schematic structural diagram of a terminal provided by an exemplary embodiment of the present invention. The terminal can be the terminal 150 in the embodiment shown in FIG. The terminal includes a processor 41, a network interface 42, and a memory 43.

The processor 41 includes one or more processing cores, and the processor 41 executes various functional applications and data processing by running software programs and modules.

There may be multiple network interfaces 42, some of which are used to communicate with the multicast server 130, with a portion of the network interface 42 being used to communicate with the FCC server 140.

The memory 43 is coupled to the processor 41. For example, the memory 43 can be coupled to the processor 41 via a bus; the memory 43 can be used to store software programs and modules.

The memory 43 can store an application module 44 required for at least one function, and the application module 44 can include a transmission module 441, a processing module 442, a receiving module 443, and the like. The processor 41 executes the corresponding steps performed by the terminal in FIG. 5 by running the transmitting module 441, the processing module 442, and the receiving module 443, with specific reference to the description of FIG.

Memory 43 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static SRAM, EEPROM, EPROM, PROM, ROM, magnetic memory, flash memory, magnetic or optical disk.

It will be understood by those skilled in the art that the structure of the terminal shown in FIG. 4 does not constitute a limitation of the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.

Please refer to FIG. 5, which illustrates a method for playing a media stream according to an exemplary embodiment of the present invention. The method for playing a media stream is applied to an IPTV system as shown in FIG. 1. The method for playing a media stream may include :

Step 501: The terminal sends a play request carrying the channel identifier of the target channel to the FCC server.

The terminal can send a play request to the FCC server in the following situations, including:

First, the terminal needs to send a play request to the FCC server when switching from the currently playing original channel to the target channel.

When the terminal is currently playing the media stream of the original channel, the user can select the target channel that needs to be switched. When the terminal detects the selected target channel, the channel identifier of the target channel is acquired, and a play request carrying the channel identifier of the target channel is sent to the FCC server.

When the terminal detects the selected target channel, the terminal also disconnects its connection with the multicast server to stop receiving the media stream of the original channel sent by the multicast server.

Second, the terminal uses the preset channel as the target channel when starting, and sends a play request to the FCC server to request to join the target channel.

The preset channel can be the default channel set by the terminal at the factory, or the channel played by the terminal when it was last turned off. Therefore, the terminal can obtain the channel identifier of the default channel or the channel identifier of the channel played last time when the terminal is currently activated, and use the obtained channel identifier as the channel identifier of the target channel, and send the channel identifier carrying the target channel to the FCC server. Play the request.

Step 502: The FCC server receives the play request sent by the terminal, and sends the first video stream of the target channel to the terminal according to the channel identifier of the target channel carried by the play request, and the fragment identifier of a video stream segment included in the first video stream. .

The FCC server receives and caches the media stream of each channel sent by the multicast server in real time, and the media stream of each channel includes a first video stream, a second video stream, and at least one audio stream.

The step may be: the FCC server receives the play request sent by the terminal, and determines, according to the channel identifier of the target channel carried in the play request, the first video stream of the target channel buffered by the FCC server, and the first target channel buffered from the FCC server. Obtaining a key frame in the video stream that is located before the current time, and acquiring the first video stream from the time corresponding to the key frame to the current time in the first video stream of the target channel buffered by the FCC server, and sending the acquired video to the terminal. a first video stream and a playback response, the playback response carrying a slice identifier of the video stream segment included in the acquired first video stream.

The FCC server can obtain key frames in multiple ways. For example, a key frame whose time is before the current time and is closest to the current time can be obtained from the first video stream of the target channel buffered by the FCC server, or a frame is randomly acquired. Keyframes before the current time.

When transmitting the acquired first video stream, the FCC server may transmit the acquired first video stream by using a receiving rate of the first video stream that is greater than its receiving target channel. For example, in implementation, the FCC server determines the receiving rate of the first video stream of the target channel that is currently received by the multicast server, calculates the sending rate according to the preset first multiple and the receiving rate, and sends the sending rate to the terminal according to the sending rate. Send the acquired first video stream. The first multiple is greater than 1, for example, the first multiple may be 1.4, 1.5, 1.6, etc., and the reception rate is multiplied by the first multiple to obtain the size of the transmission rate.

The order in which the FCC server sends the acquired first video and the sent playback response may be as follows: one is to first send a playback response, and then to send the obtained first video stream; A video stream and playback response. For the case of simultaneous transmission, the FCC server may transmit the playback response to the terminal while starting to transmit the acquired first video stream, or send the playback response to the terminal at some time after starting to send the first video stream.

The fragment identifier carried in the playback response is a fragment identifier of the nth video stream segment included in the acquired first video stream, where n is a positive integer greater than or equal to 1. Usually, n is 1 or 2 or 3.

The playback response may be an RTCP packet. Referring to FIG. 6, the RTCP packet includes a Default Bitrate/Adaptive Bitrate field, a current unicast synchronization source identifier (SSRC of Current Unicast Burst) field, User Internet Protocol (User IP) field, User RTCP Port field, Reserved field, Representation ID, Bandwidth field, and extended fragmentation A Fragment Time Stamp field is used to carry the fragment identifier of a video stream fragment included in the acquired first video stream.

Step 503: The terminal receives the fragmentation identifier of the first video stream and the video stream fragment sent by the FCC server, and sends an acquisition request carrying the fragment identifier to the FCC server.

The implementation of this step may be: the terminal receives the first video stream and the play response sent by the FCC server, and extracts, from the play response, the fragment identifier of the video fragment included in the first video stream, and sends the packet to the FCC server. Get the request for the slice ID.

The first video stream includes description information of at least one audio stream. The terminal may select description information of an audio stream from the description information of the at least one audio stream according to the capability information of the terminal, and the acquiring request may further carry the selected description information of the audio stream.

The capability information of the terminal may include a bandwidth size of the terminal, a terminal decoding capability, and/or a hardware resource of the terminal.

Wherein, if the description information of the audio stream included in the first video stream or the audio stream selected by the terminal is the default audio stream of the terminal and the FCC server, the acquisition request may or may not carry the description information of the audio stream of the one channel, or The acquisition request may or may not carry the description information of the selected default audio stream.

Step 504: The FCC server receives the acquisition request sent by the terminal, and determines, according to the fragment identifier in the acquisition request, a start time of the first video stream that has been sent to the terminal.

In this step, the FCC server receives the acquisition request sent by the terminal, and extracts the fragment identifier carried by the acquisition request, and the extracted fragment identifier is the segment of the nth video stream segment included in the first video stream that has been sent to the terminal. a slice identifier, determining, according to the extracted slice identifier, a slice identifier of the first video stream segment included in the first video stream, and acquiring the first video stream segment according to the slice identifier of the first video stream segment The timestamp of the slice determines the timestamp of the first video stream fragment as the start time of the first video stream that has been sent to the terminal.

If n=1, the extracted fragment identifier can be used as the fragment identifier of the first video stream fragment. If n is greater than 1, the slice identifier of the first video stream segment is calculated according to the extracted slice identifier and the value n.

Generally, the fragment identifier of the video stream fragment in the first video stream is continuously changed, and the difference between the fragment identifiers of the adjacent two video stream fragments is a value of 1, so that the fragment identifier is calculated for the foregoing. The process may be as follows: if the fragment identifier extracted from the acquisition request is m, the fragment identifier of the first video stream fragment in the first video stream is mn-1.

Step 505: The FCC server acquires an audio stream corresponding to the first video stream according to the start time, and sends the audio stream to the terminal.

When the play request received by the FCC server does not carry the description information of the audio stream, it indicates that there are two cases. The first type is that the first video stream only includes description information of one audio stream, that is, the target channel only corresponds to one audio stream; For the terminal, the description information of the default audio stream agreed with the FCC server in advance is selected.

In this case, the FCC server determines an audio stream corresponding to the cached target channel, and obtains the audio stream from the start time to the current time from the cached one audio stream. The acquired audio stream is sent to the terminal. In the second case, the FCC server determines the default audio stream corresponding to the description information of the default audio stream from the at least one audio stream corresponding to the cached target channel, and obtains the start time to the current source from the default audio stream. The audio stream of time sends the acquired audio stream to the terminal.

When the description information of the audio stream is carried in the play request received by the FCC server, the step may be: the FCC server determines, according to the at least one audio stream corresponding to the cached target channel, the audio stream corresponding to the description information of the audio stream, The determined audio stream is obtained from the start time to the current time, and the acquired audio stream is sent to the terminal.

The FCC server, when transmitting the acquired audio stream, may transmit the audio stream with a reception rate greater than the frequency at which it received the audio stream. For example, in implementation, the FCC server determines the receiving rate of the audio stream that is currently received by the multicast server, calculates a sending rate according to the preset second multiple and the receiving rate, and sends the audio stream to the terminal according to the sending rate. . The second multiple is greater than 1, for example, the first multiple may be 1.4, 1.5, 1.6, etc., and the reception rate is multiplied by the second multiple to obtain the size of the transmission rate. The second multiple may be equal to or not equal to the first multiple.

Step 506: The terminal receives the audio stream sent by the FCC server, and provides the received video stream and the audio stream to the player on the terminal for playing.

Since the audio stream received by the terminal is the same as the start time of the video stream, the terminal simultaneously starts playing the audio stream and the video stream to ensure that the sound of the target channel is synchronized with the video picture.

Step 507: The FCC server sends a notification message to the terminal when stopping sending the first video stream to the terminal and starting to send the second video of the target channel to the terminal, where the notification message carries the sequence number and the second of the last data packet of the first video stream. The sequence number of the first packet of the video stream.

The FCC server continuously and continuously sends the first video stream to the terminal after starting to send the first video stream of the target channel to the terminal, and the FCC server also receives the first video stream sent by the multicast server. When the data packet included in the first video stream currently sent by the FCC server to the terminal is the same as the data packet included in the first video stream sent by the current receiving multicast server, the FCC server determines that the data packet currently sent to the terminal belongs to The video stream is fragmented, and the first video stream is stopped from being sent to the terminal when the determined video stream fragment is sent. At the same time, the FCC server sends the second video stream of the target channel to the terminal, and the first video stream fragment included in the second video stream is the next video stream fragment of the determined video stream fragment.

For example, referring to FIG. 2, it is assumed that the data packet of the first video stream currently sent by the FCC server to the terminal is a data packet with sequence number 4, and it is assumed that the data packet of the first video stream currently received by the FCC server is also data with sequence number 4. The FCC server determines that the video stream fragment to which the data packet of sequence number 4 belongs is S12, and the fragmentation identifier of the video stream fragment S12 is 2; when the FCC server sends the video stream fragment S12, it sends the packet to the terminal. The second video stream, the first video stream fragment of the second video stream is S23, the fragment identifier of the video stream fragment S23 is 3, and the video stream fragment S12 (the fragment identifier is 2) is the next video stream. sheet.

Since the number of data packets included in the video stream segment in the first video stream is different from the number of data packets included in the video stream segment located in the second video stream, the sequence number of the last data packet included in the first video stream is The sequence number of the first data packet included in the second video stream is discontinuous, so that the terminal mistakenly believes that packet loss occurs when playing the second video stream. To solve this problem, the FCC server starts to send the second channel of the target channel to the terminal. When the video is sent to the terminal, the notification message is sent to the terminal, and the sequence number of the last data packet of the first video stream and the sequence number of the first data packet of the second video stream carried by the notification message are used to notify the terminal of the last data packet of the first video stream. And the first packet of the second video stream is a contiguous packet.

For example, referring to FIG. 2, the last data packet of the video stream segment S12 has a sequence number of 4. When the FCC server sends the data packet, the second video stream is sent to the terminal, and the first video stream of the second video stream is sent. The fragmentation is S23, and the sequence number of the first packet in the video stream fragment S23 is 9. Therefore, after receiving the data packet with sequence number 4, the terminal receives the data packet with sequence number 9. Since the terminal also receives the notification message carrying the sequence number 4 and the sequence number 9, it can be determined according to the message that the data packet with the sequence number 4 and the data packet with the sequence number 9 are consecutive data packets, and no loss occurs.

The FCC server may transmit the second video stream when it transmits the second video stream, less than the receiving rate at which it receives the second video stream. For example, in implementation, the FCC server determines the receiving rate of the second video stream of the target channel that is currently received by the multicast server, calculates the sending rate according to the preset third multiple and the receiving rate, and sends the sending rate to the terminal according to the sending rate. Send the acquired second video stream. The third multiple is less than 1, for example, the third multiple may be 0.6, 0.7, 0.8, etc., and the reception rate is multiplied by the third multiple to obtain the size of the transmission rate.

Optionally, the notification message may be an SCN message. Referring to FIG. 7, the SCN message includes a Final Optimal Adaptive Bitrate field, and an extended Last Sequence Number field and a new sequence number. (First Sequence Number) field, the old sequence number field is used to carry the sequence number of the last data packet, and the new sequence number field is used to carry the sequence number of the first data packet.

Step 508: The terminal receives the notification message sent by the FCC server, and plays the second video stream sent by the FCC and the multicast group added to the target channel according to the notification message, and receives the second video stream and the audio stream sent by the multicast server.

The terminal receives the notification message sent by the FCC server, and also receives the second video stream sent by the FCC server, and obtains the sequence number of the last data packet of the first video stream carried in the notification message and the first data packet of the second video stream. Serial number. The terminal determines, according to the sequence number of the last data packet of the first video stream and the sequence number of the first data packet of the second video stream, that the last data packet of the first video stream and the first data packet of the second video stream are consecutive The two packets are then played from the first packet of the second video stream.

When the terminal plays the second video stream, the terminal also sends a join request carrying the channel identifier of the target channel to the multicast server. The multicast server receives the join request, adds the terminal to the multicast group of the target channel according to the channel identifier of the target channel, and then sends the second video stream and the audio stream of the target channel to the terminal. The terminal receives and caches the second video stream and the audio stream sent by the multicast server after starting to play the second video stream sent by the FCC.

Step 509: When the data packet included in the second video stream currently played by the terminal is the same as the first data packet included in the second video stream sent by the multicast server, the second video stream and the audio stream sent by the multicast server are played.

When the terminal plays the second video stream and the audio stream sent by the multicast server, the terminal completes switching from the original channel to the target channel or the terminal joins the target channel after completing the startup. At the same time, the terminal can also disconnect the connection with the FCC server to stop receiving the second video stream and the audio stream of the FCC server transmitting the target channel.

In summary, the method for playing a media stream is provided by the terminal, and the terminal sends an acquisition request to the FCC server, where the acquisition request carries the video stream fragmentation in the video stream sent by the FCC server that has been received by the terminal. And the FCC server determines, according to the fragment identifier, a start time of the video stream that has been sent to the terminal, and obtains an audio stream corresponding to the video stream according to the start time, the audio stream start time and the video stream. The start time is the same, the audio stream is sent to the terminal; since the terminal receives the video stream and the audio stream with the same start time sent by the FCC server, the terminal can ensure the audio and video synchronization when playing the video stream and the audio stream at the same time, and the solution is solved. In the related art, the terminal does not play the sound or the played sound is inconsistent with the video picture, and the sound played by the terminal is consistent with the video picture.

The video stream sent by the server to the terminal includes description information of at least one audio stream, so that the terminal selects description information of an audio stream that matches its own capability, and the acquisition request sent to the server includes description information of the one audio stream. In this way, the terminal can acquire an audio stream that matches its own capabilities for playback according to its own capabilities.

The server sends the RTCP packet to the terminal, and the fragmentation timestamp field of the RTCP packet carries the fragment identifier, so that the terminal sends an acquisition request carrying the fragment identifier to the server, so that the video received by the terminal is obtained from the server. Streams the corresponding audio stream.

The server sends a notification message to the terminal by stopping sending the first video stream to the terminal and transmitting the second video stream of the second code rate to the terminal, where the notification message carries the sequence number and the number of the last data packet of the first video stream. The sequence number of the first data packet of the second video stream is such that the terminal determines, according to the notification message, that the last data packet and the first data packet are consecutive data packets, thereby preventing the terminal from being misjudged into a packet loss, thereby normalizing from the first A packet begins to play the second video stream.

Please refer to FIG. 8 , which illustrates a server 800 provided by another exemplary embodiment of the present invention. The server 800 may be an FCC server in the embodiment shown in FIG. 1 , FIG. 3 , and/or FIG. 5 . The transmission unit 810, the processing unit 820, and the receiving unit 830 are included.

The sending unit 810 is configured to perform the functions of at least one of the foregoing steps 502, 505, and 507.

The processing unit 820 is configured to perform the functions of at least one of the foregoing steps 504 and 505.

The receiving unit 830 is used for the function of at least one of the foregoing steps 502 and 504.

Related details can be combined with reference to the above method embodiments.

It should be noted that, when the server provided by the foregoing embodiment provides the terminal with the media stream of the target channel that the terminal requests to play, only the division of the foregoing functional modules is illustrated. In actual applications, the foregoing functions may be performed according to requirements. The allocation is done by different functional modules, that is, the internal structure of the server is divided into different functional modules to complete all or part of the functions described above. In addition, the server provided in the above embodiment is the same as the method embodiment of the method for playing the media stream, and the specific implementation process is described in the method embodiment, and details are not described herein again.

Please refer to FIG. 9 , which illustrates a terminal 900 provided by another exemplary embodiment of the present invention. The terminal 900 may be the terminal 900 in the embodiment shown in FIG. 1 , FIG. 4 and/or FIG. 5 , and the terminal includes : a transmitting unit 910, a processing unit 920, and a receiving unit 930.

The sending unit 910 is configured to perform the functions of at least one of the foregoing steps 501, 503, and 508.

The processing unit 920 is configured to perform the functions of at least one of the foregoing steps 506, 508, and 509.

The receiving unit 930 is configured to perform the functions of at least one of the foregoing steps 503, 506, and 508.

Related details can be combined with reference to the above method embodiments.

It should be noted that, when the terminal provided by the foregoing embodiment plays the media stream, only the division of the foregoing functional modules is illustrated. In actual applications, the function distribution may be completed by different functional modules according to requirements, that is, the terminal is The internal structure is divided into different functional modules to perform all or part of the functions described above. In addition, the method provided in the foregoing embodiment is the same as the method embodiment of the method for playing the media stream, and the specific implementation process is described in the method embodiment, and details are not described herein again.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above description is only an optional embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the protection of the present application. Within the scope.

Claims

A method for playing a media stream, the method comprising:

The server receives the acquisition request of the fragment identifier that carries the video stream fragment sent by the terminal, where the video stream fragment is a video stream fragment included in the video stream that the server has sent to the terminal;

Determining, by the server, a start time of the video stream according to the fragment identifier of the video stream fragment;

Obtaining, by the server, an audio stream corresponding to the video stream according to the start time;

The server sends the audio stream to the terminal to cause the terminal to play the video stream and the audio stream.
The method according to claim 1, wherein the video stream includes description information of at least one audio stream, and the acquisition request further carries description information of an audio stream;

Obtaining, by the server, the audio stream corresponding to the video stream according to the start time, including:

Determining, by the server, the one audio stream according to the description information of the one audio stream;

And obtaining, by the server, an audio stream corresponding to the video stream from the one audio stream according to the start time.
The method according to claim 1 or 2, wherein before the server receives the acquisition request of the fragment identifier of the video stream fragment sent by the terminal, the method further includes:

The server sends a real-time transmission control protocol (RTCP) packet to the terminal, and the fragmentation timestamp field of the RTCP packet carries a fragmentation identifier of the video stream fragment.
The method according to claim 1 or 2, wherein the video stream that the server has sent to the terminal is a first video stream of a first code rate;

The method further includes:

When the server stops sending the first video stream to the terminal and sends a second video stream of the second code rate to the terminal, the server sends a notification message to the terminal, where the notification message carries The sequence number of the last data packet of the first video stream and the sequence number of the first data packet of the second video stream, the first code rate being less than the second code rate.
The method according to claim 4, wherein the notification message is for the server terminal to notify the SCN message, the SCN message includes an old serial number field and a new serial number field, and the old serial number field carries the last one The sequence number of the data packet, the new sequence number field carrying the sequence number of the first data packet.
A method for channel switching, characterized in that the method comprises:

The terminal sends, to the server, an acquisition request that carries a fragment identifier of the video stream fragment, where the video stream fragment is a video stream fragment included in the video stream sent by the server that is received by the terminal;

Receiving, by the terminal, an audio stream that is sent by the server according to the fragment identifier, where a start time of the audio stream is the same as a start time of the video stream;

The terminal plays the video stream and the audio stream.
The method according to claim 6, wherein said video stream comprises description information of at least one audio stream;

Sending, by the terminal, the acquiring request of the fragment identifier carrying the video stream fragment to the server, including:

Descriptive information of an audio stream selected by the terminal from the at least one audio stream;

The terminal sends, to the server, an acquisition request that carries a fragment identifier of the video stream segment and description information of the one channel audio stream.
The method according to claim 6 or 7, wherein before the terminal sends the acquisition request of the fragment identifier carrying the video stream fragment to the server, the method further includes:

The terminal receives the real-time transmission control protocol (RTCP) packet sent by the server, and acquires the fragmentation identifier of the video stream fragment carried in the fragmentation timestamp field of the RTCP packet.
The method according to claim 6 or 7, wherein the video stream received by the terminal is a first video stream of a first code rate;

The method further includes:

Receiving, by the terminal, a second video stream and a notification message of a second code rate sent by the server, where the second code rate is greater than the first code rate, and the notification message carries a last one of the first video stream a sequence number of the data packet and a sequence number of the first data packet of the second video stream;

The terminal plays the second video stream according to the notification message.
The method according to claim 9, wherein the notification message is for the server terminal to notify the SCN message, the SCN message includes an old sequence number field and a new sequence number field, and the old sequence number field carries the last one The sequence number of the data packet, the new sequence number field carrying the sequence number of the first data packet.
A server, wherein the server comprises:

a receiving unit, configured to receive, by the terminal, an acquisition request of a fragment identifier that carries a video stream fragment, where the video stream fragment is a video stream fragment included in a video stream that the server has sent to the terminal;

a processing unit, configured to acquire a start time of the video stream according to the fragment identifier of the video stream segment, and acquire an audio stream corresponding to the video stream according to the start time;

And a sending unit, configured to send the audio stream to the terminal, so that the terminal plays the video stream and the audio stream.
The server according to claim 11, wherein the video stream includes description information of at least one audio stream, and the acquisition request further carries description information of an audio stream;

The processing unit is further configured to:

Determining the one audio stream according to the description information of the one audio stream;

Obtaining, according to the start time, an audio stream corresponding to the video stream from the one audio stream.
The server according to claim 11 or 12, wherein the sending unit is further configured to send a real-time transmission control protocol RTCP packet to the terminal, where a fragment timestamp field of the RTCP packet carries the video stream The fragmentation identifier of the fragment.
The server according to claim 11 or 12, wherein the video stream that the server has sent to the terminal is the first video stream of the first code rate;

The sending unit is further configured to: when stopping sending the first video stream to the terminal and sending a second video stream of a second code rate to the terminal, sending a notification message to the terminal, where the notification message is sent And carrying a sequence number of a last data packet of the first video stream and a sequence number of a first data packet of the second video stream, where the first code rate is smaller than the second code rate.
The server according to claim 14, wherein the notification message is for the server terminal to notify the SCN message, the SCN message includes an old serial number field and a new serial number field, and the old serial number field carries the last one The sequence number of the data packet, the new sequence number field carrying the sequence number of the first data packet.
A terminal, wherein the terminal comprises:

a sending unit, configured to send, to the server, an acquisition request that carries a fragment identifier of the video stream fragment, where the video stream fragment is a video stream fragment included in the video stream sent by the server that is received by the terminal;

a receiving unit, configured to receive an audio stream sent by the server according to the fragment identifier, where a start time of the audio stream is the same as a start time of the video stream;

And a processing unit, configured to play the video stream and the audio stream.
The terminal according to claim 16, wherein the video stream includes description information of at least one audio stream;

The processing unit is further configured to: describe information of an audio stream selected from the at least one audio stream;

The sending unit is further configured to send, to the server, an acquisition request that carries the fragment identifier of the video stream segment and the description information of the one channel audio stream.
The terminal according to claim 16 or 17, wherein the receiving unit is further configured to receive a real-time transmission control protocol RTCP packet sent by the server, and acquire a location carried by a fragment timestamp field of the RTCP packet. The fragmentation identifier of the video stream fragment.
The terminal according to claim 16 or 17, wherein the received video stream is a first video stream of a first code rate;

The receiving unit is further configured to receive a second video stream and a notification message of the second code rate sent by the server, where the second code rate is greater than the first code rate, and the notification message carries the first The sequence number of the last data packet of the video stream and the sequence number of the first data packet of the second video stream;

The processing unit is further configured to play the second video stream according to the notification message.
The terminal according to claim 19, wherein the notification message is for the server terminal to notify the SCN message, the SCN message includes an old serial number field and a new serial number field, and the old serial number field carries the last one The sequence number of the data packet, the new sequence number field carrying the sequence number of the first data packet.