Background technology
Mobile flow medium service can be divided into following two classes according to broadcast mode.
(1) online broadcast: terminal player obtains stream medium data from streaming media server in real time, plays while downloading, and streaming medium content does not need to be stored on user's the terminal equipment.If the user need repeatedly play same content, all need each time from streaming media server, to obtain data again.For the content of online broadcast, owing to be to play while downloading, result of broadcast depends on the network bandwidth to a great extent, so need select the proper compression parameter to make content according to the actual bandwidth situation of carrier network.
(2) downloading and playing: the user downloads streaming medium content and store in the local terminal, can be chosen in random time then and play.For downloading and playing, main restriction index is the disposal ability and the memory capacity of terminal, and content supplier can produce the video content (high bandwidth, high frame rate) of better quality, but needs to consider the download time of content and the memory space of terminal.
The mobile stream media system structure: the Core Feature of mobile flow medium service system is to provide the online broadcast and the downloading and playing function of Streaming Media, online broadcast also to support live simultaneously and two kinds of service forms of program request by mobile communications network to the mobile subscriber.Streaming Media is meant that data such as video, sound carry with RTP, and transmits to destination from the source end with the form of continuous stream, receives the multimedia application that certain data cached back just can play back at destination.The mobile flow medium technology has three big characteristics:
1. playing audio-video and content of multimedia in real time.Can shorten greatly like this and start time-delay, avoid the user must wait for that whole file all downloads the shortcoming that just can watch after finishing from the server source.
2. the files in stream media of Bo Fanging need not reduce the requirement to the client stores space in client storage, has reduced the demand of buffer memory capacity yet.
Since files in stream media not in client storage, thereby solved the Copyright Protection of media file to a certain extent.
Because medium can be play after being sent to playback terminal, so the business that is called as " playing " of streaming media service image while downloading.In fact stream mode is different with downloading mode, and downloading mode need download to playback terminal this locality fully with medium, just can play, need the user to wait for the long period, Comparatively speaking, stream mode gives user's impression more timely, and the user does not need to wait as long for.
At streaming server media content is sent to stream mode in the mobile device terminal process, ideal situation is that the transmission bandwidth maintenance is constant, and the media content encoded bandwidth of transmission is more smaller than transmission bandwidth.The good playing stream media content of playback terminal like this.But actual conditions are then really not so.Network environment changes through regular meeting, and when the Network Transmission bandwidth was lower than the media coding bandwidth, congested, packet loss and transmission delay will take place, and playback terminal will occur interrupting playing or mosaic phenomenon, and the media play that influence user is experienced.
Under wireless network, because moving of mobile device terminal, the decay of wireless signal transmission and wireless bandwidth are taken by voice or the higher business of other priority at any time, and the wireless link bandwidth fluctuation that makes mobile device terminal use is big, the transmission error rates height.For carrying out of mobile flow medium service increased difficulty, how to make full use of air bandwidth, for providing the Streaming Media of high-quality smoothness, the user experiences, and be the problem that system of the present invention will solve.
Summary of the invention
Technical problem to be solved by this invention is to provide a kind of bandwidth adapting system of streaming media.Use system of the present invention, can make the actual behaviour in service of streaming server, the function of bandwidth adaptation is provided according to mobile device terminal user and network system.As user during in the playing stream media content, the streaming media service platform can be surveyed current actual bandwidth of user and decoding terminals situation, sending to the user, ensure that the user can both see not having the broadcast of interrupting under different bandwidth situation then with content near the compression of actual bandwidth speed.
For solving the problems of the technologies described above, the invention provides a kind of bandwidth adapting system of streaming media, comprise: streaming server, medium provide server, mobile device terminal, and described streaming server comprises: server flows protocol module, bandwidth analysis module, bandwidth are selected module, document analysis module, medium sending module; Described mobile device terminal comprises: terminal stream protocol module, wireless module, bandwidth detection, buffer zone module and playing module, wherein:
The server flows protocol module, be used to handle the communication of streaming server and mobile device terminal, to send to the bandwidth analysis module from bandwidth conditions and the medium reception situation that terminal stream protocol module receives, and will return to terminal stream protocol module from the media content that the medium sending module returns;
The bandwidth analysis module, the bandwidth conditions and the medium that are used for sending according to the server flows protocol module receive status analysis transmission medium bandwidth and transmission speed, and the medium bandwidth result who analyzes is sent to bandwidth select module, the transmission speed result who analyzes is sent to the medium sending module;
Bandwidth is selected module, is used for to document analysis module request media content, and the media content that the medium bandwidth result who sends according to the bandwidth analysis module returns the document analysis module carries out bandwidth and switches, and the media content after will switching returns the medium sending module;
The document analysis module, streaming server and medium provide the interface of server, and being used for provides server to obtain corresponding media content from medium, and selects module to return the media content of acquisition to bandwidth;
The medium sending module, the media content that transmission speed that returns according to the bandwidth analysis module and bandwidth select module to return turns back to the server flows protocol module with media content;
Terminal stream protocol module, be used to handle the communication of mobile device terminal and streaming server, to send to the server flows protocol module from bandwidth conditions and the medium reception situation that bandwidth detection receives, and the media content that the server flows protocol module is returned returns buffer zone module;
Wireless module, the streaming server that is used on wireless network and the cable network carries out data communication, and the bandwidth conditions that obtains is sent to bandwidth detection;
Bandwidth detection is used for obtaining bandwidth conditions to wireless module, obtains medium to buffer zone module and receives situation, and bandwidth conditions and medium reception situation are sent to terminal stream protocol module;
Buffer zone module is used for sending medium according to the mobile device terminal buffer size to bandwidth detection and receives situation, and the media content that will be received from terminal stream protocol module deposits buffering area in, sends to playing module when sequential arrives playing;
Playing module is used for obtaining media content from buffer zone module, carries out media play.
Use system of the present invention, stream media system can be adjusted media file transmission speed and encoded bandwidth automatically according to network condition and medium reception situation, has guaranteed the result of broadcast of media file in mobile device terminal.
Embodiment
The basic principle of system of the present invention is: mobile device terminal is carried out data communication by wireless gateway device on the wireless network and the stream media system on the cable network after setting up data link and be connected by built-in wireless module and wireless network.Mobile device is play after choosing the media content of a certain encoded bandwidth.Mobile device terminal can be from built-in with wireless module acquisition wireless bandwidth at that time in the playing process.And the notification streams service system, streaming server selects to send fast medium, perhaps slow transmission medium according to working as Time Bandwidth.Mobile device regularly reports to streaming server and receives buffering area situation and decoded state in receiving media information.Streaming server is carrying out fast and in the slow transmission process, according to these information, is adjusting transmission speed, and the assurance medium are play with best effect in mobile device terminal.The quick transmission here is meant and increases the media coding bandwidth that sends, perhaps strengthens the speed that sends packet.Slow transmission is meant and reduces the media coding bandwidth that sends.Normal transmission speed is meant the medium transmission speed of determining according to the medium bandwidth.
According to above-mentioned principle, the course of work of system of the present invention is:
1, the bandwidth detection of mobile device terminal obtains current bandwidth conditions by wireless module, and bandwidth conditions comprises: the bandwidth that can guarantee, maximum bandwidth, maximum time delay.These information will be carried to streaming server in the 3GPP-LINK-CHAR of RTSP request message.The stream protocol processing module of streaming server receives these information.
2, streaming server is told mobile device terminal: server possesses the various code rate information of bandwidth adaptation function and the selected media content of mobile device.When mobile device and streaming server were set up the stream protocol session, streaming server carried above-mentioned information in the response message of discribe.
3, mobile device terminal sends the buffer size of oneself and the buffer time of expectation to streaming server.Can adopt SETUP, the 3GPP-ADAPTATION field of SET_PARAMETER message sends.Streaming server returns same 3GPP-ADAPTATION field information and represents to confirm in response message.
4, in carrying out the Media Stream transport process, mobile device terminal is regularly returned the idle size of buffering area of mobile device terminal to streaming server, play time-delay, the sequence number of the next RTP bag of medium bag correspondence to be decoded, the band decoded media wraps in the serial number information in the RTP bag.These information are transmitted as using bag in the RTCP bag.The stream protocol processing module of streaming server receives these information.
5, mobile device terminal receiving media stream and adopt when playing buffering area mechanism, buffering area to be divided into buffering area before the decoding, decoding back buffering area has two timers, a decoding timer, a broadcast timer simultaneously.The mobile device terminal buffering area is put into the preceding buffering area of decoding after receiving medium stream information, when the decoding timer arrives, decodes, and with content placement decoding back buffering area, after the broadcast timer arrives, from the back buffering area of decoding, take out content, in playing module, play.
6, after the stream protocol processing module on the streaming server is received information, be sent to the bandwidth analysis module, the bandwidth analysis module is according to wireless bandwidth information, the decoding speed of mobile device terminal, and the freebuf size is determined the medium bandwidth and the transmission speed that transmit.Adjust the medium bandwidth if desired, then the different bandwidth of selecting module to carry out the identical content of same medium by bandwidth switches.The medium sending module sends.
As shown in Figure 1, one embodiment of the invention is a kind of system construction drawing of bandwidth adapting system of streaming media, and each several part is respectively among the figure: medium provide server 101, streaming server 102, mobile device terminal 103.
Streaming server 102 comprises:
The server flows protocol module, be used to handle the communication of streaming server and mobile device terminal, to send to the bandwidth analysis module from bandwidth conditions and the medium reception situation that terminal stream protocol module receives, and will return to terminal stream protocol module from the media content that the medium sending module returns;
The bandwidth analysis module, the bandwidth conditions and the medium that are used for sending according to the server flows protocol module receive status analysis transmission medium bandwidth and transmission speed, and the medium bandwidth result who analyzes is sent to bandwidth select module, the transmission speed result who analyzes is sent to the medium sending module;
Bandwidth is selected module, is used for to document analysis module request media content, and the media content that the medium bandwidth result who sends according to the bandwidth analysis module returns the document analysis module carries out bandwidth and switches, and the media content after will switching returns the medium sending module;
The document analysis module, streaming server and medium provide the interface of server, and being used for provides server to obtain corresponding media content from medium, and selects module to return the media content of acquisition to bandwidth;
The medium sending module, the media content that transmission speed that returns according to the bandwidth analysis module and bandwidth select module to return turns back to the server flows protocol module with media content;
Mobile device terminal 103 comprises:
Terminal stream protocol module, be used to handle the communication of mobile device terminal and streaming server, to send to the server flows protocol module from bandwidth conditions and the medium reception situation that bandwidth detection receives, and the media content that the server flows protocol module is returned returns buffer zone module;
Wireless module, the streaming server that is used on wireless network and the cable network carries out data communication, and the bandwidth conditions that obtains is sent to bandwidth detection;
Bandwidth detection is used for obtaining bandwidth conditions to wireless module, obtains medium to buffer zone module and receives situation, and bandwidth conditions and medium reception situation are sent to terminal stream protocol module;
Buffer zone module is used for sending medium according to the mobile device terminal buffer size to bandwidth detection and receives situation, and the media content that will be received from terminal stream protocol module deposits buffering area in, sends to playing module when sequential arrives playing;
Playing module is used for obtaining media content from buffer zone module, carries out media play.
As shown in Figure 2, be the module workflow diagram of a kind of bandwidth adapting system of streaming media of the embodiment of the invention.The idiographic flow step of embodiment of the invention system works is:
1, the bandwidth detection of mobile device terminal obtains current bandwidth conditions by wireless module, and bandwidth conditions comprises: the bandwidth that can guarantee, maximum bandwidth, maximum time delay.These information will be carried to streaming server in the 3GPP-LINK-CHAR of RTSP request message.The stream protocol processing module of streaming server receives these information.
2, the bandwidth detection of mobile device terminal is obtained the idle size of buffering area to buffer zone module in needs, plays time-delay, the sequence number of the next RTP bag of medium bag correspondence to be decoded, and medium to be decoded wrap in the serial number information in the RTP bag.
3, bandwidth detection sends to the stream protocol module with the bandwidth relevant information.
4, the stream protocol module is at SETUP, PLAY, the OPTION of RTSP agreement, and the 3GPP-LINK-CHAR field of SET_PARAMETER method is to streaming server report current bandwidth situation.For the real-time buffer information of mobile device terminal (content that step 2 obtains), the stream protocol module adopts rtcp protocol to transmit to streaming media server.Mobile device terminal also adopts SETUP, and the 3GPP-ADAPTATION field of SET_PARAMETER message sends the buffer size of oneself and the buffer time of expectation to streaming server.Streaming server returns same 3GPP-ADAPTATION field information and represents to confirm in response message.
5, the stream protocol module of streaming media server sends to the bandwidth analysis module with the bandwidth relevant information of receiving,
6, the bandwidth analysis module is according to certain rule, and decision medium bandwidth value sends to bandwidth and selects module.
7, according to buffer state, adjust transmission speed if desired, the bandwidth analysis module sends to the medium sending module with transmission speed information.
8, bandwidth selects module to document analysis module request media content, can be that the bandwidth of same media content is switched, also the same bandwidth content of same media content.
9, the document analysis module is returned media content to the bandwidth analysis module.
10, the bandwidth analysis module is transmitted to the medium sending module with the media content that returns.
11, the medium sending module calculates transmitting time according to transmission speed, and transmitting time arrives, and media content is sent to the stream protocol module.
12, the stream protocol module sends to media content the stream protocol module of mobile device terminal.
13, the stream protocol module sends media information in the buffering area.
14, the buffering area of mobile device terminal is divided into the preceding buffering area of decoding, and decoding back buffering area has two timers, a decoding timer, a broadcast timer simultaneously.The mobile device terminal buffering area is put into the preceding buffering area of decoding after receiving medium stream information, when the decoding timer arrives, decodes, and with content placement decoding back buffering area, after the broadcast timer arrives, from the back buffering area of decoding, take out content, in playing module, play.Media information is delivered to the media play module, carries out media play.
As shown in Figure 3, be the working frame flow chart of a kind of bandwidth adapting system of streaming media of the embodiment of the invention.
It is example that the working frame flow chart of present embodiment begins with the stream media protocol session, setting up flow media session, carry out in the streamed process of medium, transmission bandwidth variation and the variation of mobile device terminal buffering area etc. appear, the information that streaming media server reports according to mobile device terminal is selected suitable media content to send to mobile device terminal and is finished bandwidth adaptation.Idiographic flow and step are as follows:
(step 301) mobile device terminal sends DISCRIBE message to streaming media server, and streaming media server returns the various bandwidth information of the selected medium of mobile device terminal in SDP file.SDP file form such as 3GPP PSS R6 5.3.3 joint regulation.If the SDP file that streaming media server returns does not comprise the content of 3GPP PSS R6 5.3.3 joint regulation, show that not support dynamic bandwidth adaptive, then do not need subsequent step.
(step 302) mobile device terminal is in follow-up SETUP message, carry 3GPP-LINK-CHAR and 3GPP-ADAPTATION field, wherein the 3GPP-LINK-CHAR current bandwidth information of carrying is that wireless module from mobile device terminal obtains, comprise: maximum bandwidth, can guarantee bandwidth, maximum delay.The 3GPP-ADAPTATION field information of carrying comprises the buffer time of existing buffer size and expectation.The form of these two fields meets 5.3.2.1 and the 5.3.2.2 standard of 3GPP PSS R6.
The bandwidth guaranteed that the bandwidth analysis module of (step 303) streaming media server selects mobile device terminal to submit to is a medium bandwidth to be selected, bandwidth selects module to select the information of the media content of appropriate bandwidth by the document analysis module from media content, in the SETUP response message, return to mobile device terminal by the stream protocol module, can select the immediate media content of bandwidth therewith.The media content of absolute value minimum of medium bandwidth and the difference that can guarantee bandwidth just.
(step 304) mobile device terminal receives the SETUP response message, sends PLAY message, begins to carry out media play.In the media play process, the bandwidth detection of mobile device terminal is regularly obtained current bandwidth from wireless module.
Simultaneously, when the bandwidth detection of mobile device terminal sends the RTCP bag at needs, send message to buffer zone module and obtain buffering area situation and decoded state, adopt the RTCP bag that these information are sent to streaming media server.And in RTSP message, regularly reporting bandwidth information to streaming media server, the bandwidth detection of mobile device terminal sends to streaming media server with bandwidth information.The stream protocol module adopts the 3GPP-LINK-CHAR field of SET_PARAMETER method to carry current bandwidth information.
The stream protocol module of (step 305) streaming media server sends to the bandwidth analysis module with the information of 3GPP-LINK-CHAR field, the bandwidth analysis module obtain current bandwidth and preceding twice bandwidth and mean value, if the mean value of current bandwidth and medium bandwidth are in the specification error scope, then not needing to carry out bandwidth changes, if surpass the error range of setting, then need to carry out bandwidth and change.Currency is as the criterion with the average bandwidth value.Only select the mean value mode in this example, also can adopt other algorithms.
The bandwidth analysis module sends to bandwidth with the expected bandwidth value and selects module, and informs the current playing time mark, and time mark adopts NPT (normal play time) form.Bandwidth selects module according to medium bandwidth information and current reproduction time, send message to the document analysis module, obtain corresponding media content, the document analysis module selects module to return the media content pointer to bandwidth, bandwidth selects module to send the media content pointer to the medium sending module, the medium sending module is transmitted to the stream protocol module according to the transmission speed of setting with the media content pointer, and the stream protocol module sends to mobile device terminal with media content.
Under the normal condition, the medium sending module according to npt time interval of former and later two medium bags for being as the criterion, after receiving the media data packet that will send, timer is set, the difference of the transmitting time between two packets that send before and after timer length guarantees equated with the npt time difference of two medium bags.
When mobile device terminal reports the buffer empty free space bigger, and transmission bandwidth is higher than under the situation of medium bandwidth, can shorten between former and later two medium bags transmission at interval, but guarantee to send bandwidth less than transmission bandwidth.Can simply get the quotient that the size of transmission bandwidth and packet divides exactly, the transmission between the medium bag was quotient reciprocal second at interval like this.According to this time interval transmission medium bag.
When mobile device terminal reports the freebuf size too little, account for 20% for too little with idle ratio in this example, streaming media server returns to the transmission of normal condition.Increase if comprise decoding delay in the information that mobile device reports, and freebuf minimizing excessive velocities, reach 50% of data transmission speed with growth rate in this example and be as the criterion.Expression is the too low reason of mobile device disposal ability, then needs to adjust the medium bandwidth.
Adopt the media content of lower bandwidth to continue to send to mobile device terminal, reduce excessive velocities if still find the mobile device terminal freebuf, and decoding delay is still increasing, then further reduce the medium bandwidth that sends, up to guaranteeing that the mobile device terminal freebuf keeps dynamic equilibrium under normal transmission speed.Just mobile device terminal freebuf size mean value in a period of time is constant.