CN1855824B - Method and apparatus for streaming data processing - Google Patents
Method and apparatus for streaming data processing Download PDFInfo
- Publication number
- CN1855824B CN1855824B CN200610065474.XA CN200610065474A CN1855824B CN 1855824 B CN1855824 B CN 1855824B CN 200610065474 A CN200610065474 A CN 200610065474A CN 1855824 B CN1855824 B CN 1855824B
- Authority
- CN
- China
- Prior art keywords
- data flow
- data
- server
- client
- tts
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
Abstract
The invention refers to a method and device of convection on steam processing data. Specifically, the method and device can generate real-time abundant continuous audio data from sever and provide these data to client. On one hand, the method is proposed that controls media data stream to generate from data stream player, it contains: estimate the generation time of data stream; estimate the playing time of data stream; employ data stream generation resources to generate data stream which will be transmitted into data stream player; and when the remain generation time is actually less than or equal to playing time, the data stream player will be informed. The preferably embodiment of this invention has considered the operation ability of sever and handled data by priority sequence to assure the serving quality to clients.
Description
Technical field
The present invention relates to be used for the method and apparatus of streaming data.Particularly, the method and apparatus that the present invention relates in server, to generate continuous in a large number speech data in real time and provide it to the client.
Background technology
Server receives from the client and is used for the request of processing in real time.This request needs server to carry out continuous processing, and provides treated data block to arrive the client simultaneously.Data block is handled the client from the server streaming, and the client plays these data blocks continuously.
Under the situation of no problem, the processing of server end can provide data block to arrive the client continuously.Depend on the data of being handled by streaming, the client can set not buffered data piece, buffering all data blocks or only cushion the data block of defined amount before allowing to play.
Server expects can experience the situation that is in many times under the pressure; When for example, very low or CPU usage is very high in bandwidth.This influences server probably and handles ability with streaming data continuously.To a certain extent, cushion the solution that provides this problem.Yet static client buffer is not considered the work at present load or the network bandwidth of server.
For client, the result can be variable.Two possible situations can appear.1) client stores excessive data, waits in fact that before beginning to play all data of being asked arrive.For being intended to containing of client and user, they must and receive between the broadcast of data in request and wait for undesired time span.2) client has stored fragmentary data, and buffer underrun occurs.Therefore, the end user will experience interruption when playing.
It is not inessential carrying out size estimation for Text To Speech (TTS) system, and if the size of TTS transfer of data calculate incorrectly, then some problems can appear.For the client that is under the pressure, the quality of TTS significantly descends.If the TTS server is under the pressure, then TTS can be played to calling party with bit, and this will seem not too natural.If client (typically being mutual voice response systems) detects underrun, then whole prompting meeting is reset.Under all situations, calling party can be realized the negative experience of system, and does not re-use this system in the future probably.
United States Patent (USP) publication 6766407 be Microsoft Corp.'s " Intelligent streamingframework " describe streaming and handle the framework manager, the unit that streaming deals with scheme is coordinated in its analysis according to the attribute of specific connection.This publication is not considered the live load of streaming processing maker.
United States Patent (USP) publication 6112239 be Intervu Inc.'s " System and method forserver-side optimization of data delivery on a distributed computernetwork " relate to server-side optimization and network performance information.This publication relates to according to the network information data redirection to each transmission sites and server.
European patent publication 1182875 be MatsushitaElectric Co.'s " Streamingmethod and corresponding system " be according to by the client but not the negotiation that server-capabilities drove be described, but it relates to optimization, with underflow and the overflow of avoiding buffer.The change of transmittability is detected, and it is reacted.Client terminal is responsible for using transmittability to calculate suitable buffer level and delay, thereby and indicates server to transmit with particular rate.Yet, do not consider server performance.
Summary of the invention
According to a first aspect of the invention, provide the method for the media data flow that a kind of control generated from the data flow player plays, having comprised: the rise time of estimating to be used to generate data flow; The reproduction time of data estimator stream; Generate data flow by using data flow to generate resource, so that export by the data flow player; And if the residue rise time is not more than reproduction time (that is, the residue rise time is equal to or less than reproduction time), then reminding data stream player.
In a preferred embodiment of the invention, server calculates it behind given network knowledge can do what, and sends message, with the service level that guarantees to light from a time.Time point when the client can begin play signal reliably sends to the client to " can begin to play " message.The preferred embodiments of the present invention have been considered the disposal ability of server, and are responsible for sequential processes according to priority, to guarantee the service level to the client.
The preferred embodiments of the present invention determine when that when send enough audio frequency and client should begin to play.The server client communication of control is realized in being used to send the agreement of data, and this information should be used to client application.
In this manual, be called as critical buffer point at the reproduction time of data flow and the difference of residue between the rise time of data flow.When reaching critical buffer point, just when critical buffer vanishing, reminding data stream player.
Advantageously, remind be issued after, the generating rate that makes data flow is phase same rate or rapid rate more.More advantageously, each data flow has priority, and priority is raised in generating resource, keeps speed so that be issued the back in prompting.
Preferably, generate resource from data flow at every turn and obtain the remaining rise time, it is compared with reproduction time.The residue rise time can estimate from the elapsed time of the initial estimation of rise time, or more advantageously, can make new estimation to the rise time later in initial estimation.The new estimation of rise time allowed the live load of the variation of the generation resource considered.
More preferably, prompting sends to the client from server.In a preferred embodiment, server has calculated the difference between generation and the reproduction time just, obtains in the change of server place rise time and the renewal of rise time at the server place because be easier to.Yet if the rise time is sent to the client, the client can calculate the golden hour of beginning played data.
At media data flow is speech data and when generating resource and being the Text To Speech engine, and described embodiment is the most suitable.TTS is interrupted especially easily, because it must broadcast with constant speed, but the generation engine of other type can use this technology to reduce the interruption of Media Stream, and described Media Stream for example needs the video and graphic of constant rate of speed output.Tts engine keeps the TTS controller to upgrade with the TTS rise time in TTS data flow transmission process.
Description of drawings
To only also embodiments of the invention be described with reference to the accompanying drawings now by example, in the accompanying drawings:
Fig. 1 is that the schematic client and the server of present embodiment arranged;
Fig. 2 is the flow chart of the controller server method of present embodiment;
Fig. 3 is the flow chart of customer controller method;
Fig. 4 A is the figure of the TTS server workload of example with respect to the time; And
Fig. 4 B is the figure of corresponding TTS generation and critical buffer point.
Embodiment
With reference to Fig. 1, show according to the client 10 of preferred embodiment and the layout of server 12.Server 12 comprises: TTS controller 14; Text To Speech (TTS) engine 16; And priority engine 18.Client 10 comprises: audio player 20; Buffer controller 22; And buffer 24.
TTS controller 14 is handled the TTS request from one or more clients 10, and uses TTS method 300 to carry out the TTS transmission from tts engine 16.
Tts engine 16 generates the TTS data flow according to the request from client 10.As starting a part that sends, tts engine 16 calculates and generates TTS data flow required time (rise time) and play TTS data flow required time (reproduction time).
Priority engine 18 is by carrying out the load balancing on the tts engine 16 according to the distribution of the priority of data flow control tts engine resource.At first, each data flow is assigned with average priority, but this priority can be changed during handling.If a data flow has higher priority than other data flow, then this data flow will be handled quickly than other data flow.Priority engine 18 is in response to TTS controller 14, and adjusts the distribution of tts engine resource thereupon.
The data flow that client buffer 24 receives from server 12, and store it till player 20 request msgs are used for playing.
Under simple situation, after pointing out by the interface of client's audio player 20 by the user, the request that buffer controller 22 starts data flow.Under more complicated situation, user interface is used by mutual voice response and is controlled, and after the certain user was mutual, the request to audio stream was made in described application.
Client's audio player 20 comprises having input and the interface of exporting.Input obtains and is used to the user command selecting data flow or be used to use mutual Voice Applications.
With reference to Fig. 2, described by the performed TTS controller method 200 of TTS controller 14.
Receive the TTS request from the client after, begin step 202.In step 202, TTS controller 14 calculates critical buffer point (CBP) by the reproduction time that deducts the TTS data flow from TTS the rise time.In a preferred embodiment, consider the live load of the size of the text be used to change and tts engine and calculate the TTS rise time.Another embodiment also will use the network work load in calculating.Replacedly, independent size text is the simple factors that provides the useful TTS rise time.
Step 204 is to start from tts engine 16 transmission TTS data to flow to client 10.When the TTS data flow was generated and sends, generation TTS data flow required time changed with the change of the load of tts engine.
In step 206, beginning is circulation continuously, recomputates new CBP according to the new TTS rise time.Tts engine 16 keeps upgrading TTS controller 14 with the TTS rise time in the transmission course of TTS data flow.In a preferred embodiment, improved precision widely under the continuous situation that recomputates the high work load that can change with different points to CBP in the TTS rise time.Yet useful embodiment also can comprise the CBP that only calculates once.
In step 208, check CBP, whether serve as zero or littler (that is, the remaining rise time is equal to or less than reproduction time) to check it, and it reach zero or littler before, be circulated back to step 206.In this cycle period, the TTS data just are being sent to client buffer 24, and the remaining rise time reduces.In case CBP reaches after zero, process just moves on to step 210.
In step 210, TTS controller 14 reminds client 10:CBP to reach zero by sending " can play-out buffer " message.
In step 212, TTS controller 14 submits to tts engine to generate and transfer rate by the priority that indication priority engine increases Data Stream Processing.
Step 214 is end of control method, though still but tts engine can generate still play data stream of TTS data flow and client.
Client buffer controller 22 uses customer controller method 300 to handle the TTS data flow.
In step 302, buffer controller 22 is to server 12 request TTS.
In step 304, buffer controller 22 receives the TTS data flow from server.
In step 306, buffer controller 22 is waited for to beginning play-out buffer always.
In optional step 308, buffer controller 22 is waited for the beginning play-out buffer.
In step 310, play-out buffer is played, and still receives the TTS data flow simultaneously.
Be the example of operation that needs to generate and to need in 12 seconds the preferred embodiment of the Text To Speech data flow of playing in 6 seconds below.
Fig. 4 A be the live load of TTS server 12 of example with respect to the figure of time, described server can be handled 0.5 second audio frequency by per second in preceding 2 seconds.After 2 seconds, because the live load that reduces, TTS server 12 can be handled 0.75 second audio frequency by per second.When server 12 at first received the TTS request, TTS server 12 can only transmit 0.5 second audio frequency in the per second of process.By from the time (12 seconds) of handling request, deducting the time (6 seconds) of playing request, determine 6 seconds critical point.After through 6 seconds, server process is dispatched, to send " can play-out buffer " signal to the client.
Yet after 2 seconds elapsed time, the load of server reduces, and now, system can transmit 0.75 second audio frequency in the per second of process.Time (6 seconds) by deducting playing request from the time (5/0.75=6.67 second) of handling request perhaps from receiving 2.67 seconds that initial request begins, is determined new critical buffer point (seeing Fig. 4 B) to provide 0.67 second.After 2.67 seconds rather than after 6 seconds, send " can play buffering " message (START_PLAY among Fig. 4 B).
In a word, a kind of method, equipment and computer program that the convection type deal with data is held consultation that be used for described.Particularly, it relates in server and to generate a large amount of continuous speech datas in real time and it is provided to client's method and apparatus.According on the one hand, the method for the media data flow that a kind of control generated from the data flow player plays is provided, comprising: the rise time of estimating to be used to generate data flow; The reproduction time of data estimator stream; Generate data flow by using data flow to generate resource, so that export by the data flow player; And if residue is equal to or less than reproduction time in fact the rise time, then reminding data stream player.In a preferred embodiment, server calculates it after the knowledge of given network can do what, and sends message, with the service level that guarantees to light from a time.Time point when the client can begin play signal reliably sends to the client to " can begin to play " message.The preferred embodiments of the present invention have been considered the disposal ability of server, and are responsible for sequential processes according to priority, to guarantee the service level to the client.
Claims (14)
1. method that is used on server the media data flow that control generated from the data flow player plays comprises:
Estimate to be used to generate the rise time of data flow;
The reproduction time of data estimator stream;
Generate data flow by using data flow to generate resource, so that export by the data flow player; And
If the residue rise time is not more than reproduction time, then reminding data flows player.
2. the method for claim 1 also is included in to remind and is sent out the back to keep generating rate be same speed or rapid rate more.
3. method as claimed in claim 2, wherein said stream generate has priority, and promotes described priority in generating resource, keeps speed so that be sent out the back in prompting.
4. as each described method of claim 1 to 3, the wherein said residue rise time obtains from data flow generation resource during generating data flow.
5. as each described method of claim 1 to 3, wherein said prompting sends to the client from server.
6. method as claimed in claim 4, wherein said prompting sends to the client from server.
7. as each described method of claim 1 to 3, wherein said media data flow is voice, and described generation resource is the Text To Speech engine.
8. system that is used on server the media data flow that control generated from the data flow player plays comprises:
Be used to estimate to be used to generate the device of the rise time of data flow;
The device that is used for the reproduction time of data estimator stream;
Be used for by using data flow to generate that resource generates data flow so that by the device of data flow player output; And
If be used for that residue is not more than reproduction time the rise time then the device of reminding data stream player.
9. system as claimed in claim 8 also is included in to remind and is sent out the back to keep generating rate be same speed or rapid rate more.
10. system as claimed in claim 9, wherein said stream generates has priority, and promotes described priority in generating resource, keeps speed so that be sent out the back in prompting.
11. as each described system of claim 8 to 10, the wherein said residue rise time obtains from data flow generation resource during generating data flow.
12. as each described system of claim 8 to 10, wherein said prompting sends to the client from server.
13. system as claimed in claim 11, wherein said prompting sends to the client from server.
14. as each described system of claim 8 to 10, wherein said media data flow is voice, and described generation resource is the Text To Speech engine.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB0508946.1 | 2005-04-30 | ||
| GBGB0508946.1A GB0508946D0 (en) | 2005-04-30 | 2005-04-30 | Method and apparatus for streaming data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1855824A CN1855824A (en) | 2006-11-01 |
| CN1855824B true CN1855824B (en) | 2010-06-23 |
Family
ID=34674214
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200610065474.XA Expired - Fee Related CN1855824B (en) | 2005-04-30 | 2006-03-22 | Method and apparatus for streaming data processing |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US8626939B2 (en) |
| CN (1) | CN1855824B (en) |
| GB (1) | GB0508946D0 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8891372B2 (en) | 2007-07-02 | 2014-11-18 | Telecom Italia S.P.A. | Application data flow management in an IP network |
| TW200919203A (en) * | 2007-07-11 | 2009-05-01 | Ibm | Method, system and program product for assigning a responder to a requester in a collaborative environment |
| US9076484B2 (en) | 2008-09-03 | 2015-07-07 | Sandisk Technologies Inc. | Methods for estimating playback time and handling a cumulative playback time permission |
| US8375095B2 (en) * | 2009-12-22 | 2013-02-12 | Microsoft Corporation | Out of order durable message processing |
| WO2014071971A1 (en) * | 2012-11-07 | 2014-05-15 | Telefonaktiebolaget L M Ericsson (Publ) | Pre-buffering of content data items to be rendered at a mobile terminal |
| US9734817B1 (en) * | 2014-03-21 | 2017-08-15 | Amazon Technologies, Inc. | Text-to-speech task scheduling |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1182875A2 (en) * | 2000-07-06 | 2002-02-27 | Matsushita Electric Industrial Co., Ltd. | Streaming method and corresponding system |
Family Cites Families (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH01130240A (en) * | 1987-11-16 | 1989-05-23 | Yokogawa Hewlett Packard Ltd | Data train generating device |
| US5583652A (en) * | 1994-04-28 | 1996-12-10 | International Business Machines Corporation | Synchronized, variable-speed playback of digitally recorded audio and video |
| JP2845162B2 (en) * | 1995-05-10 | 1999-01-13 | 日本電気株式会社 | Data transfer device |
| US5864678A (en) * | 1996-05-08 | 1999-01-26 | Apple Computer, Inc. | System for detecting and reporting data flow imbalance between computers using grab rate outflow rate arrival rate and play rate |
| US5928330A (en) * | 1996-09-06 | 1999-07-27 | Motorola, Inc. | System, device, and method for streaming a multimedia file |
| US6192406B1 (en) * | 1997-06-13 | 2001-02-20 | At&T Corp. | Startup management system and method for networks |
| US6112239A (en) | 1997-06-18 | 2000-08-29 | Intervu, Inc | System and method for server-side optimization of data delivery on a distributed computer network |
| US6138164A (en) * | 1997-11-14 | 2000-10-24 | E-Parcel, Llc | System for minimizing screen refresh time using selectable compression speeds |
| US6618820B1 (en) * | 2000-01-10 | 2003-09-09 | Imagex.Com, Inc. | Method for configuring an application server system |
| US6502139B1 (en) * | 1999-06-01 | 2002-12-31 | Technion Research And Development Foundation Ltd. | System for optimizing video on demand transmission by partitioning video program into multiple segments, decreasing transmission rate for successive segments and repeatedly, simultaneously transmission |
| US6466909B1 (en) * | 1999-06-28 | 2002-10-15 | Avaya Technology Corp. | Shared text-to-speech resource |
| US6557026B1 (en) * | 1999-09-29 | 2003-04-29 | Morphism, L.L.C. | System and apparatus for dynamically generating audible notices from an information network |
| US7649901B2 (en) * | 2000-02-08 | 2010-01-19 | Mips Technologies, Inc. | Method and apparatus for optimizing selection of available contexts for packet processing in multi-stream packet processing |
| JP2002091863A (en) * | 2000-09-12 | 2002-03-29 | Sony Corp | Information providing method |
| US7337231B1 (en) * | 2000-12-18 | 2008-02-26 | Nortel Networks Limited | Providing media on demand |
| US6766407B1 (en) | 2001-03-27 | 2004-07-20 | Microsoft Corporation | Intelligent streaming framework |
| US20040250273A1 (en) * | 2001-04-02 | 2004-12-09 | Bellsouth Intellectual Property Corporation | Digital video broadcast device decoder |
| US7430609B2 (en) * | 2001-04-30 | 2008-09-30 | Aol Llc, A Delaware Limited Liability Company | Managing access to streams hosted on duplicating switches |
| US7197557B1 (en) * | 2001-05-29 | 2007-03-27 | Keynote Systems, Inc. | Method and system for evaluating quality of service for streaming audio and video |
| US7200669B2 (en) * | 2001-07-31 | 2007-04-03 | Dinastech Ipr Limited | Method and system for delivering large amounts of data with interactivity in an on-demand system |
| US20030055910A1 (en) * | 2001-09-19 | 2003-03-20 | International Business Machines Corporation | Method and apparatus to manage data on a satellite data server |
| US7110995B2 (en) * | 2002-02-27 | 2006-09-19 | International Business Machines Corporation | Apparatus and method for generating graphic presentation of estimated time of completion of a server request |
| US7290057B2 (en) * | 2002-08-20 | 2007-10-30 | Microsoft Corporation | Media streaming of web content data |
| FI20021527A0 (en) * | 2002-08-27 | 2002-08-27 | Oplayo Oy | A method and system for adjusting bandwidth of a media stream |
| US20060092938A1 (en) * | 2003-02-26 | 2006-05-04 | Koninklijke Philips Electronics N.V. | System for broadcasting multimedia content |
| US7519845B2 (en) * | 2005-01-05 | 2009-04-14 | Microsoft Corporation | Software-based audio rendering |
-
2005
- 2005-04-30 GB GBGB0508946.1A patent/GB0508946D0/en not_active Ceased
-
2006
- 2006-03-22 CN CN200610065474.XA patent/CN1855824B/en not_active Expired - Fee Related
- 2006-04-10 US US11/400,952 patent/US8626939B2/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1182875A2 (en) * | 2000-07-06 | 2002-02-27 | Matsushita Electric Industrial Co., Ltd. | Streaming method and corresponding system |
Also Published As
| Publication number | Publication date |
|---|---|
| GB0508946D0 (en) | 2005-06-08 |
| CN1855824A (en) | 2006-11-01 |
| US20060248214A1 (en) | 2006-11-02 |
| US8626939B2 (en) | 2014-01-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9247276B2 (en) | System and method for progressive delivery of media content | |
| CN1855824B (en) | Method and apparatus for streaming data processing | |
| US7051110B2 (en) | Data reception/playback method and apparatus and data transmission method and apparatus for providing playback control functions | |
| CN100359949C (en) | Fast channel change | |
| US20030165150A1 (en) | Multi-threshold smoothing | |
| CN106572358A (en) | Live broadcast time shift method and client | |
| CN101406060A (en) | Time-delay video downloading service by using P2P content distribution network | |
| CN104471955A (en) | Methods and devices for bandwidth allocation in adaptive bitrate streaming | |
| JP2004527028A5 (en) | ||
| CN107911332A (en) | The system and method for media content streaming | |
| WO2010142226A1 (en) | Method, device and system for self-adaptively adjusting data transmission rate | |
| CN108184152A (en) | A kind of DASH Transmission systems two benches client code rate selection method | |
| CN109413448A (en) | Mobile device panoramic video play system based on deeply study | |
| JP2013509743A (en) | Method and system for individualizing content streams | |
| TW200904095A (en) | Network communication control method and system | |
| CN105900404B (en) | For the system and method for the dynamic transcoder rate adaptation of adaptive bitrate streaming | |
| CN106533932A (en) | Method and device for pushing instant message | |
| Liubogoshchev et al. | Adaptive cloud-based extended reality: Modeling and optimization | |
| WO2022174534A1 (en) | Resource requesting method and terminal | |
| CN103826139A (en) | CDN system, watching server and streaming media data transmission method | |
| CN107920108A (en) | A kind of method for pushing of media resource, client and server | |
| JPH09185570A (en) | Method and system for acquiring and reproducing multimedia data | |
| US8390739B2 (en) | CPU platform interface method and device for synchronizing a stream of motion codes with a video stream | |
| CN104581340A (en) | Client-side, streaming media data receiving method and streaming media data transmission system | |
| CN106657172A (en) | Method and device for realizing information push |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100623 Termination date: 20210322 |