CA2306524A1 - A system and method for enhanced streaming media viewing - Google Patents

Abstract

A system for providing an enhanced quality of service in a streaming multimedia system wherein streaming media is delivered from a media server to a requesting client media player. The streaming media is modified before transmission by the media server and includes a plurality of markers located at predetermined positions indicating new content insertion points. The media player receives the modified data stream monitors the bandwidth or data rate of the connection and uses the markers to insert locally stored content in the stream to provide playback thereof while buffering the received media stream. The media player resumes playback of the streaming media when the buffer reaches a predetermined level or the data rate of the connection increases.

Description

A SYSTEM AND METHOD FOR ENHANCED STREAMING MEDIA VIEWING
The present invention relates to networked multimedia systems, and more particularly to a system and method for improving the perceived quality of multimedia information delivered to one or more clients over a limited bandwidth network.
BACKGROUND OF THE INVENTION
The explosive growth of the Internet has resulted in a growing need to provide multimedia o material such as video and audio to users. Interactive television, movies on demand and other multimedia technologies are becoming more popular. This popularity coupled with an ever-increasing demand for a high quality of service (QoS) has placed significant demands on bandwidth and hardware requirements of the Internet.
15 The Internet, like other networks, is based on an IP protocol. Packets of data are routed around the network between a sender and recipient based on the address of the intended recipient contained in the packet. Thus, a specific connection between the sender and recipient is not required. Generally, packets may be sent via different routes in an effort to increase the likelihood of delivery, which means that data packets do not inherently have a guaranteed arnval 2o time. In fact, packets corresponding to a signal message may even be received out of order.
This fact significantly affects how certain multimedia data are delivered.
Since multimedia data requires real time delivery, a particular media stream needs to be delivered in the proper time sequence, to allow the user to play back the audio/video selection as real motion or real audio.
25 In order to support the delivery of realtime content over the Internet, a realtime protocol (RTP) was introduced to replace the conventional transmission control protocol (TCP). The RTP
provides a framework that real time applications can use directly for data transport. Currently, the RTP standard supports a particular type of message, namely one for carrying the media content data for streaming data. A separate protocol known as the realtime control protocol 30 (RTCP) is used with the RTP to pass control message for session management, rate adaptation and the like. Along with the realtime aspect of multimedia, as described above, multimedia data typically require an enormous number of bytes to represent the information contained therein.

Consequently, compression techniques are utilized to reduce the amount of data transmitted.
Generally, compression requires, an encoder to compress source data and a decoder to decompress the compressed data.
The quality of multimedia transmitted between a server and a client computer over a network depends largely on the bandwidth of the network. Bandwidth refers to the maximum number of bits that a network can transmit from one node coupled to the network to another node coupled to the network. If insufficient bandwidth is available, the number of bits transmitted is reduced and consequently, the image or sound of the multimedia is distorted or not fully represented. In to many computer networks, the primary element which limits the bandwidth is the network interface device which interfaces the server and client computers to the network. In the Internet realm, such interface devices include modems and Ethernet controllers.
A solution around this problem is the introduction and adoption of broadband delivery systems such as asymmetric digital subscriber lines (ADSLs), cable modems, and ATMs, to name only a few. These broadband delivery systems are now capable of offering the consumer a relatively inexpensive access to the Internet. While these broadband delivery systems have made the reception of the streaming media more satisfying to the user, there is still a limitation in the ultimate content that can be provided. Currently, the streaming media content is still limited to 2o the reception of simulcasts of radio broadcasts over the Internet, video clips, and very low quality or low definition television.
Even though improved protocols, hardware, compression and decompression techniques and increased network bandwidth are being introduced, the number of users requiring or requesting such streaming media is increasing proportionally. Thus, users are still experiencing fluctuations in bandwidth in the network causing choking and picture freezing, making streaming media still annoying to the end user.
Accordingly, there is a need for a system and method that is capable of allowing streaming media such as television and movies to be transmitted over the Internet while providing the user with a perceived transmission improvement over current transmissions for realtime television, audio transmission or the like.
SUMMARY OF THE INVENTION
It is an advantage of the present invention that streaming media provided to a user appears to flow seamlessly.
According to an embodiment of the present invention there is provided a method for transmitting streamed digital data from a server, the server being configured for coupling to a client computer 1o via a computer network. The method comprises the following steps. A
modified data stream is provided to the media player, the modified data stream including a plurality of markers located at predetermined positions for indicating new content insertion points. The modified data stream is transmitted to a requesting client which monitors the data rate of the received media. The client inserts locally stored content at one or more marker positions while buffering the received data IS upon the data rate being below a predetermined level. Playback of the streaming data is resumed upon the data rate returning to the predetermined level.
BRIEF DESCRIPTION OF THE DRAWINGS
An embodiment of the invention will now be described by way of example only with reference 2o to the following drawings in which:
Figure 1 is a schematic block diagram of a multimedia delivery system;
Figure 2 is a schematic block diagram of a media player according to an embodiment of the present invention;
Figure 3 is a flow chart depicting a method for multimedia display according to the present 25 invention;
Figure 4(a) and 4(b) are schematic diagrams of a data buffer in a first state;
and Figure 5(a) and 5(b) are schematic diagrams of a data buffer in a second state.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following description, like numerals refer to like structures in the diagrams. Referring to figure 1, a streaming media system is shown generally by numeral 100. The system 100 includes a streaming media server 112 for serving media files 114 over a network 116 to a requesting client 117. The client 117 is typically a personal computer, hand-held computer or similar device, which includes a memory, a processor and software for providing control instructions to the media server and for displaying the received media information on a display device 119.
Typically, this client software is termed a media player and executes within a web browser such as Windows Internet ExplorerTM or Netscape NavigatorTM. Alternately, the media player may be 1o executable without the need for a web browser, such as RealAudioTM. The client computer also includes a storage device 118, such as a hard disk or RAM memory or similar.
On the server side 112, there is included a specialized piece of software known as the media server which may perform one or more functions of compressing the media data, multicasting and multistreaming.
Most commercially available media servers stream the media information using one or more 15 protocols over the Internet standard TCP protocol. For example, a media server by VivoTM
offers an HTTP based streaming that enables almost any web site to offer streaming video clips, other vendors like VDONet, offer a UDP (user datagram protocol) based server.
In general however, both the media server and the client player are matched to communicate via a mutually recognized protocol. The latter media player is downloaded by the user when attempting to 2o access the particular media server 112. Content for the media server 112 may be derived from prestored video, television, audio or similar. Alternatively, the content may be provided by a live feed 120 such as a radio broadcast or television transmission.
While the above describes the basic components of a multimedia system, the present invention 25 provides an improvement to this system in a manner to be described below.
The improvement allows the existing multimedia system architecture as described above to be utilized in order to provide the user with a perceived higher quality display of multimedia information. For example, real motion, high resolution video should be optimally displayed at 24 to 30 frames per second. An advantage of the present invention is to achieve such a perceived quality of service.
3o The present invention attempts to replicate a user's familiarity with the format of current television broadcast programs to create the impression of smooth real time real motion video in a multicasting or multistreaming environment. More specifically, television viewers are accustomed to television programs which typically in five to ten minute segments followed by advertising or commercials which run for two to three minutes. These commercials are inserted in the broadcast video by local distributors. These commercials are generally known in advance of the broadcast.
Refernng now to figure 2, a schematic diagram of a media player according to an embodiment of the present invention is shown generally by numeral 200. The player includes an executive 202 for controlling and conducting operation of the media player and a user input interface 204 for t o receiving user commands; a module for determining the incoming data rate 206; a cache 208 for buffering an incoming data stream under control of the executive 202, a storage for prestored information 210, and a display module 219 for transferring the received media stream to the display 119 as determined by a scheduling algorithm.
15 The operation of the system may be explained with reference to figure 3.
Initially, television commercials or other predetermined information is transferred to the client computer for storage in the prestored information memory 210 at one or more of a plurality of predetermined time periods. These commercials will be used in lieu of streaming on-demand commercials from a server, and will be inserted automatically by the media player 200 when triggered by a marker as 2o discussed with reference to figure 4.
Commercial clips can be downloaded using several methods. The clips can be downloaded when the user switches the computers on. They can also be scheduled to download on a different connection to the streaming video connection in which case priority should be given to the video 25 streaming. Commercials can also be downloaded during the normal commercial breaks that are provided by the broadcasters. While local prerecorded commercials are playing, additional downloaded new commercial clips could be downloaded through a dedicated URL.
Furthermore, new commercials can be transmitted to a user's computer during off peak hours, while the computer is still connected to the Internet.

In order to facilitate the invention, the media to be delivered from the media server is modified by the addition of markers to the streaming content. Markers may be inserted in the data stream broadcast by a broadcasting station indicating the positions in the streaming media wherein the prestored information of commercials 210 may be inserted. These markers may be of various forms. For example, a first type of marker termed "a broadcast marker" may instruct the media player as to the positions for inserting commercials or a second type of marker termed a "supplementary marker" may be inserted in the video stream to indicate positions in the stream where the streaming may be interrupted without affecting the smoothness or the viewing of audio content as perceived by the user. The insertion of the markers maybe done manually on 1o prerecorded material or by using a rule-based image analysis algorithm. The rule-based image analysis algorithm may be used to determine critical points, such as a scene change in a video stream or a pause in an audio stream for insertion of the markers.
Referring back to figure 3, at start up, the media player checks if the media cache buffer 208 as shown in figure 2 has sufficient content for display. If not, the prestored sequence of commercials is sent to the display from the storage 210. The media player continues to check the buffer 208. Upon the buffer being at a sufficient level, the media content is played. In the case of video, the video images are displayed or in the case of audio, the audio is played. The media player 200 continues to stream data into the buffer 208 while playing. As described earlier, the 2o streaming data rate will depend on the bandwidth of the network. If the average bandwidth, as determined by the media player, is lower than the bandwidth for real time continuous playback then the buffered data is used for playback. Consequently, the buffered data begins to decrease.
When the buffer size reaches a critical point where choking is predicted, the playback is faded out and local commercials are shown from the storage 210, while the buffer 208 to rebuild itself to a sufficient capacity.
In order to avoid interruptions in video sequences which might be annoying to the user, the scheduling of the local content done at the appropriate marker position.
3o If the buffer 208 is depleted below the critical level, the scheduling algorithm is initiated.
Refernng to figure 4 (a) a schematic diagram of the buffer 208 is shown wherein the buffer contains a "broadcast marker" 402. The marker 402 notifies the user that a commercial is to be embedded. The buffer is allowed to deplete beyond the critical level 404, and the local commercial is switched in at the broadcast marker 402.
Refernng to figure 4(b), the buffer 208 does not contain a "broadcast marker"
but does contain "supplementary markers", which is provided by a distributor server? The local commercial will be shown at the supplementary marker that is the last one in the buffer.
The algorithm for displaying local content is a dynamic algorithm. For example, if there are two 1o supplementary markers in the buffer at detection time, and while the buffer depletes, a third marker arnves, the scheduling algorithm will wait for this latest arnval before playing the local information thereby allowing the maximum playback time without interruption.
During local information display, the buffer 208 fills up to sustainable level at the end of the ~ 5 current local information playback at which point normal video streaming resumes.
If, due to excess choking, a large number of commercials were played, the scheduling algorithm adjusts the local content playback where possible and resumes normal media streaming as soon as possible, perhaps even skipping scheduled local information playback. A
total number of 2o commercial broadcasts are stored in an accumulator. The accumulator is increased for excessive numbers of commercials. Referring to figure 5(a), there is shown a buffer having a broadcast marker 402, which is encountered with a relatively full buffer. In this particular circumstance, other broadcast markers are present in the buffer and the commercial quota has been exceeded.
Therefore, the local content playback is skipped until the next broadcast marker. Refernng to 25 figure 5(b), there are no other broadcast markers 402, but there are supplementary markers and the commercial quota has been exceeded. The local content playback showing is delayed as long as there are markers in the buffer.
During the periods illustrated in figures 5(a) and 5(b), the streaming media server transmits 3o markers while the buffered stream is being played. If the markers can be skipped as previously described, the accumulator is decreased appropriately. When the accumulator is empty, normal local content synchronization can start again at the next available marker.
The size of the buffer 208 is calculated with respect to the statistical measurement of average bandwidth on the channel, and the critical level is determined as a percentage of a full buffer.
In addition to the above scheduling scheme, the case of sustained low bandwidth is addressed. If the average bandwidth drops below the required streaming needs for an extended period, and the local commercial scheduling has been exhausted, an auxiliary connection with the server may be 1o established to instruct the server to decrease the frame rate of the transmission. In that case the server will drop, for example, to 25 or 24 frames per second, or even to 15 frames per second as required. When bandwidth quality resumes to an acceptable level, the server will increase again the frame rate to a higher level, such as 30 frames per second. The synchronization between the client and the server is controlled so that the content is played back.
Another feature for improving noticeable quality is having the media player check for the existence of other connections or routing having a higher bandwidth. When this is detected, the client connection can seamlessly switch to the better connection (similar to telephone channel switching over satellite for overseas routing).
In summary, the present invention provides a new media player and a modified content which is used by the media player to improve a user's perceived quality of service of the playback media.
As described earlier, streaming media servers use a client/server arrangement to allow insertion of streaming media advertisements (commercials). The client end begins a new stream for each inserted commercial in association with a downloaded script. In an alternate embodiment, the streaming media server uses a server push model to insert streaming media advertisements directly into a live broadcast media stream or pre-recorded stream. The server inserts the commercial at a pre-determined time and location in the streaming media, and does not require 3o any feedback or communication from the client.

In a further application of an embodiment of this invention, an on demand marketing and on demand advertising system may be implemented. In this instance, the client computer receives instructions from a user or viewer via a keyboard, mouse or any other suitable input means. The instructions received by the client is forwarded to the server for requesting certain types of advertising content. The content may be specified by the viewer or user in order to receive specifically targeted marketing information during a streaming media session.
More specifically, during a streaming media session, the client may during the playback of local content such as a commercial, interact with the client computer to connect to an advertiser streaming media site.
These sites may offer such things as motor vehicles and may also include a live video interactive to session. In this instance, the client is provided with a web camera or similar for communication directly with a live person. Thus, the client may include a teleconferencing camera with an audio input to enable a user view to talk live to a customer service representative of the advertiser or the like. During this interaction, the player may continue to stream video into the buffer for resuming playback once the user has terminated the interactive session.
A further embodiment of the invention provides for a plurality of channels which are available to the client. Each channel having a unique set of commercials. The server inserts markers corresponding to an appropriate set of commercials or locally stored media in the appropriate channel indicated provision of the allowable content. These in markers may be read by the 2o client, inform the client as to whether the appropriate group of commercials is to be played or not. Generally, the group of commercials may be a predetermined sequence of advertising clips which are thematically related.
The inserted commercial can reside on the same server as the pre-recorded clip or broadcast, or reside on a remote commercial server, and be automatically inserted at the media server end.
In yet another embodiment, a client pull streaming media model allows for communication between the streaming media server and client for determining when commercials are to be inserted. The server generates a script that is read by the client. The client then makes a request for content based on this script. The script can also be modified by the client based on a set of rules or information that it has gathered from the end user.

Once the client has made the request for insertion of content, the server responds by modifying it's current stream of information, or launches another stream for the commercial. After the commercial has completed playing, the client issues a request to resume the original content stream.
In yet an alternate embodiment, a targeted streaming media ad insertion model allows the streaming media server to gather user information from client applications installed on a user's computer.
The server collects information from the client application based on the following criteria: The user's responses to forms and surveys; the user's viewing habits and timelines; the length of viewing; the number of commercials served; the type of content viewed; the user's "hot click"
actions; the user clicking on a commercial to launch a separate HTML/DHTML/XML/JAVA
window; purchases based on commercial content; the user's geographic location;
and any other information gathered as statistics or usage logs The server then sorts and analyzes the information into a user profile, and allows for narrow target marketing. All of the data collection for this model is based upon a permissions system 2o that requires user permission to use and analyze collected data.
Further, a streaming media portal site is provided as a human indexed directory of video and audio on the Internet. The portal is being provided as a preferred location on the Internet for showcasing and indexing rich media. Users are able to search for video and audio clips that are hosted on other companies' servers, and find scheduled live events from around the world.
The use of human indexing allows for a better quality Internet experience versus sites such as YahooTM, InfoseekTM, and AltavistaTM that feature computer indexed content.
Video and audio are best experienced and described by humans.

The streaming media portal site is positioned extremely well. Current sites indexing video and audio are fragmented and developed by companies with an interest in featuring their own media formats. By offering all video formats, the streaming media portal builds customer loyalty, and can become the one stop for all media on the Internet.
The site will be expanded with weekly programming guides delivered directly to users, chatrooms, sponsored live events, and other community building features.
Although the invention has been described with reference to certain specific embodiments, 1o various modifications thereof will be apparent to those skilled in the art.

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE PROPERTY OR
PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A method for transmitting streamed digital data from a server, said server being configured for coupling to a client computer via a computer network, the method comprising the steps of:
(a) providing a modified data stream to said media player, said modified data stream including a plurality of markers located at predetermined positions for indicating new content insertion points;
(b) transmitting said modified data stream to a requesting client;
(c) monitoring the data rate of the received media at the client;
(d) the client inserting locally stored content at one or more marker positions while buffering the received data upon said data rate being below a predetermined level;
and (e) resuming playback of the streaming data upon said data rate returning to said predetermined level.