WO2007102165A2

WO2007102165A2 - A system and method for push browsing of digital content on mobile devices synchronized with ongoing broadcasts

Info

Publication number: WO2007102165A2
Application number: PCT/IN2006/000078
Authority: WO
Inventors: Divya Deepika Bhasin
Original assignee: Divya Deepika Bhasin
Priority date: 2006-03-07
Filing date: 2006-03-07
Publication date: 2007-09-13
Also published as: WO2007102165A3

Abstract

The present invention envisages a system and method to push digital content on to a mobile device, time synchronizing it with ongoing broadcasts. A clock-sync operation is performed (330, 380) to determine the approximate difference between client and server clocks, if any. A time based synchronized data transfer operation takes place by sending informative content blocks to the client in advance, so that the client stores them in its cache (410, 440). The contents blocks are displayed on the mobile device from cache or directly from the server, based on the clock-sync operations and information contained in content blocks. These steps are repeated continuously so that the digital content is time synchronized with broadcast information on a server.

Description

A SYSTEM AND METHOD FOR PUSH BROWSING OF DIGITAL CONTENT ON MOBILE DEVICES SYNCHRONIZED WITH ONGOING BROADCASTS

BACKGROUND

Field of the invention

The invention relates generally to Push Browsing of digital content on a mobile device. In particular the present invention relates to providing a system, method and computer program to improve experience of a user who is using a mobile phone to view digital content that is in synchronization with a broadcast, and to interact with the broadcast via the digital content.

Description of the related Art

Figure 1 is a block diagram of the prior art that shows the current data access mechanisms prevalent in the mobile device as well as computers. These are based on the client-server mechanism. A client requests the content from the server, upon user request. While the user waits, the server sends the content to the client. Once the client has received the data from the server, the client renders the content on the computer or mobile phone display. Depending upon the speed of connection between the client and server, the transfer of data may take unpredictable amount of time. The delay is the main determinant of the user experience in these browsing situations. In addition, all action is on user request, in other words, the user is "pulling" information from the server. Figure 2 depicts two tables of the current scenario with client side and server side activities. In the step 1, a user uses the mobile application (e.g. a browser). The application requests the content from the server depending upon user inputs. In step 2 the server returns the requested content to the client. In step 3 the user waits while the content is returned to the client. When the content is received from the server, client renders it on the mobile phone screen. Step 1 , 2 and 3 are repeated continuously with the user. In this manner the user has to wait for each new content to be delivered from the server before it is rendered on the mobile screen.

Data services on the mobile devices are becoming more available and affordable. A new and innovative data service is related to viewing and interacting with digital content which is time synchronized with broadcast information either on a TV channel or a radio station. So the user is listening to the radio or watching television while at the same time he/she can connect their mobile devices to digital content corresponding to the same broadcast channel. The complementary visual content can be viewed on the user's mobile, including options for interactivity such that the user can interact with the broadcast using the keypad/ cursor on the mobile device.

One of the efforts in the direction of client-server time synchronization has been US patent no. 5,790,805. This patent discloses a method for establishing an unlimited number of independent, client-based timers, synchronized with a timer kept on a central server. After forming a client-server connection, a client sends a synchronization message to a server. The client receives a return synchronization message from the server, and computes a round-trip interval time between sending and receiving, by sampling a local hardware clock. The sending and receiving of synchronization messages continues for a predetermined number of times, until the client receives a final synchronization message from the server, the final synchronization message including the current local server time. The client then calculates the average one-way trip interval and adds that value to the received current local server time, to provide the client with a reliable estimate of the local server time. By calculating the difference between the client's own local time and the calculated local server time, a constant is derived which can be used to calculate local server time for any and all future client local times.

However, what is required is a system and method that applies time synchronization in a case of mobile device during a live broadcast, where the server has more content information than the client and this information is pushed to the client in advance. In this case the server pushes the content towards the mobile device well in advance. The information delivered from the server already includes the time when it should be rendered on the mobile phone. The client uses this information to render the content on the mobile device at the specified time.

Therefore, what is required is a system and method that overcomes the above deficiency in the prior art to provide for a system and method to time synchronize the digital content on a mobile device.

SUMMARY

An object of the present invention is to provide a Push Browsing solution for mobile devices, in order to reduce and eliminate the delay in the user experience in various browsing situations. The first application of Push Browsing is outlined for mobile devices providing multimedia interactivity that is time synchronized with TV and radio broadcasts.

Another object of the present invention is save future Content information in the client that allows the user experience to be optimal, so that the visual content is absolutely in sync with the parallel broadcast. Yet another object of the present invention is to optimize the network bandwidth so that the server does not need to resend Content information that it knows is already available with the client

The present invention requires the client and server software to work collaboratively in the ways described below:

1. The client and the server must ensure that the digital content displayed on the mobile device is in time sync with the broadcast content being relayed on the broadcast channel (TV & radio). For instance, if the TV station plays a car advertisement at 9:03:30 then the mobile device should also be displaying the corresponding digital content related to the car advertisement. To achieve this, a time synchronization mechanism needs to be established between the client on the mobile device and the server.

2. To ensure that step 1 above can be achieved, the server must send the required content to the client in advance of the display time of that content. If the server sends the required data precisely at the time when it should be displayed, the network data transfer latency can result in unpredictable delay and the consumer will not have the optimal experience with the service. To achieve this, a caching scheme must be implemented in the client that can store some of the content in its memory prior to it being used. In addition, in order to optimise network bandwidth usage, the server is at all times aware of the content present in the client, so it sends only what is absolutely required.

3. Another requirement for achieving step 1 is that the client must know when to display particular content on the mobile device. To facilitate this, when the server sends the digital content, it sends it in terms of content blocks, and in them it appends information related to the display of the content block on the mobile device. For example, the server will add information that has the time that the content block must be displayed, the duration of the display, the reusability of that content block and other useful information. These are detailed later in another section. The "display time" information is utilized by the client to render the content block at the correct time on the mobile device.

The present invention provides a system and method to achieve all the 3 objectives mentioned above in order to implement Push Browsing. This allows the user experience to be optimal, so that the visual content is absolutely in sync with the parallel broadcast.

The present invention envisages a method to push digital content that is sent to a mobile device, such that the digital content is synchronized with that of a broadcasting server. A clock-sync operation is performed to determine the approximate difference between client and server clocks, if any. A data transfer operation takes place by sending informative content blocks to the client in advance, so that the client stores them in its cache. The contents blocks are displayed on the mobile device from cache or directly from the server, based on the clock-sync operations and information contained in content blocks. These steps are repeated continuously so that the digital content is time synchronized with broadcast information on a server.

BRIEF DESCRIPTION OF THE DRAWINGS The preferred embodiments of the invention will hereinafter be described in conjunction with the appended drawings provided to illustrate and not to limit the invention, wherein like designations denote like elements, and in which:

Figure 1 is a block diagram depicting current data access mechanisms prevalent in the mobile device as well as computers (prior art).

Figure 2 depicts two tables with client side and server side activities (prior art).

Figure 3 is a flow chart depicting client-server time synchronization mechanism on the client side.

Figure 4 is a flow chart depicting client-server time synchronization mechanism on the server side.

Figure 5 is a flow chart depicting content block handling by client.

Figure 6 & 7 are is a flow charts depicting content block handling by server.

While the invention is susceptible to various modifications and alternative forms, specific embodiments of the invention are provided as examples in the drawings and detailed description. It should be understood that the drawings and detailed description are not intended to limit the invention to the particular form disclosed. Instead, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS The present invention deals with complementary visual content on the mobile device. As per the invention, the user tunes to the broadcast station on radio or TV, and then connects the specially provided client software on the mobile phone to the same broadcast station. A specially designed content server already holds the digital content, which will be The present invention deals with complementary visual content on the mobile device. As per the invention, the user tunes to the broadcast station on radio or TV, and then connects the specially provided client software on the mobile phone to the same broadcast station. A specially designed content server already holds the digital content, which will be provided to the client software for display on the mobile device.

To explain this further, an example of email is taken. Many prevalent email systems require that the user should 'pull' information from a centralized email server. While this is an acceptable form of checking new emails in a computer system connected through a always-on computer network to an email server, this model has some limitation when extended to the mobile devices capable of data services. A mobile phone user is not tied to any location hence the concept of 'pulling' or 'checking' his emails from the centralized email server, while technically possible, is very cumbersome and tedious for the users. Mobile phones users are used to messages coming to them.

Many mobile phones and other handheld devices now have a Push email client on them. When the push email client is running, new email messages arrive at the user's mobile phone without any effort or activity from the user. In many cases, a audible tone also notifies the user of the incoming email message.

A similar paradigm is applied in the proposed invention where the concept of browsing (with implied pull techniques) is extended to Push

Browsing (based on push techniques). The first application of Push

Browsing is then outlined for Multimedia Interactivity with TV and Radio broadcasts. In this case, the broadcast content server holds all the information related to the ongoing broadcast, and has information on what the client on the mobile device must display at what time. So once the client is connected to a particular broadcast station, instead of the client "pulling" information from the content server, the server "pushes" the required information on the client mobile device, in advance of when it is to be displayed, so that when the time comes to display that content, it is already available on the client device, and can be displayed with no delay. The content which is pushed to the client has the information that needs to display as well as when it needs to be displayed.

Therefore, the present invention involves 3 main steps:

1. Time Synchronization

2. Content Block caching on client side

3. Content Block information and display

The first step is to establish a time-sync mechanism between the client and the server. This time-sync mechanism syncs the client clock with the server clock and also takes into consideration the round-trip delay of the network. To establish this, the client sends its current time (TO) to the server. The data reaches server at T0+Tu. The server sends back TO and its own clock time to the client. Let us assume it reaches the client back at TO+Tu+Td. Meanwhile the client clock has moved to T2 such that T2 = TO + Tu + Td. Assuming that Tu + Td is twice the one-way lead-time, the client can sync with the server clock.

Let us illustrate this with an example. Imagine that when server clock shows 9:12:30, the client clock is showing 9:11:30 - meaning that client clock is 1 minute behind the server clock. But this is precisely what needs to be established at the client end. So the client sends 9:11 :30 to the server. Let us assume that it takes 15 sec for the data to reach server. The server receives it at 9:12:45 and sends it back with the information 9:11:30/9:12:45. The information then reaches back to the client at 9:12:00. So now the client is able to determine that the roundtrip delay is 30 sec and thus the server clock was at 9:12:30 when the client clock was at 9:11:30. After this, client can display the content at the server requested time (by server standards) using its own clock. The time sync mechanism described here can be repeated very often to get a better estimate of the skew between the server and client clocks.

Figure 3 depicts the client server time synchronization mechanism on the client side. User starts a application on a mobile device 300 that seeks connection with the server 310. It checks whether the connection is successful or not 320. If the connection is successful, the client sends its own time T_cιr_ent to the server 330. After some time the client receives a response at time T₂ from the server with server time Tgerver- 340. Thereafter, the client calculates the network delay T_d by deducting the time T₂ with that of T_Cii_ent350 and also calculates T_server by taking into account the network delay T_d 360. Client clock adjustments Tg are calculated by taking difference of T_C|_ient and T_server and client clock Tciient is then adjusted accordingly 380.

Figure 4 depicts the same client server time synchronization mechanism on the server side. Upon receiving the connection request from client 410, it seeks to receive T_CHentfrom the client 420. Thereafter, it sends back T_cij_ent and its own time to the client T_server430. Upon completion of time synchronization at the client end, the server starts sending content data to the client.440.

In addition to the above, a number of other time synchronization methods are currently available or may emerge in time to come. These mechanisms may include an external signal received by the mobile device through:

1. Atomic time-keeping; or 2. An external signal received by the Bluetooth radio integrated within the mobile device; or

3. An external signal received by the Infrared device integrated within the mobile device; etc.

Irrespective of the mechanism used for time synchronization, the objective is to ensure that the time used by the client to render content on the mobile device is synchronized with the time on the broadcast server.

Once the client server time synchronization is established, it is important that the digital content to be displayed is available in the client's memory before the time to display it arrives. This means that the client has to cache this information in its local memory. The server sends the digital content to the client in the form of Content Blocks (CB for short). The Content Blocks contain one block of information to be displayed on the mobile phone for particular time duration. Each

Content Block contains at least the following information:

1. A unique Content Block identifier

2. Actual digital content to be rendered on the client screen corresponding to a certain part of the broadcast (e.g. a picture, a text box, a video clip etc.)

3. The time(s) at which the content is to be rendered. The same content may be rendered more than once. The time used is the server time. 4. The length of time (duration) for which the content is to be rendered on the mobile device unless the user presses a key causing other action to take place 5. Lifetime indicator, providing information on the probability of this CB's reuse 6. Details of possible payment methods, such as Premium SMS,

Credit Card, Bank debit, other internet based payment mechanisms, no payment etc. This is relevant for CBs that may trigger some payment from the users. 7. Other details related to the soft-keys on the phone for this content and the actions to be taken in case they are pressed

The server sends the digital content proactively, well before time to the client in the form of these Content Blocks. The time information as to when that content should be displayed on the mobile device is included within the Content Block as described above. The client program scans its memory for content blocks to be displayed at a particular time. With the proposed scheme in place, the client program should find the desired CB in its local memory, thus being able to render it precisely at the same time as the corresponding content on the broadcast channel.

The client program has to manage its memory efficiently. It would like to retain CB's that have possibility of reuse and discard others. In order to intelligently discard CB's from its memory, it uses the information provided by the server in the Content Block - basically the lifetime indicator. If a CB is to be rendered only once and its lifetime indicator shows no likelihood of reuse (a value of 0), this would be an ideal candidate to discard. On the other hand, CB's that are to be rendered repeatedly or have high probability of being reused are stored in the client program's local memory as described below.

The server sends Content Blocks to the client in anticipation of what is to be displayed. In many situations, the same Content Block may need to be displayed more than once. The suggested algorithm attempts to minimize any resends of the same CB from the server, thereby preserving network bandwidth. The client program knows to keep at least 'n' immediately previous CB's in memory whose lifetime indicator is not 0 (which means they have a possibility of reuse). In addition to these n "old" CB's, the client program also continues to store in its memory CB's for future display. Depending on its memory size, it can store as many future ones as possible. The server at all times knows which n "old" CB's are cached in the client, and it ensures that it does not resend these ones. If the client is getting inundated with future CB's from the server and its memory capacity does not allow it to store any more, it will send a message to the server to temporarily withhold sending any more CB's, and will also include in the message the CB-ID of the last stored CB in its memory. Thereby the server, when it restarts sending CB's to the client, will know which CB to start from. The time after which the server restarts will be a configurable parameter (for example, 2 minutes).

Figure 5, 6 and 7 describe content block handling from the client side as well as server side. Figure 5 is a flow chart depicting content block handling on the client side. Mobile Device displays the content block 510. Thereafter it checks for various conditions for the content block. It checks for the lifetime indicator of the content block. If the same is 0 (i.e. it is not required in the future) 520, it discards the same 525. Further, if the content block is already in the memory 530 it does nothing 535. If the space is available in the client memory 540, the content block is stored as the most recent one 545. If there are more than "n" or more content blocks stored in the memory 550, then discard the oldest content block and store the latest content block 555. If the client memory is full, a message is send for "not enough memory" 560. Also when the client receives next content block from the server 570, it stores the same in the memory 580. It checks if the memory is full 590,

Figure 6 & 7 are flow charts depicting content block handling by server. In figure 6, the content block is initialized to a given unique identifier 610. The server selects content block to sends to the client starting from last identified content block 620. This last identified content block is then checked whether it is present in the list of content block that client saved in the past. 630. If it does then the server increments the last content block identification 635. If this content block is not present in the list of the content block then it is send to client 640. If the server receives the message that there is an overflow of client memory 650, then the last content block that was sent is stored as the one sent successfully 655. In case no such message is received, it is checked whether the number of content blocks sent is maximum for the client 660. If not then the process of selection of content block is again repeated 620. If the maximum of the number of content blocks is achieved then the number is stored to 0670. The server application thereafter waits for another pre-defined time or trigger.

Figure 7 again depicts the handling of server side content block. The server maintains the list of past content blocks in the client. It keeps track of such content blocks that client must have displayed 710. If there is no enough memory message from the client 720, then the list of past content blocks is initialized to Unusable 725. If the lifetime indicator of current the content block is '0' 730 then server does nothing 735. Similarly if client saves this content block and is already in the list of past content blocks 740, then again server does nothing with the list 745. If however it is not saved in the list, the content block identification is saved in the list 750. However, if there are "n" or more content blocks stored in the list 760, then discard the oldest content block 770.

There are two main advantages of this scheme. Firstly, saving future CB's in the client allows the user experience to be optimal such that the visual content is absolutely in sync with the parallel broadcast. This is because the client does not have to fetch CB's from the server as and when they are required. Secondly, saving past CB's helps in optimizing the network bandwidth so that the server does not need to resend CB's that it knows are already available with the client.

Of course in this scenario there is a small possibility that the client at some point does not have in its local memory the correct CB to display, in which case it has to wait for the server to deliver it. The user experience in such a situation is no worse than the current scenario without this scheme in place. While the preferred embodiments of the invention have been illustrated and described, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the invention as described in the claims.

Claims

CLAIMS:What is claimed is:

1. A method to push digital content that is sent to a client, in time synchronization with the content of a broadcasting server, said method comprising the steps of:

a) performing a clock-sync operation to determine the approximate difference between client and server clocks, if any;

b) carrying a data transfer operation by sending informative content blocks to the client in advance, so that the client stores them in its cache; and

c) displaying the content blocks on the client from cache or directly from the server, based on the clock-sync operations and information contained in content blocks,

whereby steps a, b & c are repeated continuously so that digital content is time synchronized with broadcast information on a server.

2. A clock-sync operation of claim 1 further comprises the steps of:

connecting the client and the server applications, so as to send client time T_C)i_ent to the server;

receiving response from the server with its time T_serv_er and calculating the network delay; and

adjusting the client clock to synchronize its time with that of the server clock.

3. A clock-synch operation of claim 1 , further comprising the mechanism of the client receiving an external signal through atomic time-keeping.

4. A clock-synch operation of claim 1 , further comprising the mechanism of the client receiving an external signal by bluetooth radio integrated within the mobile device.

5. A clock-synch operation of claim 1 , further comprising the mechanism of the client receiving an external signal by infrared integrated within the mobile device.

6. A data transfer operation of claim 1 , further comprising the steps of sending digital content proactively to the client well before its display time, so that the client could locate this in its local memory for displaying at a particular time.

7. A data transfer operation of claim 1 , wherein the content blocks contain life time indicators to indicate their reusability for the purposes of cache.

8. A display operation of claim 1 , wherein the content blocks contain display time information, that is used by the client on the mobile device to display the content at the correct time, synchronized with the broadcast time.

9. A system to push digital content that is sent to a client, time synchronizing it with the content of a broadcasting server, said system comprising of:

a clock-sync module to determine the approximate difference between client and server clocks;

a time based synchronized data transfer module for sending informative content blocks to the client in advance, so that the client stores in its cache; and a module to display said content blocks on the client from cache or directly from the server, based on the clock-sync operations and information contained in content blocks,

10. A system as recited in claim 9, wherein the client is a mobile device, phone, a computer, PDA, etc.

11. A system as recited in claim 9, where the content blocks contain least information such as identifier, time to render contents, actual contents, length for which content is to be rendered, life time indicator, soft key details, etc.

12. A system as recited in claim 9, wherein content blocks are stored in cache depending on their lifetime indicator information provided by the server.

13, A system as recited in claim 9, wherein the cached content blocks are displayed on mobile device resulting in savings of network bandwidth as well as synchronized display.