EP1384362A2

EP1384362A2 - System and method for operating internet-based events

Info

Publication number: EP1384362A2
Application number: EP02721389A
Authority: EP
Inventors: Scott G. Newnam; Izet Fraanje; Martin J. Cribbins; Andrew J. Borleske; Douglas T. Neumann; Ronald Tache
Original assignee: GoldPocket Interactive Inc
Current assignee: Ericsson Television Inc
Priority date: 2001-03-13
Filing date: 2002-03-13
Publication date: 2004-01-28
Also published as: WO2002073925A2; WO2002073925A8

Abstract

In interactive Internet-based system and method provides a number of components that can be used to provide interactivity, including chat, polling, and trivia functionalities, particularly for use in enhancing other events, such as television programs.

Description

SYSTEMANDMETHOD FOROPERATINGINTERNET-BASED EVENTS

BACKGROUND OF THE INVENTION The present invention relates to a system and method for producing interactive multi-user events.

Interactive entertainment has existed for several years in the forms of video games and computer games. Using the Internet, users can compete in multi-player games using computers and Web-enabled consoles. The fast penetration of the Internet into the majority of the U.S. households has provided a sudden increase of interactive content and the ability for end-users to interact. It has also promoted the popularity of interactive Internet programming for education and entertainment.

Interactive Internet programming is applicable in a variety of markets, such as the interactive entertainment market. Currently, interactive entertainment is delivered either via the TV with side-by-side personal computer ("enhanced broadcasting") or the personal computer alone ("online interactive entertainment"). Enhanced broadcasting, providing data that supplements a video broadcast, enables consumers to purchase products or request product information (e.g., literature, coupons, or sample merchandise), access additional information (e.g., news, sports, or weather), or otherwise interact with television programming (e.g., TV game shows or polls).

Due to the dominant penetration of the personal computer, the majority of interactive entertainment is delivered via the personal computer. As the penetration of broadband Internet connectivity increases, and platform development continues, interactive entertainment may migrate to the TV with a web-enabled television, a set- top box ,or an Internet-connected console. Additionally, some interactive programming, such as simple game applications, will be available on handheld devices via the Wireless Application Protocol (WAP) and on future Internet devices.

The likely convergence of television and the Internet, and the growing demand for interactive entertainment, has created a need for interactive programming that encourages the evolution of entertainment from passive television to the types of interactive entertainment that will be possible once broadband Internet access and digital television are ubiquitous. In recent years, television networks have offered enhanced television programs and events. These offerings allow users to request additional information and to interact with the programming.

SUMMARY OF THE INVENTION

An embodiment of the overall system of the present invention includes a user- based hardware device with a controller, client software on the hardware device, server-based interactive components, and a technical director for interacting with the server and the client via the server. The client preferably has three levels of software: core or basic control software, content based on the type of event, content for a specific instance of an event. For an interactive television application, the type of event and instance of that event would be a program and specific episodes, respectively.

The interactive components can include chat functionality, whereby users can discuss program and event content, use buddy lists to create private chat rooms, and use instant messaging. These features are particularly suited to interactive games whereby users can build rivalry with, encourage, or goad each other, to raise the level of excitement and competitiveness of head-to-head match-ups. The system can perform other interactive processing to enable users to influence and drive content changes in real time. For example, the present invention can support the following other types of interactive processing: (i) real-time, interactive question-and-answer- based games (e.g., trivia) wherein a multitude of users compete against each other in real time; (ii) real-time voting and polling wherein, audience members vote to determine broadcast content (e.g., choose-your-own-adventure) or provide then- opinion on broadcast content (e.g., issues polling) and (iii) instantaneous reporting of game, voting, and polling results. The system can also be used to create games payable by the user without interaction with a server.

The producer of the interactive event can control the presentation of animated movies, audio, video streams, participant polls, trivia questions and answers, and messages through the technical director. During the event, participants can participate in chat rooms and send messages back to the host producer. The producer can change the content or generate new content based on the questions and comments from the users or the course of a live event.

The client (user) software system (i) provides premium functionality, (ii) can be used on different types of hardware platforms, (iii) is presentation layer independent, (iv) contains a messaging library that enables synchronization, and (v) is efficiently leveraged for multiple interactive events. The client used in the present invention has three layers. The first layer has several components that each provide various types of functionality. These components can be utilized with various hardware platforms (personal computers, Macintosh, set-top box, digital television, web console, handheld WAP devices, etc.) and various types of presentation software (Shockwave, Flash, etc.). The second layer has scripts, graphics and other data specific to the producer's program (e.g., a weekly show). The third layer has data specific to an event available at a specific time on a specific date (e.g., an episode of a program). This content can be made to expire to minimize storage space and protect copyrighted materials. A user participating in two different programs can utilize one version of the first layer for both programs, two versions of the second layer, and during the particular event, will utilize software from the third layer.

The system enables large television-sized audiences, i.e., thousands or more remote people, via Internet infrastructure and not on local networks to interact with each other by using a messaging code that tells the user interface when and how to present the content.

The system of the present invention (i) allows users to impact the programming in real-time; (ii) allows users to communicate with a producer of the programming; (iii) allows users to interact with each other; (iv) is scalable to a television-sized audience; (v) provides users with a similar experience regardless of modem speed; and (vi) maintains synchronization among users. The system of the present invention thus provides an improved medium for interactive entertainment by enabling consumers to interact with each other and to impact the course of the event.

The system of the present invention can scale to accommodate large audiences by (i) maintaining the state of the event across multiple servers; and (ii) utilizing global software and hardware load balancing that guarantees each user a spot in the event while distributing the load across the network of servers.

The present invention synchronizes the 'interactive content' to other content such as a television broadcast. The system provides a mechanism to generate real- time Internet content that can be synchronized to the TV broadcast and synchronize content already present on the end-users system with the TV broadcast.

The present invention enables a rich media experience for all users regardless of connection speed. By providing a rich media solution for narrowband users, the present invention enables the incorporation of sound and video to enhance the interactive entertainment. For example, rich media can be used in (i) game questions: users are questioned on video clips, (ii) advertising: sound and video enable more compelling advertising messaging, or (iii) supplementary content: behind-the-scenes footage, star interviews, etc. The present invention delivers the same rich media experience to users with high or low Internet connection speeds by using a messaging system to send message types from the event servers to the user interface which controls the delivery/ display of existing rich media content.

The invention can be used to provide advertising and e-commerce. The system's capacity to support advertising and e-commerce encourages additional commerce. The present invention enables advertising that unites the best of television and Internet models by uniting interactivity with broad reach. Users can request information on demand, enabling advertisers to create powerful, customized messages that influence consumer-purchasing behavior.

All the interactive functionality/content such as polls, trivia, chat, audio, and advertisements can be synchronized with each other. The present invention includes a mechanism to, for example, (i) highlight a picture of the person whose voice is currently in the audio stream, (ii) play a video clip related to the trivia question currently being displayed, or (iii) display a graphic related to the subject of a poll.

The inventions further include systems and methods for creating interactive events that run on or with existing or future interactive entertainment and television systems and enhance broadcast programming (over a TV, radio, Internet, or other medium). An embodiment of such a system includes at least some of the following components: a composer that has authoring tools and is capable of generating multimedia events; a content tool, which is an interface used to insert content into the interactive events generated by the composer; a simulator for testing a completed event; and a technical director for directing the timing and display of the content within the event. The system can be customized so that completed interactive event output files make up the required components for events on various interactive television systems.

The content creation components of the present invention create output files specific to the event or program being created, and also output files specific to certain instances of a recurring event or episodes of a program. The event and episode could be combined into one, such as for a specific movie or sporting event. The interactive events contemplated by the present invention can include several different types of interactive components, all of which can be customized and populated with content using the system and method of the present invention.

A content creation system according to an embodiment of the present invention preferably has a graphical interface that is convenient for event developers to use to create pieces of the interactive event, such as through drag and drop icons and drop down menus, yet rapidly creates the code necessary to interface with code used by the interactive television and other entertainment systems to provide rich interactivity. The composer allows event developers to quickly and cost-effectively create events. The composer creates both client components and server components such that a producer with the technical director on the server side can control the interactive enhanced content. The client and server components are preferably created at the same time when the producer uses the content creation tools. The content can be scheduled for display on a client's television or monitor for a specific time, created for future use at the discretion of a producer, downloaded in advance and responsive to a message from the technical director, downloaded during the event but in advance for display during that event rather than for instant display, or stored at the server side for transmission at a later time.

A developer of the interactive event can designate time and space intervals for animated movies, audio, video streams, participant polls, games, and messages using the composer and/or content tool. The animated movies, audio, video streams, poll text, trivia question and answer text, and messaging text can be added to the event using the content tool.

Spontaneous content can also be provided live (on the fly). During the event, the technical director controls the timing of the content display and the addition of spontaneous content. Participants of the event can participate in chat rooms and send messages back to the host producer. The producer can change content or generate new content based on the questions and comments from the users or the course of a live event. Because the components and the content of interactive events vary, the options available in the technical director will likewise vary. The content creation system according to an embodiment of the present invention uses the composer to generate a customized technical director.

Interactive events created by the present invention can (i) allow users to impact the programming in real-time; (ii) allow users to communicate with a producer of the programming; (iii) allow users to interact with each other; (iv) scale to a television- sized audience; (v) provide users with a similar experience regardless of modem speed; and (vi) maintain synchronization among users.

All the interactive functionality/content such as polls, games, chat, audio, and advertisements can be synchronized with each other. The present invention also includes other interactive mechanisms, for example, to (i) highlight a picture of the person whose voice is currently in the audio stream, (ii) play a video clip related to the trivia question currently being displayed, or (iii) display a graphic related to the subject of a poll.

A system according to an embodiment of another aspect of the present invention can be used to connect different client devices, such as PCs, set-top boxes, net-top devices, and wireless devices, using Internet Protocol (IP) to provide true interactivity, targeted advertising, and mechanisms for true convergence, preferably by leveraging the MPEG 4 standard set by the Motion Picture Experts Group (MPEG). A system of the present invention serves as a centralized system for connected all such external devices. Furthermore, such a system can connect closed systems, such as cable headends, and can allow different such systems to act as one cohesive system.

The system and methods of this aspect of the present invention provide a comprehensive distribution and delivery system for interactive entertainment and for coordinating various types of interactive programming on various client devices (e.g., PC, set-top, net-top, consoles, or wireless) and to facilitate interaction with each other, and with on-line and/or on-air programming. This is preferably accomplished by using the advantages of IP to integrate such devices.

A system according to another aspect of the present invention includes some or all of the following components: a server system that is capable of running multiple simultaneous interactive events; a software system including several independent components of an interactive program; an Interactive Program Guide (IPG) for assigning users to certain servers based on selected interactive programming; interactive content that populates the software components; an Electronic Programming Guide (EPG) that communicates the selected program to the IPG; a personal video recorder (PVR) that allows the user to time-shift television programming; a personal interactivity recorder (PIR) that records the interactive content and results associated with the recorded video signal; a controller that receives both industry standard triggers and also other message types and controls interactive content presentation in response to such triggers and message types; a content display interface that determines the layout of the interactive components based on the client hardware device and user preferences; and a client hardware device, such as a PC, set- top box, net-top box, console, or wireless device.

The system can provide all users with a similar interactive experience even though they might use one of several client hardware devices, and/or they might have different Internet connections speeds, they may or may not use an EPG, they might time-shift the programming using a PVR. The design of the system also creates opportunities for a new level of interactive and targeted advertising. A television producer can utilize the system to provide related and synchronized interactive content for a television program. For either a live or taped broadcast, the producer can use the technical director to control content previously created and distributed to the client devices and to send dynamic content during the broadcast.

The interactive content is provided using Internet Protocol (IP) and is therefore independent of the distribution of the television content. Accordingly, a producer of interactive content can also utilize the system to provided interactive content related to and synchronized with a television program without the cooperation of the television producer. If either the television producer or distributor is not participating in or cooperating with the interactive programming, message types sent over IP would determine the timing and content of the interactive programming.

If the television producer and the program distributor were cooperating in the production and distribution of the interactive content, they could utilize industry standard triggers (e.g., ATVEF triggers). ATVEF triggers are currently a standard set by the Advanced Television Enhancement Forum, in which URLs are embedded in the video signal during post-production to trigger interactivity. Cooperation between the television producer and the distributor is necessary for the use of some industry standard triggers because the distributor can remove them after post-production. The present invention can utilize these triggers, or an IP based messaging system, or a combination of both methods. The system of the present invention is flexible in terms of which types of triggers take precedent. The configuration of a specific event would determine the hierarchy of triggers. It can be configured to allow all types, take one over another or only allow for one type.

Existing set-top and net-top systems do not support the ability for the participant to affect the sequence and outcome of events in a program. With true interactivity according to the present invention, during a sports talk show, for example, the responses to polls and trivia questions can drive the commentary and direction of the show's host. Previous systems allowed users in a set-top or net-top world to merely play along with a show, without information being shared in real time with other participants or with the producer. An example might be playing along with a question-and-answer game show. In previous solutions, the television viewer would simply see a visual overlay on the screen allowing him or her to select the correct answer at the same time as the show. However, the participant's score is simply calculated on the set-top box for the participant's own knowledge. In fact, the participant has not in fact "interacted" with others. With the system of the present invention, information on every participant can be transmitted to a server system for real-time processing. Such processing allows for interactivity, such as display of aggregated results, bracketed tournaments for competition, fast display of competitor rankings, and voting for choices of a show's outcome.

The systems and methods of aspects of the present invention provides flexibility in interactive advertising by allowing (1) the display of advertisements that enhance the on-air advertisements without competing with them, (2) the ability to provide targeted or local advertisements, or (3) eliminating interactive advertisements to retain attention for the on-air ad. The system allows for a more targeted distribution of advertisements. For example, a broadcast might have three overlapping advertisements and the chent hardware device displays one of the ads based on user preferences and experience. The system and method of the present invention allows for one of several enhanced advertisements to be delivered to the chent hardware device based on similar preferences or according to the on-air ad being distributed.

Some targeted advertisement systems currently being deployed with television systems and set top boxes utilize profile and experience information stored on the set top box to select for display one of several ads being transmitted. The present invention allows users to answer questions, participate in polls, request information, obtain coupons, locate stores, etc. by interacting directly with the server system. Key information is stored or processed at the server system and can be used to more effectively target advertisement delivery. This results in lower bandwidth consumption and allows for a larger potential number of targeted ads or information to be distributed since every user does not have to receive every piece of content.

Many consumers currently use electronic program guides (EPGs) to inform them of available and scheduled programming and to bring them to the selected television program. If an EPG is resident on a chent hardware device, the system of the present invention allows the EPG to interact with the Interactive

Program Guide (IPG) of the present invention to communicate the selected program and assign the user to the appropriate Interactive Program Cluster of the server system.

The system of the present invention addresses problems with interactive programming in combination with time-shifting devices, such as personal video recorders (PVRs), which are individual storage devices used with the television (e.g., TiVo recorders from Tivo, Inc.). Any device that has network and video receiving/ recording capabilities can be considered a time-shifting device; e.g., a PC can have video receiving capability and can store broadcast programming and other content.

In one embodiment, a PIR (personal interactivity recorder) engine collects links to interactive content associated with a particular video frame and/or interactive content itself and records that interactive content. When the broadcast is replayed, the content from each link or the content itself is displayed as the show progresses. Playback works the same as in the previously described interactive mechanism except the content does not need to be accessible from a server after the event, and the chent device does not even have to be connected to the network. This eliminates the need for extra network storage costs, and lowers bandwidth requirements. Alternatively, the user could be connected to the network during playback. The playback feature with interactive content provides a potentially more enjoyable experience for the participant because, similar to Hve participation in the interactive program, the participant can compare his or her responses to the rest of the audience.

Another advantage of the system and method of the present invention is provided by the content display interface. The content display interface determines the size, shape, and location of the various components of interactivity based on the user's client hardware device and preferences. In one example, a user participating in interactive programming using a PC and separate television would see the entire PC monitor occupied by interactive components while a user on a net-top device might see components arranged on the left and bottom of the video screen while the television program occupies the remainder of the screen. In another example, a participant in a multi-player game not associated with a television broadcast would use the entire screen even though the participant is using a set-top, net-top or console as his client hardware device. Another participant might select only to receive programming information as a ticker along the bottom of the screen. In yet another example, two users sharing the same display might choose to each have a component for polls and contests while sharing a chat box and fun fact display. The content display interface allows users to select from available options, which may be limited based on the type of client hardware device and program.

The system of the present invention aUows for a single interactive system to provide interactivity to and among users of various client hardware devices including digital head-end set-top boxes, net-tops, wireless devices, consoles, PCs and digital televisions. The present system preferably connects to each chent device using Internet Protocol. Because each client device and sub-system (e.g., Scientific Atlanta based head-end or Sprint based wireless) is slightly different, the system connects at different points in these devices, but maintains a constant communication via Internet Protocol through ah systems. The system uses a software component referred to as an event management engine (EM Engine) to receive ATVEF triggers and other message types over IP and use them to control the interactive program.

Other features and advantages will become apparent from the following detailed description, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high level block diagram of a system showing the chent, server system, and tech director.

FIG. 2 is a block diagram of a server system according to an embodiment of the present invention.

FIG. 3 shows an example of real-time processing of a poll by the server system.

FIG. 4 shows an example of real-time processing of a trivia question by the server system.

FIG. 5 shows an example of the display of an advertisement.

FIG. 6 shows the process of submitting a question to a producer. FIG. 7 is a flow chart relating to initialization.

FIG. 8 is a block diagram of an event-creation system showing a composer, content tool, technical director, and simulator.

FIG. 9 is a block diagram of an exemplary implementation of event specific content components that are generated by authoring tools on the chent, server system, and technical director side.

FIG. 10 is a flow diagram of the authoring tools and the output of the authoring tools that create an event.

FIGS. 11-13 show examples of a composer, a content tool, a technical director, and some of their respective components.

FIG. 14 is a block diagram of a server system that coordinates interactivity across multiple client devices over Internet Protocol.

FIG. 15 shows an example of the coordination and aggregation of interactivity with a poll by the server system across multiple client devices.

FIG. 16 shows the comparison of the system of the present invention for synchronization to broadcast over IP and the prior art embedded ATVEF triggers.

FIG. 17 shows an example of the convergence of on-line and on-air programrning into a combined MPEG4 stream over Internet Protocol.

FIG. 18 illustrates a thin communication protocol between a server system and set-top chent that provides true interactivity.

FIG. 19 is a block diagram representing the synchronization of targeted on-line advertising to the on-air broadcast over Internet Protocol across multiple chent devices.

FIG. 20 is a flow diagram of coordination of EPGs across multiple platforms and the association of interactivity to these EPGs.

FIG. 21 is a block diagram of a solution for interactive programming and time- shifting devices. FIG. 22 shows the output of a content display interface on different user devices.

DETAILED DESCRIPTION

The system of the present invention creates the basis for true interactive on- line programming. This programming can be Web-only events or it can be synchronized with other activities, such as television programs, to enhance the progi-amming content. The system provides a number of forms of interactive processing, chat, and e-commerce. Content can be brought down to the chent ahead of time and accessed through messages, or content can be created and delivered on the fly as a program is being broadcast. Participants can all be in-sync regardless of differences in their respective modem speeds. When used with television or some other activity, the present invention preferably operates independently of the television signal, although the system allows broadcasters to provide content that is related to the television program being broadcast, such as trivia questions about actors in a movie that appears at the same time.

A content provider, such as a television network, can broadcast television programming to a television via cable, satellite, over-the-air, or other means. Before that telecast, the system allows the broadcaster to develop information and to send that information to a user's hardware device using the Internet infrastructure with messages preferably sent using available communication protocol.

FIG. 1 shows components of an embodiment of the present invention. The system includes a user hardware device 50, a chent 100 on the hardware device 50, a remote server system 200, and a technical director 500. The user hardware device 50 is a processor that can be a personal computer (meant here broadly and regardless of the specific operating system), a set-top (cable) box, a digital television, a Web enabled console, a handheld device connected to the Internet via Wireless Application Protocol, or any other processing device with an interface for connection to the Internet. The chent 100 is the software, preferably a thin piece of software, that resides on the user hardware device 50 and that is responsible for communication with server system 200. The system allows multiple clients 100 to simultaneously interact with server system 200. Chent 100 can interpret messages and content sent by server system 200.

The chent 100 can be part of a plug-in to a web browser or a stand-alone application.

It has multiple objects that have different responsibilities such as content management, messaging, chat, and multi-media display and streaming.

The technical director 500 is a software interface (e.g., a web page) on a computer and resides between a producer of the event, such as a television broadcaster, and the system. The technical director 500 allows users to impact the content of the event in real-time by enabling the producer to change the content based on input from the users. The technical director 500 can also, for example, receive, display, and allow the producer to select questions posed to the producer or event's host or comments made by the user. The producer can respond to the questions or comments through the real-time creation of event content. The new content could be multi-media content distributed by a broadcaster or content created by the technical director 500. For example, the technical director 500 allows producers to generate real-time content (e.g., a real-time fun fact or poll) and to trigger the display of such content on the chent 100 utilizing the server system 200.

As shown in FIG. 2, server system 200 of the present invention has multiple components. A chat component 210, allows users to write messages to, and read messages from, each other during an event. The chat component 210 is responsible for managing the information exchange between users. The user enters information using hardware device 50. The client 100 processes the entry and provides the information to the server system 200; the chat component 210 pubhshes this information out to the other clients 100 participating in the event or participating in the chat aspect of the event. The chat component 210 can also filter certain language, such as potentially offensive language, allow users to enter different chat rooms, allow users to chat with specific users that are on their "buddy lists" based on other users' personal identifiers, and provide other features that can enhance this chat feature. As can be seen from FIG. 2, the server system 200 can contain multiple chat components 210.

An event component 220 is responsible for interactive processing for activities such as games, poUs, and fun facts. Event component 220 initiates communication with the chent 100 to trigger display of content through different types of messages. The chent 100 and event component 220 utilize a messaging code to minimize data transfer. For example, if the event component 220 sends the following message: "2, Who holds the record for most homeruns in a season?, Micky Mantle, Babe Ruth, Mark McGwire, Sammy Sosa", the chent 100 will recognize the message as a message type 2 (in this case, meaning trivia question) and display the text "Who holds the record for most homeruns in a season?" as a trivia question and display the four choices that follow as possible answers. Alternatively, if the event component 220 sends the message "3,Who is the best hitter of all time?, Micky Mantel, Babe Ruth, Mark McGwire", the chent 100 will recognize it as message type 3 (in this case, a poll) and display the information as a poll, such as with radio buttons. The client displays these types of interactivity in a desired way, and with captions in the boxes in which they are displayed, based on the message type received.

The client 100 can be set to send back the answer to the trivia question or poll based on a defined occurrence, such as after a predetermined amount of time, when the user inputs the answer, or when the producer requests the results using the technical director 500. In all cases, the client 100 sends the message back to the event component 220. The event component 220 receives, aggregates and sends the results to the controller (see also FIG. 3 and FIG. 4). In most cases in which information is sent back as a response, the time for responding is short relative to the event, e.g., 60 seconds or 30 seconds or less. This allows the answer or poll result, for example, to be broadcast to all users without long waiting times. As shown in FIG. 2, the server system 200 can include multiple event components 220.

A multimedia component 230 is in charge of the broadcast and distribution of multimedia, such as audio and video, and interacts with a broadcaster 250 and a reflector 240. The broadcaster 250 is an encoding device that creates the appropriately formatted multimedia and distributes it to multimedia components 230 through the reflector 240. The controller 270 and the reflector 240 can also be executed as one component. As shown in FIG. 2, the server system 200 can include multiple multimedia components 230.

The controller 270 coordinates among the different components in server system 200, the multiple instances of the same component (e.g. multiple event components), and the communication between the components and the technical director 500. The controller 270 synchronizes the other server system components, which in turn manage the synchronization of all the clients 100. The controller 270 receives messages from the technical director 500, and distributes the messages to the apphcable server system components, which distribute the messages to the clients 100.

The initialization component 280 prepares the end-users system for programming participation. It initializes the user's computer by installing or enabling the chent 100 by downloading over the Internet infrastructure. The first time this happens, the user may be required to enter an identification and password.

Referring also to FIG. 7, the client 100 can have three components: chent base 115, client program 125, and chent episode 135. These three components together allow the user to participate in an event. However, when a user participates in a subsequent event with the same user hardware device 50, the initialization component 280 preferably only downloads the client components that are not present already on the user hardware device 50. For a new episode of a television show, these components would include a new client episode, and need not include the client base or chent program, but could include updates or enhancements to the client base and chent program.

In another embodiment, the server system 200 can directly interact with the user device 50 without the chent 100 and the three specific components shown in FIG. 7. In another implementation, user device 50 has code similar to client base 110 already resident on the system before participating in any programming related to the system of the present invention, to which the server system 200 can link. For example, software of the base software type may be included in a device as part of its operating system. In another implementation of the system of the present invention, the client program 120 can be 'flashed' down for each episode depending on the type of user hardware device 50, rather than being initialized.

When the system of the present invention is used to enhance television broadcasting content, the first time that the user uses the system, the user downloads the chent base, the chent program for the particular program being enhanced, and the chent episode for the next episode of the program. The base software includes code that manages the system and interacts with the components, including providing and displaying chat messages and handling different message types for display to the user.

The program content can include images that will be displayed with a particular program whenever that program is enhanced with the system of the present invention. For example, this content can include a background display to be shown on the monitor and can include other information, such as biographical information about actors, that does not change from episode to episode. This content need not be downloaded for each episode (although enhancements and updates typically would be).

The episode content that is downloaded can include content relating to the specific episode of the program being shown and can include, in advance, certain fun facts, trivia questions, or other items that can be accessed. In a cooking show, for example, the episode content could be recipes that will be made, and can include additional recipes to further enhance the televised content. The episode content can include a time-out feature such that the content is essentially deleted at the end of the episode. If a user wants to watch a future episode, the user can download the relevant episode content.

The terms program and episode should be broadly understood to mean any type of event and episode or instance of that event. In addition, there could be one- time events in which the client program and chent episode are essentially a single chent piece.

FIG. 3 illustrates in detail the flow of interactive content through the present invention using a poll as an example. The technical director 500 sends a poll 302, including a message type and text for the poll 302 to the server system 200. The relevant components of the server system 200 send out the poll 302 to the chent 100. The chent 100 interprets the message type poll and displays the text as a poll on the user hardware device 50. The user device 50 accepts a poU answer 306 by the user, and returns the poll answer 306 to the server system. The client 100 can be set to send the poll answer 306 when the chent receives one of a number of possible triggers, such as a message from the technical director 500 via server system 200, an input from the user through the user hardware device 50, or automatically after a specified time, preferably on the order of 30 seconds or less. The server system 200 processes and aggregates the poll results. After this processing (only a matter of seconds), the aggregated poll results 304 are sent back out to the chent 100 and displayed on the user hardware device 50.

FIG. 4 shows another example of interactive content flowing through the system of the present invention. A trivia question 400 can, for example, reside on the server system 200 or be provided from the technical director. If on the server, the technical director 500 can initiate the sending and the display of this trivia question 400 in real-time. This ability allows the producer to synchronize the display of the question with hve multi-media content (e.g. an audio stream of the host reading the question or a video clip related to the question), and to synchronize with the program. The technical director 500 sends trivia message type 405 and the server system 200 then takes the trivia question 400 and sends it on to the chent 100 along with the message type 405. The client 100 interprets message type trivia 405 and displays the text in trivia question 400 as a trivia question on the user hardware device 50.

The user selects an answer and the client 100 sends back the trivia answer 410 to the server system 200 when it receives one of a number of triggers, such as a message from the technical director 500, an input from the user through the user hardware device 50, or automatically after a specified time, preferably on the order of 30 seconds or less. The server system 200 processes the trivia responses 410 and determines if each response is correct or incorrect. The server system 200 sends back the score 415 indicating whether the answer was correct or incorrect and the time used to answer the question.

Alternatively, the correct answer can be sent with the trivia question 400, and the client 100 can determine if the response is correct based on previously downloaded information and then record the time for sending the results (correct or incorrect) and the time back to the server system 200. The system can display a running decrease in the value of the question based on the time it takes to respond - e.g., answering in 2 seconds may be worth one number of points, and in 8 seconds a reduced number of points. In either case, the server system 200 aggregates and records the results. The server system 200 can send out a message type indicating a leader board and the leader board text 420 to the chent 100. The chent 100 interprets the message type as a leader board 420 and displays the leader board text 455 showing the cumulative results of the trivia contest. The leaders can be shown based on user input names and shown to all users.

FIG. 5 gives an example of an advertisement trigger and display. As discussed relating to FIG. 2, the episode specific content can be brought down to the chent 100 at the beginning of the event by the initialization component 280. An advertisement 510 (e.g., in the form of a graphic image like a banner ad) can be part of the episode specific content. This ad can thus already reside on the client 100 but is not displayed until triggered by a message type from the technical director 500 to display that particular ad. The technical director 500 sends message type ad 505. This message type is distributed by the server system 200 to all the clients 100 at the same time. Once the client 100 receives message type ad 505, the chent causes the ad to be displayed. The ad can be a link to another web site if selected by the user, such as with a mouse or a remote control. In that case, a separate browser instance is opened while the user accesses the web site.

FIG. 6 shows another example of interactive processing of the present invention. The user can submit a question or comment to the host or producer. The chent submits a question or comment to the host by entering the question or comment in the client 100 using the user hardware device 50. The client 100 sends the question or comment 605 through the server system 200 to the technical director 500. The person/system directing the technical director 500 (the producer) can select questions or comments 610 submitted by the users. The producer can convey the questions or comments to another host of the program or respond to the questions and comments using multi-media content or real-time generated fun facts or polls. The producer can also select the question or comment and send the selected question or comment 610 back to the chent 100 through the server system 200 for display by the user hardware device 50.

The technical director 500 can thus send messages in different ways, such as a message type and message content, such as text, to be provided to the client for processing the message type and using the text; a message type that causes content in the server system to be provided to the chent; or a message type that is provided to the server system 200 which is provided to the chent for causing content already resident on the chent to be displayed. As also indicated, the technical director can cause content to be displayed at predetermined times, or hve and on the fly in response to questions or comments from users, or in response to something that is occurring during the programming.

In the case of scheduled information, a program may be divided into segments to be separated by commercials. The system can be used to program those segments which can then be initiated automatically or with human intervention when commercials end, as it is possible for the length of commercials between segments to be adjusted very close to the airing of the program.

The interactive functionality, such as polls, trivia, chat, audio, and advertisements, can be synchronized with each other. The present invention allows the system to highlight a picture of the person whose voice is currently in an audio stream, play a video clip related to the trivia question currently being displayed, or display a graphic related to the subject of a poll.

The chent can be used on different types of hardware platforms and can be made presentation layer independent. Because of the way the chent is downloaded in pieces and because content can be directly provided or reside and accessed through messaging, it is efficient in its use of memory space, and is efficiently leveraged for multiple interactive events. A user participating in two different programs can utilize one version of the first layer for both programs, two versions of the second layer, and during the particular event, will utilize software from the third layer.

The system of the present invention can scale to accommodate extremely large audiences by maintaining the state of the event across multiple servers; and utilizing global software and hardware load balancing that guarantees each user a spot in the event while distributing the load across the network of servers.

FIG. 8 shows components of an embodiment of a system according to the present invention for creating content and an interactive event generally. An authoring composer 300 is a software program residing on a computer and is the basis for the event creation process. The authoring composer 300 has tools, responsive to a content creator, to allow the content creator to create interactive events to enhance programming. Such tools can include convenient drag-and-drop icons and drop down menus. The authoring composer 300 allows producers (or those working with or for broadcasters) to produce a skeleton event. This production can include some or all of the following: identifying segments in the event, determine interactive functionality in the different segments, such as chat between viewers, interactive communication with the producer, trivia, polling, and fun facts, and also setting the visual appearance of the event.

The content tool 600 is used to collect and create episode specific content for an event scheduled in the future. Content tool 600 is a software interface, such as a web page, on a computer and resides between a producer of the event, such as a television broadcaster, and the server system. The content tool 600 allows users to produce and store content for a specific episode of a series of events.

The technical director 500 is accessible to the producer during an event to interact with the system and to monitor and control the event. The technical director 500 aUows a producer to control the content of the event in real-time by enabling the producer to change the content based on input from the users. The technical director 500 can also, for example, receive questions, display questions and comments, and allow the producer to select questions posed to the producer for display to sohcit comments from viewers. The producer can respond to the questions or comments through the real-time creation of event content. The new content could be multimedia content distributed by a broadcaster or content created by the technical director 500. For example, the technical director 500 allows producers to generate real-time content (e.g., a real-time fun fact or poll) and to cause the display of such content.

The simulator 400 allows event developers using the system to test events and or episodes created in their own environment (e.g., desktop computer). It provides mechanisms to simulate the interactive entertainment/television system on which the event will be running. The simulation is designed to provide a high level of reliability.

As shown in FIG. 9, an embodiment of the present invention has several components that are generated to create a specific event. The underlying interactive system that provides the infrastructure to actually run events generally includes a chent 100, a server system 200, and the technical director 500. The chent software runs on an end-user hardware device, such as a home computer or set top box. An event may have a series of episodes that have common features but with content or specific implementation that can change across multiple episodes. In the chent 100, there is a client event shell 110 that defines the features and contains graphics and behavior that are constant across multiple episodes, i.e., the general program content. The chent episode files 120 are the elements on the client side that each relate to a specific episode. Examples of elements in the client event shell 110 include a background graphic that is present in each of the episodes and biographical information about individuals who appear regularly in multiple episodes. The client episode files 120 contain, for example, the specific graphics for an episode (e.g., advertisements) that are typically downloaded in advance of the broadcast. These can include banners that can be clicked by the user if desired. The elements in client event shell 110 can be reused for future episodes and thus do not have to be initialized for each event, although they may be updated during a download or flashing process.

Parallel to the client 100 and within the server system 200 is a server event shell 215 that defines the specific features of an event. An example of a feature that is defined in the server event shell 215 is a chat server component. If chat is included in the server event shell 215, the server system 200 will provide this functionality. The server episode files 225 define the specifics for that episode, e.g., the names of the chat rooms for that episode.

The tech director 500 has a tech director event shell 515 and tech director episode files 525 that respectively define the event and the specific episode within the event similar to the client 100 and server system 200. The tech director event shell 515 can, for example, define the availability of polls in a segment of the event, while the tech director episode file 525 contains the specific content for a poll in a segment.

As described above, content for a specific episode can reside in three locations: the client episode files 120 on the chent 100, the server episode files 220 on the server system 200, and the tech director episode files 525.

FIG. 10 shows an exemplary implementation of the system of FIG. 8 in more detail, including a flow for the authoring tools and how the event-specific and episode-specific files are created. The authoring composer 300 generates a specific event (and series of episodes within this event) on the underlying system. The authoring composer 300 is an application that runs on a hardware device 50 such as a producer's computer. Composer 300 allows users to drag and drop components, insert behaviors and graphics, insert text, and perform other tasks to make up an event.

While the producer is using the pull down menus, and drag and drop icons of the graphical interface of the authoring composer 300 to create an event, the authoring composer 300 is creating event setting files 310 and the chent event shell 110. The event setting files 310 convey information about the newly created event that is used by the system to create customized versions of the content tool 600, the tech director event shell 510, and the server event shell 210. The event setting files 310 relate to the features, behavior, images, and other aspects of the newly created event. Some information in the event setting files 310 is used by the system to generate a content tool 600 with fields for inserting the content required for the newly created event. Further, some information in the event setting files 310 is used by the system to generate server event shell 210, server code necessary for the functionality of that specific event. Finally, some information in the event setting files 310 is used by the system to create a tech director event shell 510 that contains the buttons and fields used to control the newly created event. The producer then inserts the content for each episode into the content tool 600. This content will then be exported and converted into the episode specific files, namely, tech director episode files 520, client episode files 120, and server episode files 220.

In another embodiment, the components of FIG. 10 can be generated in a different order or different combination.

FIG. 11 shows an example of an implementation of the authoring composer 300. The producer can lay out a storyboard for an event or episode using a program layout 330 section. Easy to use drag-and drop menus of different program options are presented to the producer to create the segments and perform other tasks.

An insert segment functionality 340 is used to determine the functionality that can be used for each of the segments. The producer can, for example, decide to turn on the chat functionality in the introductory segment and continue throughout or turn off in a later segment, send certain fun facts in segment #2, send five trivia questions in segment #3, provide a combination of polls and trivia in segment #4, etc. The producer can thus indicate a segment #1 and enable a variety of interactive component elements and capabilities during that segment. For example, the producer can turn on the chat functionality, turn on the ability of users to provide comments and ask questions to the producer, and then can enable a number of polls, trivia questions, and fun facts. As indicated above, each of these functionalities can be turned on or off through a user interface with drop down menus or drag and drop icons, such that the chat feature is dragged into the segment causing that feature to be activated, and polls, fun facts, and trivia questions can have separate icons used to enable them.

Change attributes 350 allows the producer to further tailor the look and feel of the event across multiple episodes. It provides the mechanism to determine the size of windows, change fonts and font colors, insert (background) images, and further design transitions. This includes, for example, how trivia and poll questions will be displayed, what the chat box looks like, and positioning of boxes being displayed. The images can include graphics that can be imported from saved files, and can be, for example, jpg images.

As the producer creates the functionality for all of the segments for the programming through the user interface of the authoring composer, the system generates the corresponding event setting files 310 and client event shell 110 as shown in FIG. 10 and discussed above.

At this time, a simulation can be run to see how the text boxes and background will appear during the event.

As shown in FIG. 12, the content tool 600 allows producers to create and insert specific content. The content tool 600 includes a software interface specific to the event generated by the authoring composer 300. Interactive components 620 sorts the available content files according to functionality. The menu of interactive components 620 will only contain that functionality pre-selected in the event settings file 310 that was generated through authoring composer 300 (FIG. 11).

Timing component 630 allows the producer to create fields for entering in the pre-determined content (e.g., poll or fun fact) and further sorts the content fields for the functionality chosen in interactive components 620 by display time. When a particular time is selected in timing component 630, the related fields are shown in content edit 640. Content edit 640 allows the user to enter the content and adjust the timing and duration.

In the example shown in FIG. 12, the producer can use the content tool 600, for example, to select the different polls, trivia, and fun facts and then create or edit the content for display. As shown in the example of FIG. 12, the first poll is scheduled to be displayed 13.5 minutes into segment #1. When the producer selects that poll in timing 630, the content tool 600 displays a field to enter a poll question, fields to enter poll responses, and a start and end time for the poll to be displayed. This poll will be displayed for one minute. Typically after that time, the poll results are accumulated at the server and then displayed back to all the users. The content tool 600 can be used to add any number of polls, trivia, and fun facts provided that the related functionality was established through the authoring composer. In this case, for example, the authoring composer was used to determine that there would bepolls provided during segment #1, and then the content tool was used to determine the number of polls, set their timing, and enter the specific content.

As discussed above and shown in FIG. 10, the content entered is used to build the technical director interface. The technical director interface varies by episode to reflect the functionality selected when the event was created and the content added by content tool 600. FIG. 13 shows an exemplary implementation of the technical director 500. The producer uses the technical director 500 to control the event while it is running. The technical director includes a software interface that can be used to generate new content during the event and/or to trigger the display of content inserted before the event using the content tool 600. Content 530 is the component that is shown in the example in FIG. 13. Content 530 has a trigger manager 560 and also the ability to generate content on the fly. In this example, eight polls and eight fun facts were created with the content tool and the buttons used to trigger them are represented by the boxes labeled Polll through Poll8 and Factl through Fact8, but there could be different numbers and there could be trivia questions and other content as well. In that case, buttons to trigger that content would be displayed in content 530 as well. These fun facts and pohs could have been created through the content tool 600 before the event or some or ah of them could be created with the technical director 500 during the event using content editor 565. Selecting one during an event causes the content to be sent to the event participant. The trigger manager 560 allows the producer, for example, to trigger polls and fun facts.

The poll, fun fact, trivia question, or other item of content can be selected to be viewed and edited if desired.

A content editor 565 creates the ability to generate content real-time. This ability is especially interesting in the case of hve TV shows and the synchronization of on-line content with on-air broadcast, because information can be added, such as "who should be the Most Valuable Player?" during a live sporting event, or a fun fact can be added if a lesser known player becomes significant to the broadcast event. Selecting this causes a text box to open, and when ready, the entered content is sent to users.

Open/Close windows 570 allows for closing and opening windows for all chents. This causes a message to be sent to chents to cause the base software to take the desired action.

Configuration 575 is a general function that provides a mechanism to change a broad range of setting files.

Event Segments 540 provides the ability to synchronize the individual segments and/or specific content components within the segments.

Chat host 550 is a mechanism to moderate discussions, select questions and present the selected questions and answers to the audience.

Accordingly, while graphical user interfaces, drag-and-drop tools, and drop down menus are generahy known for building drawings (e.g., selecting an icon with a circle to put a circle in a drawing field), software programs, and HTML content (such as tools for creating web pages), the authoring composer and content tool use tools such as these to enable a producer to create an interactive event. An interactive event under this system generahy requires customized chent side program and episode files, customized server side files, a customized content tool and customized content. The system described above creates all the code in these custom files and interfaces in the background while the producer uses the interface to simply create the desired event. The output not only contains the chent side and server side code, but also an interface for inserting each episode's content and controlling the interactive user experience during the event.

As indicated above, the client files specific to an episode are downloaded, typically in advance of the programming or provided during the programming, to create a file or functionality specific to that content. Meanwhile, other more general chent functionality, such as how different types of content are displayed generally or the ability to perform chat functions would be downloaded in advance and continue to reside on the chent device through all the episodes, with the ability to be updated between episodes as desired.

The system of the present invention provides a novel way of coordination and synchronization of interactivity across multiple client devices based on Internet Protocol (IP). The chent devices can be a range of different devices, the most-used current examples of which are a computer chent (further referred to as PC chent but can also run on different platforms such as Macintosh), set-top chent, net-top client (e.g., WebTV, a return path over dial-up), consoles, and wireless clients. The server system is responsible for interactive processing for activities, such as games, polls, and fun facts. The server system can communicate with the chent devices to initiate display of content through different types of messages (over IP). The client device will leverage a message library to interpret the messages.

The on-hne programming can be synchronized to the on-air broadcast by the server system's messaging system.

The server system preferably has an interactive program cluster for each concurrent interactive event. Each interactive program cluster can include an administrative server, multiple event servers, ad servers, multiple chat servers, multiple streaming servers, and a technical director. One or more of these components can be combined on a single computer or allocated among separate computers.

FIG. 14 shows the components of an embodiment of the present invention for the coordination of interactivity and 'on-line' programming across multiple chent devices. The server system 200 sends messages across Internet Protocol to control the interactivity on the chent devices: PC chent 102, set-top chent 112, net-top chent 122 and wireless client 132. This is an illustrative selection of client devices that can interact with server system 200. Examples of other chent devices are consoles and handheld devices. The messages are interpreted on the chent devices using a message library. In one embodiment, the server system 200 detects the type of client device. Based on the type of client device, a device-specific configuration file can be provided to the client device that supports the interpretation of messages sent by server system 200. In another embodiment, the chent device software interprets messages and tailors messages to the specific device.

The communication from these client devices back to the server system 200 is also preferably provided over IP. The system thus allows multiple client devices 50 (and different types of chent devices) to simultaneously interact with server system 200. In general, the communication is initiated by the server system 200 after a user has indicated a preference to participate in interactive programming. To provide true interactivity the server system 200 can initiate this communication and keep this connection open during the full duration of an interactive program.

The technical director 500 is a software interface on a computer and is the interface between a producer of the event, such as a television broadcaster, and the system. The technical director 500 allows viewers to impact the content of the event in real-time by enabling the producer to change the content based on input received from viewers. The technical director 500 can also, for example, receive, display, and ahow the producer to select questions posed to the producer or event's host or comments made by the user. The producer can respond to the questions or comments through the real-time creation of event content. The new content could be multimedia content distributed by a broadcaster or content created by the technical director 500. For example, the technical director 300 allows producers to generate real-time content (e.g., a real-time fiin fact or poh) and to initiate the display of such content.

FIG. 15 shows an example of interactivity (namely, a poh) across multiple chent devices by server system 200. A producer can initiate a poh using the Technical Director 300. The server system 200 receives poh 310 and sends poll 310 to the different chent devices: PC chent 102, set-top chent 112, net-top client 122 and wireless chent 132. The users of these chent devices send back their responses: answer PC chent 320, answer set-top client 330, answer net-top chent 340, and answer wireless chent 350. The server system 200 processes ah these results and sends back out the aggregated results 360 to each of the different chent devices.

Other types of interactivity can include fun facts (which do not require a response), and trivia questions, whereby the aggregated results can include answers, scores, and a leader board of high scores. The system can also provide chat functionality among users, and can allow textual messages to be sent to and from a producer. In response to chat or other textual messages, the producer can tailor or generate content provided during the broadcast. These functionalities can each be provided by a separate server, or multiple functionalities can be combined in one or more servers.

Content can be provided in advance of the start of the broadcast, during the broadcast for storage during the broadcast, or in real time from the server. In the first two cases, a message is sent from the server to cause the previously provided content to be displayed. The approach of providing the content before display is particularly useful for graphics and other items of content with higher bandwidth. In the third case, the content is provided in real time, preferably with a message type that causes that message to be processed. The content can thus be interpreted in real time or from storage. The types of content and the methods of transmission are also described in the incorporated pending applications.

In FIG. 16, and embodiment of the present invention is compared to a prior art system. In a prior art system, a head-end facility 400 sends an on-air broadcast signal with embedded ATVEF triggers 430 to the set-top client 112. The ATVEF trigger specifies a URL. The set-top client 112 sends a URL request 440 to the web server 422. The web server 422 serves up a web page 450 to the set-top chent 112.

The system of the present invention decouples the distribution and production of on-line enhanced TV programming from the production and distribution of the on- air broadcast. The server system 200 provides the mechanism of synched on-line program 210 over IP without embedding ATVEF triggers in the on-air broadcast signal. The on-air broadcast signal 412 is dehvered by the head-end facility 400 to the set-top chent 112. In the system of the present invention the ATVEF triggers do not have to be embedded in the on- air broadcast 412. This ability to avoid embedded triggers aUows producers other than the on-air broadcast producers to create on-hne programming. Furthermore, the system of the present invention can prevent MSOs (the operators of the head-end facility 400) from restricting the availability of interactive programming by removing ATVEF triggers from the signal 430 provided to the set-top chent 112.

FIG. 17 illustrates the convergence of the on-line program over IP 212 with the on-air broadcast 412 in on-air and on-line programming 222 over IP. In one embodiment, the head end facility 400 provides the on-air broadcast 412 (e.g., based on MPEG2) to the client device 50. The server system 200 provides the on-line program over IP 212 to the chent device 50 using the built-in synced-to-broadcast mechanism as described above. In another embodiment the server system 200 takes over the functionality of the head end facility 400 by combining on-air and on-line programming over IP 222 into one signal. One of the standards for this convergence over IP is MPEG4.

FIG. 18 shows the lightweight communication protocol 105 that allows for true interactivity for chent devices that do not have this true interactivity today. The base software 115 on the set-top client 112 knows how to interpret the messages from server system 200 and provides light and fast communication with the server system through which the user of set-top chent 112 can actually participate in interactive features such as real-time polls and trivia. The server system 200 initiates and maintains the connection with the base software 115 that is part of the set-top chent

112 using the communication protocol 105 (based on IP). The base software can be part of the operating system of a user device, or it can be incorporated in other software, such as in a browser, or it can be a provided later like an application program, such as by being downloaded over the Internet or by flashing it to the memory of a chent device.

As shown in FIG. 19, the server system 200 can synchronize the display of targeted advertising on the client devices to on-air advertising. The server system 200 sends message type ad 520 over IP to the PC client 102, set-top chent 112, net-top chent 122, and wireless chent 132. These chents will interpret this message type 520 to display Ad 510. The initiation of message type ad 520 can be synchronized to the broadcast such that the on-air commercial 530 will coincide with the display of Ad 510 on ah the different chent devices. Ad 510 can be personahzed to the user of the client device based on profiling of the server system 200.

An example of targeting Ad 510 is a locahzed list of car dealers. A car commercial 530 for a brand or model airs on television; in response to that ad, and at the same time, a list of local dealers for that brand or model of car is presented to the users on the different client devices based on the zip code of the users. Other factors in the profile could be used, such as age. The concept of the Ad 510 will be similar for all the user devices: PC chent 102, set-top chent 112, net-top chent 122 and wireless client 132. The hst of dealers close by, however, will depend on the profile of the user.

In FIG. 20, another embodiment includes a system for interacting with electronic programming guides (EPGs). The server system 200 coordinates the presentation of EPGs over IP 252 across multiple user devices. The EPGs on the different devices have the interactive programming associated with the broadcast programming. Each user on one of the user devices 50 can select an interactive program 262 to interact with other users on the different user devices 50. The EPG implementation of the present invention includes the association of interactive programming with the EPG and the coordination of the EPG across multiple chent devices 50 over IP by server system 200. The server system 200 uses the IPG component 270 to provide a mechanism for assigning users to the appropriate servers and services depending on their chent device 50, chosen interactive programming and potentially their profile. In another embodiment the user can select interactive programming by clicking on an interactive identifier, using a remote control or using a keyboard. In all cases the IPG component 270 on the server side will assign the users to the appropriate servers and services.

FIG. 21 shows a solution for interactive programming associated with time- shifted on-air broadcast. A PVR is a local client 555 that can cache programming received from the head-end 400 in a PVR storage 550 (such as a resident hard drive). A PVR allows for record and playback, but also allows one to pause programming, such that the device records while earlier content is being displayed.

According to the present invention, as this recording occurs, a PIR (personal interactivity recorder) engine 582 stores interactive content in interactive storage 562 (which can be physically the same device as PVR storage 552) originating from server system 200. The PIR engine 582 subscribes to ah video control messages to the PVR client 555, as well as the video frame markers. The PIR engine associates the interactive content (or links to other interactive content) to the video frame markers received by the PVR engine 572 and stores them on the interactive storage 562. The PIR engine then can perform all time modifying operations (fast forward, rewind, pause etc) of the PVR engine on the interactive content.

The PIR can have its own processing capability through a microprocessor or a dedicated processor or programmed logic, or it can use the processing capability of the PVR. The PIR can be programmed to know how to process the messages and content in advance, and further programming or modifications can be provided by downloading or flashing.

This system allows a user to respond to a poll and then get poh results (even if not affected by that user). The PIR records that a poll was sent at a particular time during the broadcast, or that a message was sent to display a previously downloaded or flashed poll. When the PVR plays back the broadcast program, the poh is displayed at the same time during the broadcast event as it would have been during the live broadcast, and can include any timeout mechanisms included in the content or governed by operating software. The user can then enter a response to the poll, and the server system can respond with poll results based on ah results previously provided including the user with the PVR, or ah results from when the program was live. By waiting for the user to enter a response the user is not influenced by seeing the poll results.

It may be desirable to disconnect the user from the server during playback, in which case, the PIR would record the display of the poll, any timer showing remaining time to enter a response, and the poh results at the time of the broadcast being recorded. In this way, the user has the experience he or she would have had, with the exception that his or her response may not be used in the poh results calculation.

In another example, in the case of a question-and-answer game, the system ahows the user to continue to play along with the broadcast event in what appears to the user to be real time question and answer. The user can play under the same conditions, including if the trivia question has variable points depending on the speed of the response, get his or her own personal results, and get a leader board of the scores of other players as they were during the broadcast, but the user may not be able to play with others and be a part of the leader board because of the delay involved, especially if the trivia has scoring that varies with the speed of the response. However, because the interactive program mformation is recorded, the user can compare his or her score to those of other users who participated in real time.

The PIR can also store for playback chat messages displayed during the broadcast, fun facts, and any other one-way or interactive content provided from the server and with the broadcast.

Those skilled in the art would appreciate that there could be other ways to subscribe to the video control messages and markers, and that video frame markers can take many forms, including, but not limited to absolute and relative time code, and frame sequence numbers. In addition or in the alternative, the PIR can also store triggers or other content or address to obtain content and associate them with times during the broadcast program for display during playback.

An example of a hard drive device is a system made by TiVo, Inc., which is also assignee of U.S. Patent No. 6,233,389, which is expressly incorporated by reference.

FIG. 22 illustrates the content display interface that determines the layout of interactive components based on the chent hardware device. PC chent 102 uses the full screen for interactive programming. Fun fact 720, game 730, and chat 710 are displayed across the full screen while TV 10 displays the on-air broadcast 410. In the case of the set-top client 110, the content display interface adjusts the placement of the interactive components based on the different type of client device. The on-air broadcast 410 takes up a large part of the screen real estate of set-top chent 112 while fun fact 720, game 730, and chat 710 share the remaining space.

Having described embodiments of the present invention, it should be apparent that modifications can be made without departing from the scope of the appended claims, and without necessarily having all the features and advantages recited herein. For example, additional interactive components can be provided and a system need not include all of the components that are included here. What is claimed is:

Claims

1. A method for enhanciiig a broadcast event comprising: a server system providing content related to the broadcast event to remote users having local devices that can interpret content and process messages received from the server system to cause information to be displayed; and

during the broadcast event, the server system providing to the local devices messages that cause the local devices to display the content; wherein each of the content and messages are provided at the same time to at , least two different types of local devices, including two or more of personal computers, set top boxes, net top boxes, wireless computers, consoles, and hand held computers.

2. The method of claim 1, wherein the content and messages are sent via Internet Protocol.

3. The method of claim 1, further comprising, in response to an advertisement being broadcast, the server system selecting one additional advertisement from a plurality of different advertisements tailored to different users, the one advertisement being related to, and for display at the same time as, the broadcast advertisement.

4. The method of claim 3, further comprising maintaining user profiles, wherein the server system selects the one additional advertisement based on the user profiles.

5. The method of claim 3, wherein the server system selects the one additional advertisement based on the user's location.

6. The method of claim 1, wherein the broadcast event is broadcast over television, radio, and/or the Internet.

7. The method of claim 1, further comprising transmitting the interactive content to the local device while the event is occurring, the content being stored for later display in response to a message after the content has been transmitted.

8. The method of claim 1, further comprising transmitting the interactive content before the broadcast event begins.

9. The method of claim 1, wherein the content is provided by downloading or by flashing.

10. The method of claim 1, wherein the messages do not include Internet addresses but are sent at the same time as Internet addresses that are used for accessing web pages with similar content, the local devices displaying content in response to the messages and not using the Internet addresses on receiving a message.

11. The method of claim 1, wherein at least two of the different types of local device are programmed to display interactive content in a manner different from each other in terms of size and or location of content on a display.

12. The method of claim 11, wherein the broadcast content is displayed and the broadcast content and the interactive content are provided on the same display in different windows.

13. The method of claim 11, wherein the broadcast content is displayed and the broadcast content and the interactive content are provided on separate displays.

14. The method of claim 1, wherein the interactive content includes content applicable to multiple broadcast events for display during the event, and content applicable to specific events for display during the respective specific event.

15. The method of claim 1, wherein at least some of the local devices include a recording device for receiving and storing the broadcast event, the content, and the messages and associating the timing of the content messages with the programming such that the playback of the broadcast event from the recording device includes the content and messages being provided at the same relative time as during the broadcast.

16. The method of claim 15, wherein the server system is responsive to a user entering data in response to content displayed during playback of a broadcast event for providing follow-on content related to the user entering data.

17. The method of claim 1, wherein the server system determines the type of local device and provides information in response to that determination.

18. The method of claim 1, wherein a first type of local device is programmed to use the content from the server system in one manner and a second type of local device is programmed to use the content from the server system in another manner.

19. An interactive system including a server system for providing to local users messages and content related to a broadcast event, the server system maintaining multiple local advertisement messages directed toward different users or groups of users, the server system, responsive to an advertisement being broadcast with the broadcast event, for selecting one of a plurality of the local advertisements and for causing that advertisement to be displayed additionally to the user at the same time as the advertisement in the broadcast event.

20. The system of claim 19, wherein the selected local advertisement is based on the user's location.

21. The system of claim 19, wherein the selected local advertisement is based on a profile of the user.

22. The system of claim 19, wherein the selected local advertisement is provided to a computer and the broadcast event is to a television.

23. A methods for a user to interact with a server comprising:

receiving client software for operating on a user-based hardware device that has the ability to control a display, the client software including base software for causing information to be displayed on the display and including software responsive to multiple message types relatmg to different types of interactivity, program content particular to a certain type of event, and episode content particular to an episode of the event, the receiving of program content and episode content occurring by downloading before the beginning of an episode;

the client receiving from the server messages, each with one of a plurality of message types indicating one of a plurahty of types of interactivity related to the episode of the event; and

the base software using the message type to cause to be displayed on the display content associated with that message type and in a form particular to the message type.

24. The method of claim 23, further comprising, for a next episode, downloading the episode content without again downloading the program content and without downloading the base software.

25. The method of claim 23, further comprising, for an episode of a different program, downloading the program content and the episode content without downloading the base software.

26. The method of claim 23, wherein the program content relates to a television program, and the episode content is for a particular episode of that television program.

27. The method of claim 23, wherein at least two of the types of interactivity include a question to which the user can respond.

28. The method of claim 27, wherein the types of interactivity include a trivia question and a poll.

29. The method of claim 23, wherein the message type indicates a trivia questions, and message includes text for the question, the base software being responsive to the message type for formatting the question as a trivia question and displaying the trivia question.

30. The method of claim 29, wherein the formatting further includes displaying a point count for the question that changes with time.

31. The method of claim 29, wherein the base software, responsive to a user entering an answer, causes the answer to be provided to the server to determine if it is correct.

32. The method of claim 29, wherein the wherein the base software, responsive to a user entering an answer, checks the answer against an answer stored as episode content.

33. The method of claim 29, wherein one of the chent and server corrects the answer, the chent receiving from the server and displaying a point total for multiple users interacting at the same time with the server.

34. The method of claim 23, wherein the chent is presentation layer independent.

35. The method of claim 23, wherein the chent operates with a particular type of presentation software.

36. The method of claim 23, further comprising sending messages to and receiving messages from other users via the server.

37. The method of claim 23, wherein the episode content includes an advertisement, the base software being responsive to a message type for causing the advertisement to be displayed.

38. The methods of claim 23, wherein the client is responsive to the receipt of a message type and audio and/or video information for synchronizing the audio and or video information to the interactivity indicated by the message type.

39. A method for a server to interact with a user comprising:

providing client software to a user-based hardware device having the ability to control a display, the chent software including:

base software for causing information to be displayed on the display,

program content particular to a certain type of event, and

episode content particular to a specific episode of the event;

the program content and episode content being downloaded before the beginning of an episode;

the server providing to the chent messages, each with one of a plurality of message types indicating one of a plurality of types of interactivity related to the episode of the event so that the user can cause to be displayed on the display content associated with that message type.

40. The method of claim 39, further comprising, for a next episode, providing the episode content without again providing the program content and the base software.

41. The method of claim 39, further comprising, for an episode of a different program, providing the program content and the episode content without again providing the base software.

42. The method of claim 39, wherein the program content relates to a television program, and the episode content is for a particulai- episode of that television program.

43. The method of claim 39, wherein one of the types of interactivity include a question to which the user can respond, the message including a message type and a question.

44. The method of claim 43, wherein the types of interactivity include a trivia question and a poh.

45. The method of claim 43, wherein the server is responsive to the base software providing either an answer for correction or an indication that the answer was correct for providing to users point totals indicting points in response to the answers provided.

46. The method of claim 39, wherein the client is provided to be presentation layer independent.

47. The method of claim 39, wherein the chent is provided and adapted to operate on a particular type of presentation software.

48. The method of claim 39, further comprising a technical director for managing the server and providing messages to and receiving messages from the server.

49. The method of claim 48, wherein the server provides messages from users to the technical director, wherein the technical director is staffed by an individual that can providing responses to users via the server.

50. The method of claim 48, wherein questions for responses are initiated from the technical director to the server, and thereafter to the users.

51. The method of claim 48, wherein the program is a television program, wherein the technical director provides content to the server for transmission to the users, the content being stored and timed to be received by the user in such a way that the content is synchronized to the television program to display content relevant to the program at that time.

52. The method of claim 51, wherein the content is stored in advance of being displayed and is timed to appear at selected times.

53. The method of claim 52, wherein the technical director receives content as a human input and provides that content on the fly as a supplement to the content stored in advance.

54. The method of claim 51 , wherein the technical director receives content as a human input and provides that content on the fly to appear at a desired time such that it is relevant to the program at that time.

55. The method of claim 39, wherein the episode content includes an advertisement, the server sending a message type to indicate to the chent that the chent should cause the advertisement to be displayed.

56. The methods of claim 39, wherein the server provides audio and/or video information synchronized to textual messages to cause the textual messages to be displayed with the audio and/or video content.

57. The method of claim 39, where messages can be provided to thousands of users at the same time.

58. A system for interacting with users comprising:

a server having interface for connection to the Internet infrastructure to provide messages to users that have chent software for receiving messages from the server;

a technical director for providing to the server different message types that indicate one of a number of types of interactivity with users, and for providing message content to the server, the message content relating to a specific episode of an interactive event, wherein the technical director can provide stored messages and message types for display at desired times during the event, and individually input information on the fly;

the server, responsive to messages and message types from the technical director, for providing such message types and message content to users.

59. The system of claim 58, wherein, in some instances, the technical director provides only a message type, and the server provides related content.

60. The system of claim 58, wherein, in some instances, the technical director provides only a message type, and the server provides the message type to the user, the message type indicting to the user that content previously stored by the user is to be displayed.

61. The system of claim 58, wherein, in some instances, the technical director provides a message type and the content to be provided.

62. The system of claim 58, wherein the server provides questions with a message type and maintains scores of users who answer questions, the server periodically providing a hst of scores to the users.

63. The system of claim 58, wherein the technical director, responsive to individual input, responds to messages input by users and provided to the technical director via the server.

64. The method of claim 23, wherein the base software is downloaded through Internet infrastructure.

65. The method of claim 23, wherein the base software resides on the hardware device without being downloaded through Internet infrastructure.

66. The method of claim 39, wherein the base software is downloaded through Internet infrastructure.

67. The method of claim 39, wherein the base software resides on the hardware device without being downloaded through Internet infrastructure.

68. A system for creating an interactive event in which a chent file that is responsive to messages from a server is provided to remote clients, the system comprising a content creator that includes tools, responsive to inputs from a producer, for generating chent files and a server-based user interface for controlling the display of content on the remote chents during the event.

69. The system of claim 68, wherein the user interface is responsive to the producer for causing messages to be sent from the server to the chent to cause previously transmitted content to be displayed at the chent side.

70. The system of claim 68, wherein the user interface is responsive to the producer for receiving content and causing that content to be displayed at the chent side.

71. The system of claim 68, wherein the content creator is used to create polls such that during creation of a poll, a representation indicating that the poh is to be displayed is created for display on the user interface.

72. The system of claim 68, wherein the content creator is used to create trivia questions such that during creation of a trivia question, a representation indicating that the trivia question is to be displayed is created for display on the user interface.

73. The system of claim 68, wherein the content creator has fields for designating a time during the event for when specified content will be displayed.

74. The system of claim 73, wherein the event is a broadcast program and the timing is based on the beginning of a segment of the broadcast program.

75. The system of claim 73, wherein the technical director allows the producer to override the designated time or displaying content.

76. The system of claim 68, wherein the user interface stores content created in real time during the event and causes the server to transmit that content to the chents.

77. The system of claim 68, wherein the server sends messages to the chent using an Internet protocol.

78. The system of claim 68, wherein the user interface has icons representing all the items of content to be displayed during at least a segment of the event.

79. The system of claim 68, wherein the content creator includes a first program for allowing a producer to identify types of items of interactive functionality.

80. The system of claim 79, wherein the first program also ahows the producer to create a look and feel for multiple events.

81. The system of claim 79, wherein the content creator further includes a second program that receives from the first program the types of items of interactive functionality, the second program being used to enter quantities and the content for each item.

82. The system of claim 81, wherein the content entered for each piece of content is used to generate files for transfer to a chent.

83. The system of claim 79, wherein the content creator creates the user interface using the quantity of items and content of the items of interactive functionality.

84. The system of claim 68, wherein content is provided to the clients before the event.

85. The system of claim 68, wherein content is provided to the chents during the event but prior to display during the event.

86. The system of claim 68, wherein content is provided to the chents during the event for immediate display during the event.

87. The system of claim 68, wherein some content is provided to the clients before the event, some content is provided to the chents during the event but prior to display during the event, and some content is provided to the chents during the event for immediate display during the event.

88. The system of claim 68, wherein the chent file is transferred to viewers in advance of an episode of an event and includes content for display for multiple events and content specific to a single episode event, wherein the content creator can create both the multi-episode content and the individual episode content.

89. The system of claim 68, wherein the content creator is responsive to a producer for enabling a chat functionality during some or ah of an event.

90. An authoring system for creating interactive content to be sent to remote viewers during an event, the system responsive to user inputs for selecting from among a plurality of different types of interactive functionality and for entering content for each of a number of items of interactive functionality, the system responsive to the types of interactive functionality and content for creating a user interface showing representations of each item of content to be displayed during the event.

91. The system of claim 90, wherein the representations are icons.

92. The system of claim 91, wherein at least some of the icons represents questions and responses for display to viewers of an event, the questions being related to the event.

93. The system of claim 90, wherein the user interface indicates interactive functionality available during an event.

94. The system of claim 93, wherein the interactive functionality includes the ability of one viewer to communicate with another viewer or with a producer of the event.

95. A method for enhancing content during a broadcast event for remote viewers having a local storage device for storing the broadcast event and playing back the broadcast event, the method comprising:

the local storage device storing the broadcast event;

a personal interactivity recorder (PIR) storing interactive content provided from a server system and related to the broadcast event and temporally associating the interactive content with the broadcast event; when the broadcast event is played back from storage, the PIR providing to the user the interactive content during times within the stored broadcast event when such content would have been displayed when the event was broadcast.

96. The method of claim 95, wherein the local storage device includes functionality for fast forward, rewind, and pause functions.

97. The method of claim 95, wherein the temporal associating includes using one or more of absolute time codes, relative time codes, and frame sequence numbers.

98. The method of claim 95, wherein the interactive content includes trivia questions, the user has an input device for entering an answer, and the PIR stores the correct answer and provides to the user an indication of a correct or incorrect answer after the user enters an answer to a question.

99. The method of claim 95, wherein the interactive content includes poll questions, the PIR storing poh results, the user has an input device for entering a response, and the PIR provides poh results after the user enters a response to the poh question.

100. The method of claim 95, wherein the interactive content and video broadcast event are stored on the same medium.

101. The method of claim 95, wherein the PIR uses the processing and storing functionality of the local storage device.

102. The method of claim 101, wherein the local storage device includes a hard drive.

103. The method of claim 95, wherein the local storage device includes a hard drive.

104. The method of claim 95, wherein the PIR stores and plays back messages sent by other viewers using a chat functionality during the broadcast event.

105. The method of claim 95, where in the broadcast event and interactive content are sent over different channels of transmission.

106. The method of claim 95, wherein the PIR includes processing and storage.

107. A system for use with a local storage device for storing the broadcast event and playing back the broadcast event, the system including a personal interactivity recorder (PIR) for storing interactive content related to the broadcast event and transmitted at the time of the broadcast event and temporally associating the interactive content with the broadcast event, such that when the broadcast event is played back from storage, the PIR provides to the user the interactive content during times within the stored broadcast event when such content would have been displayed when the event was broadcast.

108. The system of claim 107, wherein the local storage device includes functionality for fast forward, rewind, and pause functions.

109. The system of claim 107, wherein the PIR is programmed to use one or more of absolute time codes, relative time codes, and frame sequence numbers to temporally associate the content with the broadcast event.

110. The system of claim 107, wherein the interactive content includes trivia questions, and the PIR stores the questions and answer provided during the broadcast.

111. The system of claim 110, wherein the PIR provides to the user an indication of a correct or incorrect answer after the user enters an answer to the trivia question.

112. The system of claim 107, wherein the interactive content includes poh questions, and the PIR stores poh questions and results during the broadcast event for display when the broadcast event is played back.

113. The system of claim 107, wherein the PIR uses the same storage medium as the broadcast event.