WO2008074022A2 - Non-intrusive interactive video and audio sales and advertising - Google Patents

Abstract

Methods and apparatus are disclosed for interactive shopping from and advertising within multimedia programs, without any modification, adaptation, mutation, conversion, alteration, or special preparation of the programs or the program mediums and, therefore, applicable to legacy programs such as existing DVDs and on-demand old movies. These methods and apparatus enable users of video and audio programs to select any item of interest shown in a program and buy or obtain the advertising information specific to that item. While convenient for the viewers, these modes of sales and advertising are non- intrusive and do not interrupt the programs.

Description

NON-INTRUSIVE INTERACTIVE VIDEO AND AUDIO SALES AND ADVERTISING

CROSS-REFERENCE TO RELATED APPLICATION(S)

[01] This application claims the benefit of U.S. Provisional Patent Application No.

60/874,862, filed on 13 December 2006 and U.S. Provisional Patent Application No. 60/946,005, filed on 25 June 2007.

TECHNICAL FIELD

[02] Disclosed embodiments relate, in general, to sales and advertising and, in particular, to non-intrusive and interactive video and audio advertising and to apparatus and methods for shopping from a multimedia program played on any platform.

BACKGROUND

[03] There is a growing potential for online advertising that is replacing the traditional advertising in all kinds of public media such as radio and television. One of the fundamental changes introduced by this new technology is the ability for an advertiser to have an immediate quantitative measure of success in reaching its target clients.

[04] To date advertising is not an integral part of the video and audio presentations, whether live, prerecorded, streamed, DVD, or in other forms. The video and audio presentations that have integrated advertising are either infomercials or merely show products that bear trademarks and logos. Traditionally, the advertisements are interleaved with selected parts of a video or audio presentation by stopping the presentation and playing the advertisement clip. As a result, multimedia programs and programs such as TV, DVD, on- demand movies, and radio programs are frequently interrupted to advertise various products and services. This method of advertising is not tolerated by most viewers or even practical in many forms of video or audio entertainment, for example, in rental movies. In many instances of traditional advertising the viewers or listeners, also referred to as "users," completely ignore the advertising clips and engage in other activities for the duration of the advertisement, which wastes much of the advertisers' investment. [05] In addition to being intrusive and time consuming, the traditional video or audio advertising is not interactive and cannot take advantage of the "impulse buying" tendencies of many users. For example, if a viewer of a movie is interested in an item that is shown in a scene of the movie, or even in a traditional advertising clip, played during the movie, the opportunity for introducing or selling that item to that viewer is mostly lost. Currently, such viewer cannot interactively order, buy, or even get additional information about her item of interest. Moreover, in traditional advertising, often the user is forced to watch or listen to the advertisements of products and services that she is not interested in.

[06] Of the billions of dollars spent on shows and movies, and millions of consumer hours spent watching them, not much advertising revenue is currently being generated. Furthermore, the traditional means of advertising can even be circumvented and/or eliminated by users through many available tools such as Tivo ® and free TV on websites that stream TV content.

[07] The developing interactive technologies modify and alter the programs to create an interactive relation between the users and the programs - an approach that is very labor intensive, costly, and time consuming.

BRIEF DESCRIPTION OF THE DRAWINGS

[08] Figure 1 depicts an exemplary computing environment for non-intrusive interactive advertising, according to an embodiment of the present invention.

[09] Figure 2a and 2b schematically depict user and administrator clients in the computing environment of Figure 1 that supports the disclosed non-intrusive interactive advertising method, according to another embodiment of the present invention.

[10] Figure 3 is a flow diagram of processes involved in an administrative preparation of data that is utilized in a non-intrusive interactive advertising which employs a video, audio, or multimedia program, according to yet another embodiment of the present invention.

[11] Figure 4 is a flow diagram of processes involved in a user purchase utilizing a non-intrusive interactive advertising method in accordance with an alternative embodiment of the present invention. [12] Figure 5A schematically illustrates an arrangement of frames of a multimedia program and a corresponding frame number associated with each frame, wherein the frame numbering starts from a specified frame of the program in accordance with an alternative embodiment of the present invention.

[13] Figure 5B is a schematic representation of a timecode format, which illustrates a common method of organizing and addressing a particular frame in a multimedia program. The timecode is a multi-segment number having an Hour- Minute-Second-Frame format.

DETAILED DESCRIPTION

[14] The embodiments described in this application disclose methods and apparatus for user-driven, non-intrusive, interactive advertising embedded in or provided with any media such as television, radio, multimedia, etc., and in any mode of presentation such as live, prerecorded, streamed, on-demand, DVD, etc. In these embodiments, for example while watching a streamed movie, a DVD, or playing with a computer game, the viewer can request to receive more information about an item that is portrayed in the movie or buy the item while she watches the movie. As described below, the users of these embodiments will have multiple choices of services and options for interaction with the programs. Similarly the advertisers of products and services will have multiple advertising opportunities. Additionally, by utilizing the proposed methods and apparatus, the item providers can post their own programs online and sell any item presented in their programs. Users also, given permission, can advertise and sell their own items in any multi-media program, much like users of on-line trading services, such as ebay.

[15] Utilizing disclosed methods, traditional advertisements can also be treated as part of the main program, from which the users can interactively shop. The information presented to a user as a result of her inquiry, may be single-layered or multi-layered. For example, a user may be given the name, address, and the telephone number of a seller of the requested item, its website address, or both. The information retrieval methods used may also be single-layered or multi-layered. For example a scene may be associated with a pointer in a data-base that points to a list of items each of which points to a manufacturer that, itself, points to the manufacturer's information, etc. [16] One of the advantages of the disclosed methods and apparatus is that in such modes of advertising, the products are featured in real-use environments while their portrayal does not seem biased by the suppliers of the products. Therefore, the implied advertisements will better convince the users to buy the products than searching, for example, through the suppliers' websites does. Some of the other advantages are the ability to attract spontaneous decision-makers and impulse buyers, to offer user convenience, and to provide measurable results.

[17] In some embodiments, upon receiving a user-generated item-information- request signal, the requested information is retrieved from a database and is either presented to the user or stored for later use. In some other embodiments the particulars of the item- information-request itself, for example the time of the request or the frame number of the scene of interest, or a combination thereof, is captured for immediate or future information retrieval.

[18] It should be noted that in the following detailed description, wherever the visual rather than the audio aspects of a presentation/program are intended the word "video" is used and, similarly, wherever the audio rather than the visual aspects of a presentation are intended the word "audio" is used. However, the word "program" is used to collectively denote presentations and programs from any broadcaster or content provider or the like, such as television, radio, multimedia, on-demand facilities, DVDs, streamed programs, computer games, etc. Similarly, the word "user" is used to collectively denote those persons or apparatus or software, or the like, that utilize these "programs" to take advantage of their embedded advertisement or information.

[19] Throughout this description, for the ease of understanding and clarity, the examples are mainly limited to visual presentations and programs; however, the concepts are equally applicable to all kinds of programs such as radio programs or movies played in movie theatres. Occasionally, specific reference may be made to audio presentations and programs if it is deemed helpful to a better understanding of the subject matter.

[20] The following explanation provides specific details for a thorough understanding of and enabling description for these embodiments. One skilled in the art will understand that the invention may be practiced without such details. In other instances, well- known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. Embodiments of the present invention can be extended to other systems as well.

[21] Utilizing different aspects of this invention, users of all kinds of programs, are able to select, for example, by "clicking" on an item with a pointing device such as a computer mouse, on different items, products, services, etc. that are featured or embedded in those programs expressly or impliedly (hereinafter called "items") and conveniently obtain the advertising information specific to those items or even order and buy the items. In some embodiments the users are free to view the programs at their own leisure and are able, for example, to "flag" or save any item of interest during a program by simply clicking on the desired item or scene. In other embodiments the advertisement information may reside on the same storage medium as the program itself, such as on the same rental DVD.

[22] It should be noted that the term "click" or "clicking" represents any signal generated by a user for selecting an item, items, a scene, or in general a "segment" in a program. Hereinafter, the word "segment" signifies a short portion of a program. Such short portion can be a single frame of a movie, a 5-second scene of a TV show, or three minutes of a radio presentation. A user signal may be of any form or nature, such as being wired or wireless. For example, in one embodiment a user may clap or whistle while in another embodiment the user may use a wired or an infrared mouse and in yet another embodiment the user may have the option to use multiple signal forms. Similarly, in an embodiment the user employs verbal commands and the signal receiver is capable of voice recognition and command processing. In such embodiment the signal receiver may, for example, recognize the words "buy shoe," as a command to buy a shoe in a selected movie scene. In such an embodiment, the sound of "buy" may be interpreted as a "click" for scene selection and the sound of "shoe" may be interpreted as narrowing the request to the information regarding a shoe that is featured on the selected scene. In other embodiments, a signal may comprise touching a touch-sensitive screen or even winking at a screen.

[23] In general, any action that can be recognized and picked up by an apparatus can be utilized as a "click." In one embodiment a click may be performed by, for example, a light/laser pointer targeting a specific item on the screen. In yet other embodiments the result of an item selection can be visually acknowledged on the screen by, for example, a green and/or a red symbol, or by a sound. As mentioned above, in this description the well-known structures and functions such as remote pointers pointing at and communicating with the screen or different types of wired and wireless mice have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. Such technologies are well-known among those skilled in the relevant art.

[24] Figure 1 depicts an exemplary computing environment 100 for non-intrusive interactive advertising, according to an embodiment of the present invention. Items numbered 101 in Figure 1 exemplify client or user devices. Many existing electronic devices, mobile or otherwise, such as a cell phone, a laptop computer, a desktop computer, a remote controller, a Blackberry, a palm device, etc., can act as client devices. Network 108 exemplifies any network that can connect a user device to a service provider 102. However, in some embodiments, the service provider capabilities may have been downloaded to a user device, in which case a user device can include all elements of Figure 1. Network 108, for example, may be the World Wide Web or a LAN. Service provider 102, for example, includes the server device 104 and the data storage 106; however, different server providers may provide the server device 104 and the data storage 106.

[25] In one embodiment, upon clicking on a scene, a list of all available items in that scene appears on the screen so that the viewer can narrow down her/his choices. This list may overlap the program or occupy a separate part of the screen. In another embodiment the list of the available items may stay on the screen for a specific span of time or indefinitely. In yet another embodiment, what is shown to the viewer is a result of her profiling and her shopping habits. In yet another embodiment, the user specifies what kind(s) of items she desires to be presented to her.

[26] These disclosed modes of advertising are user-driven, non-intrusive, and do not interrupt the programs. The items of interest to the users can be products, services, or anything else that can be presented in a program, whether physically, symbolically, verbally, etc. For example clicking on a cell-phone may provide information about the cell-phone itself, different available service programs, history of the cell phone, or take the user to a cellphone manufacturer's website to purchase a cell phone. Or clicking on an actress's leg can provide information about a gym, a tanning salon, an available exercise program, a workout DVD, or the life of the actress. Similarly, clicking on an actor's head may provide information about his hair implant or his resume, or clicking on a scene of a music video provides information about the music being played and/or an opportunity to buy the singer's CD(s).

[27] One of the advantages of the disclosed embodiments is that there is no need for an actual interaction with a program and, therefore, no need for modification, revision, adaptation, mutation, conversion, alteration, adjustment, or special preparation of the program (collectively also called "modification"). In contrast, a proposed "interaction" is merely an "association" of a user signal with a segment of a program and using the association to retrieve information about the items within the associated segment. (Or, the interaction is an association of the user signal with a segment of an advertisement signal/clip running in parallel with the program and, therefore, indirectly associated with a segment of the program.) The information retrieval may, for example, include facilitating the purchase of the items or performing any other action in relation to those items or to that segment of the program. Such an advantage makes it inexpensive and easy to utilize all existing programs, such as legacy DVDs, and makes it unnecessary to put any additional effort to prepare a program medium for advertising purposes.

[28] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number, respectively. Additionally, the words "herein," "above," "below" and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. When the claims use the word "or" in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.

[29] In one embodiment, prerecorded programs on any medium, whether recorded by the user or by a provider, can be utilized for interactive advertising, for example a season of "Sex and the City" on DVD or as data stream from HBO network. Users interested in the fashion sense of Carrie Bradshaw, played by actress Sarah Jessica Parker, will be able to click on her wardrobe or on the scene and, for example, get a list of one or more suppliers offering that wardrobe along with other related information, or an opportunity to buy the wardrobe. The same possibility will be available for those interested in her shoes, lipstick, eye makeup, handbags, hair products, and even her exercise program and tanning salon.

[30] In different embodiments, a user may elect to get the requested information (1) instantaneously, as the show continues, (2) while pausing the show for viewing the information, (3) storing the information for later viewing and retrieval, or (4) a combination thereof. If a user is interested in multitasking, it will be possible, for example, to open a separate information window while the content continues to play. Please note that same can be done with a radio program in which a dress of Sarah Jessica Parker is being discussed.

[31] In some embodiments, the user has the choice to specify what subset of all available advertisements she wants to be presented with. In general, the user can limit the advertisements and the for- sale items that are shown to her by any criteria, such as the geographic location of the manufacturers, the price range, item category, etc. Similarly, these embodiments allow the providers to send different information for the same item to different demographic areas. For example for the same item in the same movie, a user in California may be presented with different information than a user in Paris.

[32] The disclosed methods can be also applied to all kinds of TV programs, whether received wirelessly, through the cable, or available as online stream, such as shows provided by FOX Networks, AT&T Broadband TV, and other free TV on the Net sites such as Channel Chooser, ChooseandWatch, FreeTube, NetChannels, Streamick, and several others. The disclosed methods can similarly be applied to the recorded audio, online audio, downloadable audio, radio programs, etc.

[33] In alternative embodiments that employ a combination of, for example, television and internet, the users are able to specify objects of interest during any TV program and obtain advertisement information by clicking, for example, on a remote control mouse that is in communication with the television screen or give voice commands. In these embodiments the internet is used for processing, and retrieval of the requested data, feed back to the seller, data storage for the user, product ordering, payment, mailing arrangement, etc. In yet another embodiment, it will be possible for the user to open a separate window and get online access while the program/movie continues to play or even postpone information retrieval for a future time, such as after the program has ended. Using, for example, a cell phone, a blackberry, or any other mobile device, same services can be available in movie theatres or while listening to radio programs, etc. In another embodiment, TV set-top boxes may be programmed to take advantage of the disclosed methods.

[34] Figures 2a and 2b schematically depict consumer/user and administrator clients in the computing environment of Figure 1 that supports the disclosed non-intrusive interactive advertising method, according to another embodiment of the present invention. As seen from Figures 2a and 2b, their architectures, 110 and 120, are similar in this example and, therefore, will be described together. The multimedia streams and other data and information may arrive to a consumer or an administrator from a network 106, such the World Wide Web, n optical device 202, such as a DVD player, or from both sources. The multimedia streams, as known by those skilled in the art, pass through their corresponding interfaces 208 and 210, respectively, to be usable by the consumer or administrator applications 204 and 212. The Database Interface 206 facilitates two-way communication between the consumer application 204 and the network 106. The connection between the network interfaces 208 and the multimedia interface 210 facilitates replacing the multimedia stream from the optical device 202 with the multimedia stream from the network 106, in this example.

[35] While specific circuitry may be employed to implement the disclosed embodiments, aspects of the invention can be implemented in a suitable computing environment. Although not required, aspects of the invention may be implemented as computer-executable instructions, such as routines executed by a general-purpose computer, e.g., a server computer, wireless device or personal computer. Those skilled in the relevant art will appreciate that aspects of the invention can be practiced with other communications, data processing, or computer system configurations, including: Internet appliances, hand-held devices (including personal digital assistants (PDAs)), wearable computers, all manner of cellular or mobile phones, multi-processor systems, microprocessor-based or programmable consumer electronics, set-top boxes, network PCs, mini-computers, mainframe computers, and the like. Indeed, the terms "computer," "host," and "host computer" are generally used interchangeably herein, and refer to any of the above devices and systems, as well as any data processor.

[36] As mentioned above, in this paper the well-known structures and functions such as remote pointers pointing at and communicating with the screen or different types of wired and wireless mice have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. Such technologies are well-known among those skilled in the relevant art.

[37] In some embodiments there are at least some functions of a remote control on at least one part of the screen, which has typical video viewing functions such as volume control, pause, rewind, and fast forward content, and adjusts the size of the display screen. This remote control toolbar can also be moved, minimized, and maximized at the viewer's leisure.

[38] In an alternative embodiment the available/selectable items in a scene may be visually marked, such as with a little blue dot on an available wardrobe. In another embodiment a special marking, for example at a corner of the screen, indicates the presence of selectable/for- s ale items within the instant scene. In yet another embodiment a change in the shape of the mouse cursor on the screen, for example from an arrow to a hand, indicates a selectable item. In yet another embodiment, for all or some scenes, a list of items and/or small pictures/visual-representations and information of the available items may be simultaneously presented on the screen. Such visual data may be intermittently or continuously present. Viewing this visual data can be either optional or mandatory.

[39] In some embodiments, a screen may contain, at all times or upon the user's request, a list of the "flagged favorite" items (the products on which the user has clicked during the program). For example, once an item is identified, a miniature version of the actual selected item can be stored with an embedded link to the manufacturer's website, along side the brand name of the item, its price, local stores that carry the item, and/or the related links to manufacturers who sell similar items, etc. When the user clicks on a product's miniature icon, the user will be directed to the corresponding site. The flagged favorites list can also be moved, minimized, and maximized at the user's leisure. The flagged favorites list can also be printed with or without some of its detail information. In one embodiment the user is able to sort the flagged favorites list, according to different criteria such as the chronological order of clicks, price, product categories such as accessories, cosmetics, electronics, personal hygiene, etc. A shopping cart is an embodiment of the flagged favorites list.

[40] In yet another embodiment, for the convenience of the user, she is able to choose whether she wants to be directed to an informational or a shopping site while simultaneously viewing the program, to stop the program while visiting the site, shop after the program is over, or a combination thereof.

[41] Downloadable or free software may be offered to those users who intend to utilize the advantages of the proposed methods. Aspects of the invention can be embodied in a special purpose computer or a data processor that is specifically programmed, configured, or constructed to perform one or more of the processes explained in detail herein. Aspects of the invention can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices, which are linked through a communications network, such as a Local Area Network (LAN), Wide Area Network (WAN), or the Internet. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[42] Aspects of the invention may also be stored or distributed on computer- readable media, including magnetically or optically readable computer discs, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, or other data storage media. Indeed, computer implemented instructions, data structures, screen displays, and other data under aspects of the invention may be distributed over the Internet or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme). Those skilled in the relevant art will recognize that portions of the invention reside on a server computer, while corresponding portions reside on a client computer such as a mobile or portable device, and thus, while certain hardware platforms are described herein, aspects of the invention are equally applicable to nodes on a network.

Time-Based Information Retrieval

[43] In some embodiments the item selection is time-based and, as mentioned above, does not require any preparation or modification of the program. For example, in one embodiment a clock or timer keeps track of the clickings of the user and for each click it determines the time-lapse between the beginning of the program and the click (e.g., in seconds or milliseconds, excluding pauses or in some embodiments including pauses such as regular advertisements). This time-lapse, which uniquely corresponds to a specific segment within the program, is then mapped into a list of available items in that specific slice/ segment of the program. Therefore each click is ultimately mapped into a list of items. Such mapping process may employ, for example, a simple table that relates the time lapse measurements of the clicks with the scenes of the program and, as a result, with the list of available items at each scene Please note that the described table is merely an example of many data-mining schemes which may be devised to relate a clicking with one or more entries of a database.

[44] Each entry of such a table includes the specifications of a "time-window" and one or more corresponding items that are featured in the program during that time-window. If a user-clicking happens during any time- window, the user is presented with the information about, or an opportunity to buy, the items corresponding to that particular time-window. Several clicks may be mapped into the same entry of a table (the same item or items) if all clickings happen during the time-span of the same time-window. In contrast, one click may invoke more than one entry of a table if two or more time- windows have time- overlaps.

[45] For a slow paced movie, the list may contain the items shown within each consecutive 5-second time windows or for a fast pace movie, the list may contain the items shown within 0.5-second time windows. In different tables, the time boundaries of the time- windows can be recorded/presented in various ways, as arrangements can be made to make such time boundaries known or computable. For example if a table has similar consecutive time-windows, it is easy to calculate the time boundaries of its n^ώ time-window. Furthermore, since several minutes may pass in a program without featuring any selectable item, there is no need for the table to account for uninterrupted consecutive time- windows. Similarly if a selectable item is featured, uninterruptedly, during a 10 minute scene, one entry in the list corresponding to a 10-minute time window will be sufficient and any click during that period will be mapped into the same entry. Therefore, in some embodiments the list may comprise windows of different durations with or without time-gaps or time-overlaps between some windows. A time-window in a table may be specified by its start and stop time relative to the start of the program or relative to any other starting point or by its duration and its start or stop time, etc.

[46] In some situations, such as during TV programs, where the starting time of the programs are or can be known, the clock, or the time keeper, may keep track of the absolute/actual times (e.g., time of the day or Greenwich time) rather than relative times and the logged times can be similarly mapped into a list of items. For example, a click at 8:38:42pm during a TV program, along with the program name and the name of its affiliated network, is sufficient to obtain a list of selectable screen items corresponding to that click. The table of available items for the mentioned TV program may be kept, for example, in a data base of a service provider that can be queried via the internet. Using a mobile device, a cell phone, a blackberry, a laptop, etc., such item inquiry may be performed in real time and while the program is playing.

[47] Table 1 is an exemplary table arranged by time- windows and the items shown in the program within those time periods. In this table, 5' 10" signifies 5 minutes and 10 seconds past the start of the program. Please note that, for example, the first and the second time- windows are consecutive non-interrupted similar-duration time- windows, wherein the second and the third time-windows are consecutive unequal-duration time-windows with a time gap, and the third and the forth time-windows are consecutive unequal-duration time- windows with some time-overlap.

Table 1

[48] As mentioned above, the same method is applicable to audio and other multimedia programs. For example, while listening to a radio program in the car, a user can time stamp segments of interest in the program (e.g., by actual times) and obtain the information corresponding to the items presented in those segments via the radio station's website or any other specified website. For convenience, a time stamping apparatus, for example, can be integrated with the car radio.

[49] In some embodiments an electronic device, such as a cell-phone or a blackberry or any other mobile apparatus, may be used or even preprogrammed to order an item upon a user click. For example, before entering the movie theatre a user may enter the name of the movie "A Beautiful Mind" and while watching the movie just press a button each time the user wants to buy an item. The device can also be preprogrammed to either buy one specific item within the scene, if the buy signal is sent, or buy some or all of the items within the chosen scene, or to just return the information about the available items in the chosen scene and give an opportunity to the user to choose among them before a transaction is made, or any combination thereof. An electronic device like a cell-phone may also be used to transmit any retrieved information, instantly or with some time latency, to other destinations, such as to a friend or a remote data storage device.

[50] The measurement of a time-lapse for pinpointing a segment of a program may be with respect to a reference time different from the start-time of the program. Such a reference time, in some embodiments, is marked or indicated by a reference signal provided by the program itself or the presenter of the program or the user, etc. For example, an audible, inaudible, RF, electromagnetic, light, or other signal forms may be sent by a TV or a radio program or in a movie theatre to synchronize the beginning of the time-lapse measurements. Dedicated or commonly used apparatus such as digital cell-phones or other mobile devices may be utilized to synchronize themselves with the synchronization signal and subsequently measure the time-lapses between the synchronized start-time and a user signals. The synchronization may be manually performed by the user, such as pressing a specific button on the cell-phone or a mobile electronic device, or automatically done by the device. Such recorded or logged clicking time-lapses, for example, while the user is in a movie theatre or while driving her car and listening to a radio station, can be immediately or later utilized to retrieve information from or to order and buy items on the internet.

[51] In another embodiment that uses time-based item selection, the program may be interrupted for different reasons, such as for traditional advertising or by the user for rewinding, pausing, etc.; however, even in such situations the time measuring device keeps track of all the interruptions and accounts for the lost time. Once a click is mapped, the information retrieval is similar to the above mentioned embodiments. In yet another embodiment the timer logs the actual lapsed time but the time-windows of the item table compensate for the timer reading which includes different interruptions. In these tables, the interruptions appear as gaps between the time-windows of the list. In an alternate embodiment, the regular advertisement clips are also treated as part of the main program and the users can interactively order and buy from such advertisement clips.

Item Identification Methods

[52] In one embodiment the position of the curser/pointer on the screen is recorded at the time of clicking (e.g., its X-Y coordinate or the quadrant in which the curser/pointer lies) and is used to identify a particular item in that scene. In the embodiments wherein a click is a touch on a touch- sensitive screen, the X-Y coordinate of the click is also easily determined. In these embodiments the recorded time maps, for example, to all the selectable items within the screen at the time of clicking and the location of the curser/pointer maps to the particular item within the screen upon which or in the vicinity of which the curser lies at the time of clicking.

[53] In some embodiments the item tables include the geometric boundaries of all available items in corresponding frames or time segments. This geometric boundary information may be stored with different levels of accuracy. For example at a lower level of accuracy, two X's and two Y's may represent the outer limits of an imaginary rectangle on the scene (item-rectangle) within which a particular item lies. These geometric measurements preferably use the same coordinate system as the cursor or the system used for determination of a touch on a touch screen. However, the information related to any other coordinate system may be transformed such that to find out if a click is within an imaginary item-rectangle or not.

[54] Another simple boundary information scheme is to store the coordinate of the center and the radius of an imaginary circle that substantially encircles the item. Those skilled in the art will appreciate that other schemes can be devised for storing the information about the location of an item in a scene and comparing it with the location of a cursor or a touch on the screen, etc. [55] Statistical and/or probabilistic calculations may also be performed to identify the particular item to which a clicking corresponds. For example if a curser is clicked between two or more items, such calculations will determine which item is more probable to have been targeted. In some embodiments more than one item may be determined as being the target of a click.

[56] In one embodiment of the invention even recorded programs can be used with this advertising capability. A recorded program in this context means any program for which the time of playing is under the control of the user; for example a pre recorded program on user's recording device, an on-demand movie, a DVD, etc. In this embodiment, a clicking log with relative times, along with the program information such as the name of the program, is sufficient for interactive shopping. For example, the clicking log and the program information can be uploaded to an appropriate website or entered into a specific database or a computer program that maps the clicking log into the product information. In another embodiment the advertisement information itself may reside on the same storage medium as the program itself, such as on a rental DVD. Again, same method is applicable to the recorded radio or other multimedia programs. In general, the clicking information may either be kept by the user or be saved remotely, such as by a predetermined website.

Frame-Based Information Retrieval

[57] In some embodiments the item selection is based on frame-counting, which, similar to the time-based selection, does not require any modification or alteration of the program. A frame is one indivisible unit of image or other multi-media content that serves as the smallest addressable unit in a sequence of images, such as sequences in a movie segment. A common method of organizing and addressing a particular frame is the use of timecode. A timecode is a multi-segment number having an Hour-Minute-Second-Frame format, wherein each second has 29 or 30 frames, each minute has 60 seconds, and each hour has 60 minutes. An individual frame may be uniquely specified by providing a timecode. For example, a timecode of 1.03.12.08 indicates the eighth frame in the twelve-th second, in the third minute of the first hour of a movie. In one embodiment a frame-counter counts the number of frames shown (e.g., from the beginning of the program) and at each click of the viewer the frame number (the frame count up to the instant of clicking) is stored. The frame numbers are then mapped into a table of items present on the screen at the instant of clickings. Each entry of such a table includes available items present on the screen during a frame-window within which the clicking happened. Frame-window boundaries are specified by the frame count of the first frame of a scene and the frame count of the last frame of the scene. For example, for a slow pace movie, the mapping table may contain the items shown within each 80- frame windows or for a fast pace movie, the list may contain the items shown within 16- frame windows. Please note that while program modifications are not required by the disclosed embodiments, modifications such as adding start indicators to video and audio programs can facilitate some aspects of the interactive processes.

[58] Figure 5B is a schematic representation of a timecode format 550. Segment 552 represents a time in hours. Segment 554 represents a time in minutes. Segment 556 represents a time in seconds. And Segment 558 represents a number of frames passed the total time indicated by the first three segments. In this example, the umber of frames passed the total indicated time represents the number of frames gone-by during the very last second, at which time an item selection was made. This frame number for DVDs is between 1 and 30.

[59] Figure 5A schematically illustrates an arrangement 500 of frames of a multimedia program and a corresponding frame number associated with each frame, wherein the frame numbering starts from a specified frame of the program in accordance with an alternative embodiment of the present invention. Slices 502 and 504 show two frames of a program with respect to time 506 and their corresponding frame numbers, counted from a specified frame of the program. In Figure 5A, the frame number for the frame of slice 502 is N+l and the frame number for the frame of slice 504 is N+2.

[60] Since several frames may pass in a movie without featuring any selectable items, there is no need for the list to account for uninterrupted consecutive windows. Similarly if a selectable item is featured, uninterruptedly, during a 1000-frame scene, one entry in the list corresponding to a 1000-frame window will be sufficient and any click during that period will be mapped into the same entry. Therefore, in some embodiments the table may comprise windows of different durations or windows of different frame numbers, with or without gaps or overlaps between the windows.

[61] Table 2 is an exemplary table arranged by frame-windows and the items shown in the program within those scenes. In this table, for example, 700 signifies the 700^th frame counted from the start of a video program. Please note that, for example, the first and the second frame-windows are consecutive non-interrupted similar-duration frame-windows, wherein the second and the third frame-windows are consecutive unequal-duration frame- windows with a gap, and the third and the forth frame-windows are consecutive unequal- duration frame- windows with some overlap.

Table 2

[62] In all embodiments, regular advertisement clips that are interleaved with a program can also be treated as part of the program and, therefore, the users can interact with the advertisement clips the same as any other part of the program. In other embodiments, depending on the table formation, the frame counts may exclude the interleaved advertisements.

[63] Figure 3 is a flow diagram of steps involved in an administrative preparation of data that is utilized in a non-intrusive interactive advertising which employs a video, audio, or multimedia program, according to yet another embodiment of the present invention. Figure 3 is also an example of how an administrator populates a table for a specific program. At step 300 the administrator starts the data entry routine. At step 310 the administrator opens the multimedia data stream, for example by playing a movie on a DVD. At step 320 the administrator moves to a new scene or segment of the program. At step 330 the administrator identifies any item(s) which can be sold or advertised or that can be interesting for a viewer. At step 340 the administrator registers the time or frame information during which one or more items are featured in the program. At step 350 the administrator resisters the identified item(s) in association with its corresponding time or frame information. At step 360 the administrator decides whether to continue to a new scene or segment of the program or to stop the process at step 370.

[64] Please note that other attributes and variations of the time-based embodiments equally apply to frame-based embodiments. Please also note that any frame identifier can replace the frame count for associating a click with a segment of a program. For example, storing the frame identifier at the time of a click will uniquely associate the click to the instant scene and is transformable to the item list of that scene. In general, any form of segment identifier within a program, such as a frame identifier or a frame number, can be used for associating an item list with the clicked segment.

[65] In time-based and frame-based embodiments an item featured in a video program may be added to the list of selectable items or removed from the list of selectable items in a matter of minutes, in a manner transparent to the viewers, while thousands of copies of the program DVDs are in circulation or the program is scheduled to be shown on different TV channels. The methods used in these embodiments are independent of the video programs, therefore, a list of selectable items and services may be altered at any time while the video programs are in the hands of viewers/consumers or even being watched at the time of alterations.

[66] Because there is no need to alter or modify any program, another advantage of these embodiments is that a user, given permission, can herself add her desired items to the list of the for-sale items of any program. For example, the user can sign-on to a specific website and identify the name and the segment of the program (e.g., by lapsed time) in which the desired item is to be advertised and be allowed to add her item information to the corresponding table of that program. In such embodiments all or most of the item-tables can be populated by the users themselves.

[67] It must be noted that different viewers may use different embodiments when watching the same video program. Basically, the embodiments used by the viewers for taking advantage of this method of advertising depend on the technologies available to the individual viewers and are not a function of the video program being watched. In other words, the same video program can be viewed using any of the presented embodiments if the required technology/apparatus is available to the viewer. As a result, a viewer may have the capability of using attributes of multiple embodiments at the same time. In any embodiment, time-based or frame-based approach can be replaced by other means of associating a scene with the list of the items and/or the information related to the items featured in that scene. For example, a TV network can send a synchronized advertisement-banner signal that overlaps the scenes of a video data stream such that the advertisements shown in the banner are synchronized with their corresponding scenes. These embodiments also allow the providers to send different information for the same item to different demographic areas, such as different prices or addresses of stores.

[68] In general, depending on the table-provider to which a user connects (e.g., different websites), or different item-tables that a specific table-provider may offer for a particular movie, the advertisements and the for-sale items presented in the movie may be different. Or, if a manufacturer sponsors the entirety of a movie, all advertisements and for- sale items of that movie may exclusively belong to that manufacturer, regardless of the table provider; however, this exclusivity is only a legal matter - other manufacturers and item sellers can still prepare other item-tables for the same movie and present their own advertisements and for-sale items, for example, through their own websites.

[69] An exemplary apparatus for implementation of an exemplary embodiment of the disclosed advertising method has means for (1) receiving a user's click signal; (2) associating the viewer's signal to at least one item shown in the video program at or about the time of sending the signal; and (3) retrieving the advertising information associated with the selected item(s), for example through internet or from a storage medium, or storing the selected item(s) for later information retrieval. The exemplary apparatus may also have means for presenting the retrieved information to the viewer and means for ordering/buying selected items, for example by taking the viewer to the seller's website. The exemplary apparatus may also have means for performing all or some of these tasks while the video program continues playing, while paused, and/or after the video program is over, or a combination thereof.

[70] In an alternative embodiment, upon clicking on a segment of a program, a list of all or part of available items in that segment is presented to the user. For example while watching an on-demand movie, this list may overlap the scene or occupy a separate part of the screen. In another embodiment, at all times some icons, such as a banner, can remind the user what items are available for sale in that scene before the user even clicks on the scene. In such a case the user's clicking on the scene or on the icon will present her with more information about the items within the scene. In yet another alternative embodiment, a simple sign can indicate availability of at least one item in the immediate scene or future scenes.

[71] In yet another alternative embodiment, any user who is not interested in watching or listening to a program but still is interested in the advertisements and for-sale items in that program can obtain a list of all or some of the advertisements and for-sale items of that program. For example, the user can log-on to an appropriate website and by indicating a program identifier, such as the program name, receive the list of its advertisements and for-sale items and/or be able to utilize the list as does any user who is watching the program and clicking on it.

[72] In any embodiment, the time-based or frame-based approach can be replaced by other means of associating a scene or a segment of the program with the list of the items and/or to the information related to the items featured/presented in that scene/ segment. For example, each time a user requests item information, an RF signal that carries relevant identifying information is sent to a predetermined station where the receiving station, using its database, associates the signal with the scene of interest and a list of available items and transmits back the requested information. Or, as another example, the frames or scenes of a movie may have identifiers associated with them, such as frame numbers, which can be captured at the time the user signals and be used to retrieve the information associated with that particular scene or frame.

[73] Figure 4 is a flow diagram of some example processes involved in a user purchase utilizing a non-intrusive interactive advertising method in accordance with an alternative embodiment of the present invention. At step 400 the consumer starts the purchasing routine. At step 410 the consumer opens the multimedia data stream, for example by playing a movie on a DVD or by going to a website of her choice. At step 420 the consumer plays the multimedia content by a specified application, which is not required by some embodiments. At step 430, while watching or listening to the program, the consumer selects one or more items featured in a segment of the program by, for example, clicking on a scene. At step 440, for the selected items, the consumer receives information and shopping options from the database. At step 450 the consumer adds the item(s) to a shopping cart. At step 460 the consumer decides whether to continue shopping or to complete the purchasing the item(s). At step 470, if the consumer decides to finalize purchasing, the consumer reviews and edits the content of the shopping card. At step 480 the consumer completes the purchase of the content of the shopping card and ends the process at step 490.

Synchronized Information Retrieval

[74] In another embodiment, some advertisements and some for-sale items that are shown or aired in parallel with the main video or the audio program, are synchronized (e.g., time- or frame- synchronized or a combination thereof) with the main program in such a way that one or more advertisements, or for-sale items appear to be related to a scene of the video or a segment of the audio program. In this embodiment instead of measuring the time or the frame-count corresponding to a "click" and retrieving the advertisement and/or the item data that corresponds to the click measurements, advertisements and for-sale item data remain synchronized with the program. In other words, relevant advertisements and item-data are constantly being retrieved and made available to the user. In such embodiments, a click is merely a tool for the interaction of the user with the available data rather than a trigger for data retrieval/mining.

[75] The above detailed description of the embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific embodiments of, and examples for, the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified. Each of these processes or blocks may be implemented in a variety of different ways.

[76] Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. [77] The teachings provided herein can be applied to other systems, not necessarily the system described herein. The elements and acts of the various embodiments described above can be combined to provide further embodiments. All of the above patents and applications and other references, including any that may be listed in accompanying filing papers, are incorporated herein by reference. Aspects of the invention can be modified, if necessary, to employ the systems, functions, and concepts of the various references described above to provide yet further embodiments of the invention.

[78] Particular terminology used when describing certain features or aspects of the invention should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the invention with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed embodiments, but also all equivalent ways of practicing or implementing the invention.

[79] The above detailed description of the embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above or to the particular field of usage mentioned in this disclosure. While specific embodiments of, and examples for, the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. Also, the teachings of the invention provided herein can be applied to other systems, not necessarily the system described above. The elements and acts of the various embodiments described above can be combined to provide further embodiments.

[80] In general, the terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed embodiments, but also all equivalent ways of practicing or implementing the invention under the claims.

[81] While certain aspects of the invention are presented below in certain claim forms, the inventors contemplate the various aspects of the invention in any number of claim forms. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the invention.

Claims

CLAIMSI claim:

1. A method of interactive sales, advertising, and information exploration in a multimedia environment, wherein a signal is transmitted to select one or more items featured expressly or impliedly in or in association with a scene or a segment of a video or an audio program in order to receive information about or an opportunity to buy the one or more items, the method comprising:

computing with respect to the program, an absolute time, a relative time, a frame-count, or a frame-identifier during or about which the signal was generated, transmitted, or received, wherein the computed result also corresponds to the scene or segment of the program in which the one or more items are featured;

identifying the one or more items featured in the scene or the segment of the program that corresponds to the signal;

retrieving advertising or shopping data or other information, or a combination thereof, related to the one or more available items; and

presenting the retrieved data or information and shopping options or storing them.

2. The method of Claim 1, wherein the signal is a mouse-click, a touch on a screen, a voice command, a wired signal, a wireless signal, a body movement, a light signal, a sound signal, a brain- wave signal, or a combination thereof.

3. The method of Claim 1, wherein the video or audio program is a television program, a DVD, a CD, a film in a movie theatre, a radio program, a streamed program, an on-demand program, a multimedia program, or any multimedia program transmitted wired or wirelessly from a transmitter to a receiver, or downloaded to an electronic device, or played from a local data storage source.

4. The method of Claim 1, wherein a segment of the program is a single- frame- or a multi-frame- scene, a time -period of a radio program, or any time-measurable portion of a multimedia program.

5. The method of Claim 1, wherein the identifying step, the retrieving step, or both, utilize at least one table that includes:

a list of time-windows, frame- windows, or frame-identifiers, or a combination thereof, each of which represents a specific scene or segment of the program;

a list of item-identifiers, item information, or item pointers, or a combination thereof, each of which represents or provides some data and information and shopping options for one or more items featured during or around at least one listed program scene or segment; and

a table arrangement that correlates at least one entry of one list to at least one entry of the other list.

6. The method of Claim 5, wherein the program is divided into equal size, different size, overlapping, non-overlapping, uninterrupted consecutive, or interrupted, or a combination thereof, of time- windows, frame- windows or frame-identifiers.

7. The method of Claim 5, wherein a user, a system administrator, or both are enabled to populate or modify the table entries.

8. The method of Claim 5, wherein the signal has a screen manifestation and the "computing" step also includes computing coordinates of the screen manifestation of the signal, and wherein the table includes geometric boundary information of one or more items to enable the identification of specific item or items featured on the screen on or in the vicinity of the screen-manifestation of the signal.

9. The method of Claim 1, wherein the results of any of the steps are stored and are used by other steps at a future time, and wherein some steps are performed by logging in to a specified website and some steps are performed locally or through a local network.

10. The method of Claim 1, wherein a visual indicator on a screen signifies the availability of an advertisement or a for- sale item in the instant scene or segment of the program, and wherein the indicator is symbolic or is a substantially recognizable representation of the advertisement or the for- sale item, and wherein the indicator is visible while an advertisement or a for- sale item is available in the immediate scene or segment or the visibility of the indicator is controlled by a user.

11. The method of Claim 1, wherein a user is enabled to limit presentation of the retrieved data, information, and shopping options by specifying conditions.

12. The method of Claim 1, wherein: the absolute time is the actual time of a day in which the signal is generated, transmitted, or received;

the relative time is a time measured from any specified point of the program;

the frame-count is a result of counting the number of frames from any specified frame of the program to the frame at or about which the signal is generated, transmitted, or received; and

the frame-identifier is any number, sign, or symbol that distinguishes a frame or a group of frames from the rest of frames of the program.

13. An interactive method of sales, advertising, or information discovery, wherein at least some advertisements are presented in synchronization with some scenes or segments of a multimedia program such that each advertisement is related to at least one item portrayed in each corresponding scene or segment of the program, and wherein, using the advertisements, a user is able to buy or to receive additional information about the portrayed item or items, the method comprising:

presenting an advertisement in parallel with the multimedia program; synchronizing the advertisement with the multimedia program such that at least some information in the advertisement is related to one or more items portrayed in the corresponding synchronized scene or segment;

correlating a user request signal with a simultaneous advertisement; and

providing the user with additional information and shopping options based on the simultaneous advertisement.

14. The method of Claim 13, wherein to synchronize the advertisement with the program, a synchronization signal is transmitted in connection with the program.

15. The method of Claim 13, wherein the user is enabled to specify conditions to limit the additional information and shopping options provided to her.

16. The method of Claim 13, wherein the results of any of the steps are stored and are used by other steps at a future time, and wherein some steps are performed by logging-in to a specified website, and wherein some steps are performed locally or through a local network.

17. The method of Claim 13, wherein one or more users are enabled to present some advertisements in parallel with the multimedia program.

18. The method of Claim 13, wherein the correlating step, the providing step, or both are single-layered or multilayered processes.

19. The system of Claim 13, wherein no modification of the multimedia program is required.

20. An interactive sales and advertising system, wherein a query-signal requests and receives information or purchasing options regarding an item featured expressly or impliedly in, or in association with, a multimedia program, the system comprising:

a data storage facility for storing a list of absolute times, relative times, frame- counts, or frame-identifiers and a list of item- identifiers, wherein at least one entry of the item-identifier list corresponds to one entry of the time and frame list, and wherein each item-identifier denotes one or more items featured in a scene or segment of the program that is identified by the corresponding entry of the time and frame list;

a detection facility for determining, with respect to the program, a time, a time- span, a frame, or a frame- window at or during which the query- signal has been generated, transmitted, or received; and

a retrieval facility, wherein the facility correlates the detected time or frame information of the query- signal with the entries of the time- or frame-list to find corresponding item-identifiers and to retrieve requested item information and purchasing options.

21. The system of Claim 20, wherein no modification of the multimedia program is required.

22. The system of Claim 20, wherein one or more users are enabled to populate or modify at least a portion of the data storage facility lists.

23. An interactive method of shopping and information gathering, the method comprising:

Selecting a time- slice or at least one frame of a multimedia program;

associating the selected time- slice or the at least one frame of the multimedia program with one or more items featured in the program at or about said time-slice or said at least one frame;

retrieving advertisements, shopping options, data, or information related to the one or more items; and

managing the retrieved advertisements, shopping options, data, or information.

24. The method of Claim 23, wherein the program is divided into equal size, different size, overlapping, non-overlapping, uninterrupted consecutive, or interrupted, or a combination thereof, of time-slices, frame-groups, or both.

25. The method of Claim 23, wherein the association step, the retrieval step, or both are single-layered or multilayered processes.

26. The method of Claim 23, wherein managing step further comprises storing shopping options, data, or information; presenting shopping options, data, or information to a user; connecting the user to at least one appropriate website; performing one or more shopping transactions for the user; or a combination thereof.

27. The method of Claim 23, wherein selecting the time-slice or at least one frame of the multimedia program is performed by computing, with respect to the program, an absolute time, a relative time, a frame-count, or a frame-identifier during or about which a user-query- signal has been generated, transmitted, or received.

28. The method of Claim 23, wherein some users are enabled to limit the retrieved material by specifying user-conditions.

29. The method of Claim 23, wherein the method is non-intrusive and no modification of the multimedia program is required.

30. An interactive method of shopping and information gathering comprising:

playing advertisements in parallel and in synchronization with a multimedia program such that one or more of advertised items are portrayed in a corresponding synchronized scene or segment of the program;

selecting an advertised item;

retrieving shopping options or information related to the selected item; and

presenting or storing the retrieved information and shopping options.

31. The method of Claim 30, wherein to synchronize the advertisements with the program, a synchronization signal is transmitted in connection with the program.

32. The method of Claim 30, wherein one or more users are enabled to specify conditions to limit the advertisements.

33. The method of Claim 30, wherein the results of any of the steps are stored and are used by other steps at an appropriate time, and wherein some steps are performed by connecting to a specific website and wherein some steps are performed locally or through a local network.

34. The method of Claim 30, wherein one or more users are enabled to play some advertisements in parallel with the multimedia program.

35. The method of Claim 30, wherein the selecting step, the retrieving step, or both are single-layered or multilayered processes.

36. The method of Claim 30, wherein for selecting an advertised item, a time at which a selection-signal is generated, transmitted, or received, is correlated with an advertisement instance and a corresponding advertised item.

37. A computer- implemented method of associating items included in a multimedia segment with information about the items using a data structure, wherein the data structure includes a time or a frame field, each entry of which represents a scene or a segment of a multimedia program, and an item-information or an item-pointer field, each entry of which represents one or more items featured in or around a scene or a segment of the program, the method comprising:

associating a requisition signal to an entry of the time or the frame field; and

correlating the requisition signal, through the associated time or frame entry, to one or more corresponding item-information or item-pointer.

38. A computer-readable storage medium having instructions encoded thereon that, when executed on a computer, perform the method of Claim 37.