WO2016073907A1

WO2016073907A1 - Interactive electronic data-sharing platform with virtual card interface

Info

Publication number: WO2016073907A1
Application number: PCT/US2015/059558
Authority: WO
Inventors: Emil LAU
Original assignee: Kube-It Inc.
Priority date: 2014-11-07
Filing date: 2015-11-06
Publication date: 2016-05-12

Abstract

A client device receives and executes a distributed application for sampling output by a player device that is playing audio-video content, and obtains information (e.g., advertising) relating to an object that is imaged or referenced in a portion of the playing content. The client device outputs the information on the client device while the content is playing on the player device, and optionally receives user input responsive to the information. The user of the client device receives the advertising or other information without any interruption of the content playing on the player device. A server handles requests for the information. Prior to receiving the request, the server records the information in association with an identifier for the portion of the recorded audio-video work and tracks requests for the information from client devices.

Description

INTERACTIVE ELECTRONIC DATA-SHARING PLATFORM WITH VIRTUAL

CARD INTERFACE

FIELD

[0001] The present disclosure relates to applications for signal processing and the operation of visual user interfaces to provide information related to independently playing content and a user interface in an interactive user interface device that organizes and shares the information based on user input.

BACKGROUND

[0002] People spend an increasing amount of time consuming various types of recorded or computer-generated content during their leisure time, for example video from both professional production studios and amateur sources through traditional broadcast or cable sources, or over computer networks; various types of video game or similar interactive content, content posted on online social media sites, or just content from the World Wide Web generally. A significant fraction of this content or the systems used to deliver it are funded by advertising. Such advertising includes traditional broadcast ads, streaming video or audio ads, web page banner or pop-up ads, product placement within content, interactive icons or regions in streaming video, and other forms. Such ads interfere with the user experience in various ways, such that much advertising is viewed as a negative.

[0003] Another aspect of content consumption and advertising is the tendency for content to be consumed by a user in many different venues and using many different delivery platforms. For example, a user may view content on a television; or using a mobile phone, a notepad device, a laptop or desktop computer; or may view content in a cinema or on an electronic billboard. Some delivery platforms by be devices owned or controlled by the person viewing the content, while other platforms may be under the control of other people. The diversification and fragmentation of content delivery channels makes it correspondingly hard to integrate delivery of advertising or other information across multiple platforms or channels.

[0004] It would be desirable, therefore, to develop computer-implemented methods, network systems, user interfaces, and other new technologies for applications of signal processing and the operation of visual user interfaces to provide information of interest and a user interface in an interactive user interface device that organizes and shares the information based on user input, that overcomes these and other limitations of the prior art, and enhances the ease, convenience and utility of electronic communication.

SUMMARY

[0005] This summary and the following detailed description should be interpreted as complementary parts of an integrated disclosure, which parts may include redundant subject matter and/or supplemental subject matter. An omission in either section does not indicate priority or relative importance of any element described in the integrated application. Differences between the sections may include supplemental disclosures of alternative embodiments, additional details, or alternative descriptions of identical embodiments using different terminology, as should be apparent from the respective disclosures.

[0006] Aspects of the disclosure include a method performed using a client device. As used herein, a "client device" may be, or may include, a device including a computer processor, memory, video output device (e.g., an LCD display screen) and related components, for example, a smartphone, notepad computer, laptop computer, or similar computing device for consumer use. The method may include generating, by a client device, a signal sample based on at least one of an audio or video signal output by a player device that is playing a recorded audio-video work, wherein the player device is distinct from the client device. The client device may be, for example, a mobile smart phone with an interactive touch screen, microphone, camera, transceiver for a wireless phone network, processor, and memory, among other things. The player device may be any other machine, computerized or not, that produces audio or video output from a digital or analog recording, including output from streaming audio or video content. The sample may be generated by recording audio output from the player device using the microphone of the client device, or by some other method.

[0007] The method may include obtaining, using the client device, information relating to an object that is imaged in a portion of the recorded audio-video work from which the signal sample is generated, based on the signal sample. The obtaining may include, for example, transmitting the signal or a representation of the signal to a database application operating on a server, obtaining the information, and receiving the data record from the server. In the alternative, or in addition, the obtaining the information may include identifying, using the client device and/or the server, the recorded audio-video work and the portion thereof from which the signal sample was generated, based on the signal sample, and locating the information based on an identity of the recorded audio-video work and the portion. The object may be imaged in a visible portion of output from the recorded audio-video work. In the alternative, or in addition, the object, or metadata about the object, may be imaged in a non-visible portion of the recorded audio-video work. The audio-video work may be recorded in a non- transitory medium, for example, an electronic memory storage device, a magnetic computer-readable medium, or an optical computer-readable medium.

[0008] The method may further include outputting the information in an interactive user interface of the client device, in response to obtaining the data record, while the audio-video work is playing on the player device. Outputting the information further comprises displaying an interactive Graphical User Interface (GUI)-enabled data object containing the information on a display screen of the client device. In some embodiments, the GUI-enabled data object may resemble an "card" when displayed on a displayed device. The GUI-enabled data object, sometime referred to herein or in the accompanying materials as a "KUBE-iT"™ card, or "card" for short, may comprise a user interface object that includes information about the imaged object, links to further information about the object, and command objects or links to a command interface for user-controlled disposition of the KUBE-iT™ card. Disposition may include, for example, saving the card in a default category of a user database, saving the card in a user- specified category of a user database, sharing the card with a designated user, adding a card to a wish list, expressing interest in a card, expressing disinterest in a card, discarding a card, requesting further information about an object described by a card, a combination of the foregoing or other disposition. Data regarding cards received by the client device and any related dispositions may be stored in a memory related to a user account, for example, in a cloud-based storage collection of records associated with a particular user name.

[0009] Absent receipt of a user command directing a specific disposition of a card, the method may include removing the interactive GUI-enabled data object containing the information from the display screen upon the earlier of: expiration of a designated period of time since initiating the displaying, or receiving later information relating to a different object that is imaged in a subsequent portion of the recorded audio-video work.

[0010] The method may further include saving, in a computer memory, an identifier for the information in association with a user identity of a user of the client device. For example, the method may include saving an indication of affirmative user interest in association with the information, in response to receiving user input via the interactive GUI-enabled data object. The saving may include, for example, associating the identifier with a category indicator indicated by the user. In another aspect, the saving may include sending the information and category location to the computer memory located in a network location in association with an account identifier enabling access to the information and category using other client devices.

[0011] In another aspect, the method may include tracking indications of affirmative user interest in additional interactive GUI-enabled data objects for different objects, and crediting a user account, for example with reward points, based on at least one of a number or a frequency that the user provides the indications of affirmative user interest.

[0012] The method may further include displaying a user interface screen for purchase of the object by the user, in response to user selection of an interface control object for initiating a purchase transaction. The method may include displaying a list of eligible information categories on the display screen configured for enabling user selection of the category indicator. The method may include providing a user interface enabling the user to designate at least one of the information categories for publication to a designated social group as a category of special interest (e.g., a wish list) to the user. The method may include, in response to user selection of a category on the list, obtaining and displaying a collection of card like data objects each previously categorized under the category by the user. The method may include, in response to user selection of a category on the list, obtaining and displaying a collection of card like data objects each previously categorized under the category by other users who have designated the category as shared with the user.

[0013] In another aspect, KUBE-iT™ card data may be provided based on geo-location or other location-based trigger such as, for example, receiving a wireless signal from a signaling device placed in a retail location. Accordingly, the method may include obtaining, using the client device, object information related to an object selected based on a current location of the client device, and outputting the information at least in part by displaying another interactive GUI- enabled data object containing the object information on the display screen. This provides the convenience of the KUBE-iT™ card information management in contexts other than viewing recorded audio-video content.

[0014] The client device comprising an interactive touch screen, microphone, camera, transceiver for a wireless phone network, and memory coupled to a processor. The microphone and/or camera may capture the audio or visible signal samples and transmit to the processor. The processor may perform a digital signal analysis of the samples and send information from the sample or the samples to an identification server. The touch screen may receive user touch input and send electronic signals to the processor in response to the touch input, and display the interactive KUBE-iT™ cards to the user. The memory may hold encoded instructions, that when executed by the processor, cause the client device to perform operations of the method as described. The memory may comprise a non-transitory computer-readable medium.

[0015] In another, complementary aspect, a server-side method may include receiving, via a user interface of a server enabling user input, information related to an object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location. For example, the server may transmit a user interface object to a terminal, in response to a receiving a request direct to a Uniform Resource Locator (URL) of the server. The user interface object may include code that executes on the terminal in response to user selection of a user interface element of a web page that is accessed on the terminal via a web browser. The code may include one or more modules for describing or identifying the object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location in response to user input to the terminal, and relating the object to a portion (e.g., time line segment) of a particular audio-video work or to a geographic location.

[0016] The server-side method may further include recording, in a data structure accessible over a computer network, the information in association with an identifier for the portion of the recorded audio-video work or for the geographic location. For example, the server may transmit the information and an identifier for the audio-video work or location received from the terminal to a database server, with instructions for saving as related records in a database. The recording operation may further include associating the information with a category indicator based on a signal received from a client device, which is interpreted as indicating user selection of a category to which the card should be assigned.

[0017] The method may further include receiving, via the user interface, further information for controlling generation of an interactive GUI-enabled data object containing the information for display on a display screen. The information may include, for example, format information, image data including an image of the object, text data describing the object, and a set of user interface controls enabling a user to control disposition of the object. [0018] The server-side method may include tracking requests for the information from client devices that identify the portion of the recorded audio- video work or the geographic location in connection with the requests. The tracking may include, for example, saving, in a computer memory, user identities each associated with ones of the requests. The method may further include tracking additional information requests from the client devices that indicate a user interest in at least one of purchasing the object or obtaining further information about the object. The method may further include comprising tracking distribution of information based on one or more category indicators designated by end consumers of the information, or tracking distribution of information based on one or more social groups designated by end consumers of the information. The method may include crediting user accounts, for example with reward points, based on at least one of a number or a frequency that of the requests per user. The method may include determining an advertising value for distribution of the information based on a quantity of at least one of the requests or the additional information requests within a define time period.

[0019] An server apparatus may include a processor, a memory coupled to the processor, and a network interface coupled to the processor, wherein the memory holds instructions that when executed by the processor, cause the apparatus perform the operations described above, and elsewhere herein. The memory may be, or may include, a non-transitory computer-readable medium.

[0020] To the accomplishment of the foregoing and related ends, one or more examples comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative aspects and are indicative of but a few of the various ways in which the principles of the examples may be employed. Other advantages and novel features will become apparent from the following detailed description when considered in conjunction with the drawings and the disclosed examples, which encompass all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The features, nature, and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference characters identify correspondingly throughout and wherein:

[0022] Fig. 1 is a block diagram illustrating overall aspects of a system for signal processing and the operation of visual user interfaces to provide information of interest and a user interface in an interactive user interface device that organizes and shares the information based on user input, including a KUBE-iT™ card information management system.

[0023] Fig. 2 is a block diagram showing a high-level technical architecture pertaining to a cloud-based implementation of a system as shown in Fig. 1 .

[0024] Fig. 3 is a block diagram showing aspects of a technical architecture pertaining to a consumer platform of a system as shown in Fig. 1 .

[0025] Fig. 4 is a block diagram showing aspects of a technical architecture pertaining to an advertiser platform of a system as shown in Fig. 1 .

[0026] Fig. 5 is a block diagram showing aspects of a technical architecture pertaining to an administrative platform of a system as shown in Fig. 1 .

[0027] Fig. 6 is a concept diagram illustrating other aspects of systems as shown in Figs. 1 -5.

[0028] Fig. 7 is a sequence diagram illustrating aspect of a use case for a KUBE-iT™ card information management system.

[0029] Fig. 8 is a sequence of screenshots illustrating aspects of a user interface of a client device for display and management of KUBE-iT™ cards.

[0030] Fig. 9 is a sequence of screenshots illustrating aspects of a user interface of a client device for setting up a category indicator for managing saved KUBE-iT™ cards.

[0031] Fig. 10 is a sequence of screenshots illustrating aspects of a user interface of a client device for purchasing a product described on a KUBE-iT™ card. [0032] Fig. 1 1 is a sequence of screenshots illustrating aspects of a user interface of a client device for sharing a KUBE-iT™ card via instant messaging.

[0033] Figs. 12A-B show screenshots illustrating aspects of a user interface of a client device for sharing a KUBE-iT™ card via various modalities (tweet, e-mail, or social networking site).

[0034] Fig. 13 is a flow chart illustrating aspects of a method by a client device for synchronizing and display of an interactive GUI-enabled data object on a the client device to external events detected.

[0035] Figs. 14-17 are flow charts illustrating additional aspects of the method of claim 13.

[0036] Fig. 18 is a block diagram illustrating aspects of a client-side device for performing a method as shown in Figs. 13-17.

[0037] Fig. 19 is a flow chart illustrating aspects of a method by a server that services the synchronizing and display of an interactive GUI-enabled data object on a the client-side device.

[0038] Fig. 20 is a block diagram illustrating aspects of a server-side apparatus for performing a method as shown in Fig. 19.

DETAILED DESCRIPTION

[0039] Various aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that the various aspects may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing these aspects.

[0040] The present technology provides a digital tool that allows consumers to select, save and share information from their favorite TV show or the internet, and then initiates an electronic purchase or information request -- with a single "click" or other simple user input action. The technology enables a new medium for advertising enabling fewer interruptions for the consumer, and reaching those who fast forward through interruptive commercials. The technology enhances user enjoyment of video content and enables shortening of the timeline to conversion.

[0041] Referring to Fig. 1 , in some embodiments a system architecture 100 may include three distinct application platforms, each with its own user interfaces and web services, that share state-machine services and work together to create an overall technical solution and user experience, including providing KUBE-iT™ cards in coordination with separately playing content. Each application platform may be implemented as software executed by a computer server or servers coupled to a computer network. These platforms may include an End User Application Platform 106 that enables end users 102 (e.g., customers) to use the card-based information web/mobile applications and access system 100 consumer services & products. The platforms may further include an Ad-Agency Application Platform 108 that enables advertisers and content providers 104 to access Ad Agency Services, including, for example, placing and managing advertisements in the form of electronic information cards synchronized to portions of video content, or making content available for ad placement. The platforms may include an Administrative & Internal Services Platform 1 10 that enables administrators and customer service personnel to provision and manage the overall system 100 infrastructure and services. Modules and resources underlie these platforms, for example a class of application logic objects 1 12 that define operational characteristics such as business rules, ad persistence, user transactions, and analytical services. The system may also be serviced by a class of 3^rd party Application Program Interfaces (APIs) 1 14 for handling functions such as automatic content recognition (ACR), social media integration, and retailer integration. The system may also be supported by backend processing modules 1 16 handling actions such as user notifications, image processing, and job scheduling.

[0042] Referring to Fig. 2, a high-level technical design 200 of an electronic information card solution using Cloud Platform components such as, for example, Google™ App Engine, BigQuery, Datastore and Cloud Storage. The proposed architecture will have strong elements of scalability, reliability and security across all components. The design 200 includes backend services that may be developed, deployed and managed using the Google™ application engine (App Engine) or similar cloud resource. Using a cloud-based App Engine may enable development and launch of the system very quickly with minimal focus on infrastructure setup and management. For example, a cloud-based App Engine may be used to develop, deploy and manage the backend modules 208, 210, 212 that service requests from front-end mobile/web applications 202, 204, 206 for receiving and sharing card information. A web application 202 for ad agency functions enables access to advertising analytics developed from user interactions with KUBE-iT™ cards. An end user web application 204 may provide a user interface for user-facing KUBE-iT™ card function, for example receiving KUBE-iT™ cards and interacting with KUBE-iT™ cards, using a web browser application. An administrative web application 206 enables administrative management and configuration of information provided in KUBE- iT™ cards, coordination with content, and business function. The App Engine may be used to implement the user interface for browser clients, and expose APIs using Cloud Endpoints to mobile and desktop clients (for example consumers, ad agencies, internal admins). An cloud-based App Engine may function as the main system controller and manage all workflows for creating, serving and processing electronic information cards and products, and provide connectivity to other services (for example Maps API, Search API) and 3rd party components.

[0043] As seen in Fig. 2, the App Engine backend services may be divided into three different modules: End User Module 210, Ad Agency Module 208 and IT Admin Module 212. Modules are logical components that share state machine services and can communicate in a secure fashion. Each module may have one or more instances running based on configuration and workload. Splitting the overall application into modules provides certain advantages, for example:

• The different modules can be configured to use different runtimes and operate with different performance settings (e.g. the End User module can have higher-level performance settings than the Admin module).

• The modules can scale independently with appropriate growth in traffic. If there's a huge growth in consumers for any reason, only the End User Module will need to scale up.

• Breaking into modules reduces the overall code and configuration for a specific instance and thereby decreases the loading time when a new instance needs to be started.

• Each module can have its own version. So if a new Ad Agency feature needs to be added, that will not impact the End User Module.

[0044] The proposed system architecture 200 may also leverage the following App Engine components for delivering backend services. For example, backend services may include an Image Processing Service that provides dynamic image processing services such as thumbnail generation, resizing, and cropping for App Engine applications. This feature may help to optimize consumer mobile app 204 performance (size and speed) for image downloads of shows, networks and products. Backed services may further include a memory cache (Memcache) 316 (Fig. 3) that provides a distributed in-memory data cache service to store and retrieve frequently accessed data. This feature may further optimize database lookups for frequently accessed information such as, for example, video content (titles) and products that were aired in the last ten days. Backend services may further include Google Task Queue & Scheduled Tasks, that provides mechanism to perform asynchronous tasks outside of a user request such as, for example, daily maintenance tasks that run at scheduled times, or push notifications to users when a new show and products are uploaded.

[0045] The proposed system architecture 200 may also incorporate persistent storage, for example, hosting a data model and application content using Datastore 216, which may include a fully-managed and scalable NoSQL database solution. Like a cloud-based App Engine, Datastore 216 may support automatic scalability as traffic increases, automatic replication across multiple datacenters, high availability of reads and writes and continuous monitoring by a cloud service provider. Persistent storage needs for an effective solution are may include, for example:

• User Data: User login info & preferences, account settings, billing info;

• Ad Agency Data: Ad agency info, account settings, access permissions, billing info;

• Network & Show Data: Networks & Show/episode information;

• Product Data: Product info (including details, pictures, vendors, links, etc.);

• Performance Data: User lookups for KUBE-iT™ card-enabled shows and downloads/click-thru of Billing & Commerce: Billing transactions, invoices, payments from Ad Agencies

[0046] The proposed system architecture 200 may also use Cloud Storage 218 for storing static content (for example, product pictures placed on electronic information cards) and server logs. Cloud Storage 218 should be implemented as a durable and highly available object storage service that supports redundant storage, flexible access and strong, configurable security through access controls. Storage requirements for an effective technological solution may include:

• Network/shows/products media (pictures, videos);

• Audio fingerprinting of video titles;

• Server logs; and

• Analysis inputs/outputs and artifacts.

[0047] Where possible, the system 200 may use an existing cloud-based low- latency, edge-caching global network to deliver content faster to the customers. The proposed system architecture 200 may also offer a comprehensive analytics platform for Ad Agencies to analyze shows/product/user data and generate reports/dashboards on consumer interests and behavior. For example, a Google™ BigQuery module 214, or similar query service, may service a backend analytics platform. Such a module should be capable of analyzing massive datasets and providing real time insights to ad agencies on consumer behavior, for example tracking geographical distribution and demographic profiles of users downloading products for a particular video title. BigQuery 214 supports easy integration with App Engine for direct querying, bulk upload of data from Cloud Storage and streaming updates of up to 100,000 rows per second.

[0048] The analytic platform may further include Automatic Content Recognition (ACR) APIs 224 that provide advanced audio or video fingerprinting technology to let smart phones and tablets identify movies and TV programs by simply "listening" to a few seconds of the program dialogue and soundtrack. ACR APIs 224 also provide Web APIs for text based search of TV shows and episodes. Since many audio recognition and content validation APIs are already available, a system administrator may license a third party API for a usage-based fee. Popular ACR APIs currently in the market include GraceNote™, Audible Magic™ and Civolution™.

[0049] The system 200 may include may include social media sharing APIs 222 configured for enabling users to share information from Kube-iT™ cards with other users in social media environments. The system 200 may include may include retailer fulfillment and shopping APIs 220 for enabling users to place orders for goods or services advertised in Kube-iT™ cards.

[0050] Fig. 3 shows a technical architecture of a system 300 for servicing a consumer platform via an end user application 302 operating on a mobile client device, for example a smart phone or notepad computer equipped with wireless capability. Consumers can access the electronic information card system either through the a dedicated mobile application (for example, on an iOS and Android mobile phone) or directly through a mobile/desktop web browser. The system 300 may enable users to download a mobile app from the Apple/Google App store. Similarly, the system may enable users to scan an optical (e.g., QR) code available on a system website or online/print marketing collateral to directly go to download page for the application within the user's App Store. Users may login and use this App by: • Creating new account with email and password. Users may go through a two-step verification process.

• Authenticating using their existing media accounts (e.g.

Facebook™, Twitter™, Google+™ or Linkedln™ accounts.

[0051] The mobile/web application 302 may directly connect to the Automatic Content Recognition API 334 (e.g., GraceNote™ Entourage™) to recognize TV shows through audio fingerprinting. The End User Module 304 may provide an App Engine backend service to serve all incoming requests from the consumer mobile/web application. The client APIs for the end user module may be built using Cloud Endpoints - a service that generates client libraries for Android, iOS and web-based clients from one source. The End User Module 334 may support the following functionality for the users:

• Identify TV Shows: By using the Listen feature, the user can use the App 302 to listen to a show clip and identify the show and episode. Alternatively, users can directly search for the TV show and episode on the device.

• Interact with Products: Along with each show/episode, the App 334 may also download information regarding the products associated with that episode (in the form of electronic information cards). Each card may contain multimedia rich product information including product description, product related media (pictures & videos), product vendors and shopping services.

• Share/Shop/Save: users may browse downloaded cards to buy the product directly (through available sources), share with friends through email or social media and also save their card to their own personalized wish list.

[0052] An End User App 302 may include a sharing feature, which will allow users to share electronic cards (e.g., KUBE-iT™ cards) that they like or products they have purchased, with their friends on popular social networks. The system 300 infrastructure may provide integration to the social networking sites 322 (Facebook, Google+, Pinterest, etc.) and also leverage an Open Graph Protocol for defining the content to share. The system 300 may provide a Sharing and Purchasing Integration Service 310, enabling multiple options for the recipient to interact with the shared cards, for example:

• Enable recipient to directly view and buy the product from retailer sites 320;

• Enable recipient to view product information on a system website (esp. when viewing through a non-mobile device) and buy from available vendors (optionally create a new user login);

• Enable recipient to share items of interest from a Kube-iT™ card in social media posts, wishlists, or other formats; and

• Encourage recipient to download the End User App 302 and/or create a new user login to view the shared product.

[0053] In addition, the system 300 may perform analytics on effectiveness and social "buzz" caused by the shared content. For example, all tweets of a "KUBE- iT™" card product may include a "#kubeit" hashtag enabling analysis on number of reshares, responses to tweets, click on shared content, or other metrics.

[0054] Integrated shopping experience

[0055] The End User App 302 and system 300 may enable an enhanced, integrated shopping experience for users by providing direct integration with popular retailers. The different levels of integration are as below and may vary by the APIs supported by the retailer:

• Direct linking to Retailer store 320 (Minimal Integration): Allows the user to directly go to the store website within the App itself and browse the store or complete the transaction.

• In-App Product Detail and Preview: Allows the user to directly look up product details and pictures from the retailer's store directly within the system

• In-App Shopping Experience (Complete Integration): Allows the user to view all product information/details from the retailer website and also complete the actual transaction within the App itself (including features to select options like color, size; sign into retail store account; add promo codes)

[0056] To provide an enhanced shopping experience for the users, the system 300 may Leverage direct integration with storefronts where available (e.g. Amazon supports the Mobile Associates API that provide integration ranging from direct linking to complete shopping cart integration). The system 300 may use a universal shopping cart service for all other retailers (e.g. "Two Tap" supports over 200 retailers and can provides an integrated in-app shopping experience including signing into the retailer accounts).

[0057] Notifications & Geo-Proximity Alerts/Search: the system 300 may enable users to sign up for push notifications on their devices using one or more push notification services 328, 330. For example, push notifications may be provided for new products added for a show that the user is interested in, when the user is near a retail location that sells one of the products on the user's wish list.

[0058] The End User Module 302 may use Scheduled Tasks 312 and Task Queues 314 (Google™ App Engine services) to perform asynchronous notification tasks. For example, a "New Product" Scheduled Task may run every day to check new products added for all supported video titles and send a push notification to users who have added the show as a favorite. In this case, the Scheduled Tasks may identify all users to send notification alerts and add the notification tasks to the Google App Engine "New Product Notification" Task Queue. The Task Queue may be so configured that either the end user module (running on App Engine) or a separate service (running on Compute Engine) can pick up the items on the task queue and send out notifications to the users.

[0059] Similarly the system may enable users to perform Geo-Proximity Search for retail locations selling products listed on information cards of the system 300. For example, if a particular product is sold on Nordstrom™, system users may search using a Google™ Search API to perform the geo-proximity search. The Scheduled Tasks service 312 may also perform daily maintenance and clean-up tasks that need to run at pre-scheduled times. [0060] A technical architecture for an Ad Agency system 400 is shown in Fig. 4. Ad Agency advertisers may access Advertiser services through a system website (available on any JavaScript enabled mobile/web browser) serviced by the Ad Agency Module 306. The Ad Agency Module 306 may be an App Engine backend service to implement the user interface and serve incoming requests from the Ad Agency web application 402. Though the application is web-based, the client APIs for this module may be built using Cloud Endpoints so as to enable easy integration with a mobile application platform.

[0061] The Ad Agency Module may support the following functionality for advertisers:

• Account & Billing Preferences: Once the account is setup and provisioned by admins, Ad Agency advertisers can manage their account info and settings. They can add other users within their

organizations to access the services and setup necessary logins and permissions. They can update their billing details and also register for additional services (e.g. enhanced analytics

service).

• Manage Shows/Vendors/Products: Advertisers can search for specific shows/episodes and create/manage Product Cards (product description, images/videos, vendors and sale links) for that episode.

• Promotions & Coupons: Advertisers will be able to send special offers and coupons, only through the systems described herein, to their consumers to entice them to buy the items on their Product Cards.

• Billing & Transaction Details: Advertisers may access all billing information including weekly/monthly billing summary, billing by show/products and past transactions.

[0062] The system 400 may provide a comprehensive Advertiser Analytics interface for Advertisers to measure the performance of their advertisements and also better understand consumer behavior. The analytics platform will provide valuable insights that advertisers can use to push targeted promotions and coupons through the system 400. Using the Analytics dashboard, advertisers may:

• See real-time data and generate reports on consumer metrics for their advertisements (consumer downloads, likes, shares, click-thru to buy, click-thru to detailed info, etc.);

• See real-time data and generate reports on billing data and metrics for their advertisements (e.g. consumer engagement, reach, CPC, CPA, CPM, CTR, CTA); and

• Analyze consumer purchasing and behavior trends based on demographic information (age, gender, marital status, locale, income, etc.) for system advertisements (by product category, show, etc.).

[0063] In the backend, the basic analytics requests (e.g. clicks associated with a particular product, etc.) may be handled by the Ad Agency Module that will query the memory cache 316 or datastore 324 for the required information. The memory cache 316 that provides a distributed in-memory data cache service to store and retrieve frequently accessed data. An image service 318 may serve graphic images uses in Kube-iT™ cards. Less frequently accessed data for Kube-iT™ cards may be retained in the App Engine datastore 324. Other complex analytics that require big data processing (e.g. demographics and location of users who have downloaded any product for a particular show) will be handled by the Google BigQuery analytics 406 engine.

[0064] The system 400 may include an automated character recognition (ACR) Web search API 404. The ACR Web search API 404 may enable Web search functionality for system users.

[0065] A technical architecture for an Administrative and Internal Services system 500 is shown in Fig. 5. Administrators may access the following services to provision and manage the services and also monitor the overall infrastructure. A Web Admin module 506 enables administrators and customer service representatives to provision and manage system services for customers and advertisers. The Admin module 308 running on Google App Engine will service all incoming request from the Web Admin module 506 and will provide the following services:

• For Advertisers:

o Admins will be able to provision a new account for an ad agency (i.e. add users, set up logins, provide access to system services, create/update company and billing contact/details) o Admins can create/manage network/show/vendor information as required

o Admins can view all currently live and disabled products being advertised and create/manage products on behalf of the advertisers

o Admins will be able to view/update Advertiser billing details, current balance and pastp ayment transactions

• For Consumers:

o administrators will be able to perform password resets for consumers with logins.

[0066] The system 500 may include a Monetization Dashboard module 504 to review/analyze advertisement performance metrics as well as overall ad agency revenue and performance (understand month to date, quarter to date and year to date usage of information card services, revenue generated, etc.).

[0067] The system 500 may include a Monitoring Dashboard module 502 for monitoring system infrastructure. The Monitoring Dashboard 502 may communicate with an App Engine Instance Dashboard 508 to review the health and performance of the overall KUBE-iT™ card infrastructure. The Dashboard will show the currently running instances assigned to the system applications along with details on average QPS, average latency, number of requests over the last minute, current memory usage and age of instance. Users can shutdown/create new instances as well as modify application settings that impact application performance and scalability. [0068] Other operational details of user interface features may be provided as described below.

[0069] A consumer using the KUBE-iT mobile app or the KUBE-iT web user interface is enabled to save and organize their KUBE-iT™ Cards on their KUBE- iT Wish List by categories. For example, if a consumer is shopping for his or her mom's birthday, they can add KUBE-iT™ cards to a category called "Mom's birthday ideas" as they discover various products throughout the year. In addition, when saving a KUBE-iT™ Card to the KUBE-iT Wish List, a consumer can associate the KUBE-iT™ card to a person or multiple people on their mobile phone contact list. The system may enable the consumer to view their KUBE-iT Wish List categories and the associated KUBE-iT™ cards contained in those categories from any device connected to their KUBE-iT account, including the KUBE-iT app or web browser on their mobile devices, PC, or Macs. The consumer can save new KUBE-iT™ cards to existing categories on their KUBE- iT Wish List, create new Wish List categories for their KUBE-iT™ cards, and move KUBE-iT™ cards from one category to another. A consumer using KUBE- iT can set reminders on particular KUBE-iT™ cards or KUBE-iT Wish List categories. For example, a consumer has created a KUBE-iT Wish List category called Mom's Birthday Ideas. The consumer could input their mother's birthday on that wish list category and set a reminder to be alerted about the upcoming birthday in advance. For example, reminders of 1 -week, 2-weeks, or a custom inputted time-frame may be set for the reminder.

[0070] The system may enable consumer to earn points and/or participate in games so that there are rewards for the more KUBE-iT™ cards the consumer downloads and saves, the more points and/or rewards they can earn from the advertiser or others. Consumers using KUBE-iT can share an individual KUBE- iT™ Card, multiple KUBE-iT™ Cards, and one or more KUBE-iT Wish List categories containing their associated KUBE-iT™ cards with friends and family on their social network. A consumer can choose the recipient(s) that can view their KUBE-iT™ Card(s), or KUBE-iT Wish List categories. For example, a consumer can set up a KUBE-iT Wish List category for "My Birthday Wishes." The consumer can then populate that KUBE-iT Wish List category with KUBE- iT™ Cards for products they see on TV and that they might want for their birthday. The consumer can then share that birthday wish list with his or her parents and siblings so they can view the KUBE-iT™ Cards and buy products that the consumer wants for their birthday. At any time, the consumer can see who has access to view their KUBE-iT Wish List and revoke their access if desired.

[0071] The system may enable a consumer to sort/search their KUBE-iT Wish List and KUBE-iT™ Cards in several ways. For example, sorting or searching may be organized alphabetically (e.g., by product name or show with which it is associated), by recent activity (display categories that have had cards most recently added), by time: newest to oldest, and oldest to newest, by vendor, by category, by price, or by person to whom the card is associated with.

[0072] Fig. 6 is a concept diagram illustrating other aspects of systems as shown in Figs. 1 -5. A system 600 may include a client device 604 (e.g., smartphone or other mobile device) including a processor 606 coupled to a memory 608, to a display screen 610 and audio output transducer (not shown), and to a user input device, for example a touchscreen 612 that may be layered into the display device 610. The processor 606 may further be coupled to other input sensors, for example an audio input transducer 616 (for example, a microphone) and optionally a digital camera 614 including a lens and digital image sensor. A user 650 may view the display screen 610 and provide touch input via the user input device 612 or audio input via the input transducer 616. The client device 604 may be located in a locality 624 which may be, for example, an interior room of the user's residence, a portion of a retail space, or anywhere sufficiently close to the player device 618 to enable the user 650 to experience audio and video content output by the player device or and within range of the sensors 614, 616.

[0073] The player 618 may receive encoded audio-video content 622 comprising a video title (e.g., a show or program) from any source, for example a content management server 626. The player device may output the show or program on a display screen, for example an LCD, DLP, LED projector, or other display device with audio output from an audio output transducer 620. Other elements of system 600 may include an advertising terminal 628, ad management server 630 and data store 632, which like the client device may be coupled through one or more wired or wireless networks, for example, a Wide Area Network (WAN).

[0074] Fig. 7 is a sequence diagram illustrating aspect of a use case 700 for a card information management system like system 600, involving a player device 702, a client device 704, and content analysis (e.g., ACR) server or module 706, and an electronic information card management server 708 in communication with the content analysis module 706 and client device 704 via one or more wired or wireless networks. The player device 702 need not be connected to other nodes of the diagram via any network, and in the general case may be considered uncoupled. A user may initiate playing of content 712 by the player 702 or encounter the player already playing the content, comprising some audio- video title or segment. Recognizing that the content is "KUBE-iT™" card enabled (for example by a logo presented onscreen) the user may start up the end user application. Accordingly, the client device 704 may receive user input 710 and activate an end user application 714 from a device memory. Subsequently, the application may cause the client device 704 to initiate a session 716 with a content recognition service 706, e.g., an ACR server. The service 706 or device 704 may in response initiate a session 718 with a card management server 708.

[0075] Once the session 716 is initiated, the client 704 may detect electromagnetic signals (e.g., optical, radio, or infrared) from the player device or audio signals (sound) resulting from audio-video output 720. The client 704 may initiate a sampling routine to capture a sample or representation 722 of the detected signals and provide the sample 724 or representation to the content analysis service 706. The content analysis service 706 may analyze the sample 726 to identify the content title playing on the player device 702, and optionally, a time or portion of the title from which the sample was taken. Once the sample is recognized, the content analysis service 706 may provide 728 a content identifier, with or without a portion/time identifier to the card management server 708. In response to receiving the identifier(s) 728, the card management server identifies a set of information for an electronic card that is associated with the identifier(s), using a relational database or the like. The card information may include, for example, a product image(s) and description, price and vendor(s) identity.

[0076] Upon identifying the card information, the card management server 708 may provide 732 the information to the client device 704. In response to receiving 732 the information from the server, the client 704 may display an interactive electronic information card including the information 734. The user may view the card and provide responsive input 736 through its interactive features, or provide no response. The client 704 may detect the user input 736 or lack of user input and generate a disposition instruction regarding the card 738, for example, "keep in such-and-such wishlist," "share with so-and-so," or "discard - no interest." Upon generating such instructions the client 704 may provide the instructions 740 in a coded transmission to the card management server 708.

[0077] Upon receiving the instructions, the card management server 708 may execute the same and save a record of the disposition in a tracking database. Optionally, the server may transmit a confirmation 744 to the client that the instructions were executed. In the alternative, the instructions may be provided directly from the client 704 to another third-party server, for example to a social networking site for sharing, or to an e-commerce server for initiating a purchase or other transaction.

[0078] Upon receiving confirmation of execution 744, the client may display a confirmation message 746 on its display screen. The operations 712-744 may be repeated for so long as the content will play or the user desires. When it is time to stop the information card session, user input may be received 748 indicated a termination request. In response, the client device 704 may initiate a closing sequence 750, including, for example, closing its session with the analysis server 706, which in turn may cause the server 706 and 708 to close a session. After closing operations are completed, the client device may terminate 756 the end user application.

[0079] Novel aspects of the technology include the user interface configurations, enabling advertising to end users in a way that enhances and does not interrupt their enjoyment of video content, using sensor-based sampling and display on a client device auxiliary to the content player device. Fig. 8 is a sequence of screenshots 800 illustrating aspects of a user interface of a client device for display and management of KUBE-iT™ cards. During signal sampling, an end user application may display a "listening" screen 802. Once the content portion is identified and the client receives card information, an initial interactive electronic information card 810 may include an application status bar 850, content identification field 852, product description 854, product image 856, purchase interactive object 860 and wishlist interactive object 858. The user may scroll through a sequence of such cards 820, 830, 840 until finding one advertising a subject of interest. Old cards may be retained for a period of time, and automatically deleted from the client device if not saved in a wishlist.

[0080] Fig. 9 is a sequence 900 of screenshots illustrating aspects of a user interface of a client device for setting up a category indicator for managing saved KUBE-iT™ cards. The user interface enables a user to save cards in a personal wishlist for organization, and/or share them and send out. A user may also share contents of a category with others, as a wishlist or gift suggestion list. An initial screen 910 may include an interactive object 950 for adding a new category (i.e., wishlist) and a list of existing categories. Upon selecting the "add category" object 950, a second screen may include a text entry field 954 and keypad object 954; voice-to-text may also be used. Once the new category name is specified based on user input as shown in screen 930, the user may select a "save" object 958 to save the category or a "cancel" object 960 to undo. If the user input indicates "save," a next screen 940 may show the new entry 964 in the category list with a confirmation indicator 962 showing that the item has been saved in the new category. Alphabetical icons in the left-most column of the category list may be used to drill down to categories beginning with the specified letter only. [0081] Fig. 10 is a sequence 1000 of screenshots illustrating aspects of a user interface of a client device for purchasing a product described on an interactive electronic information card. The interface may enable a user to view complete a purchase of a product described on an information card, view product reviews before or after completing the purchase, and to submit a product review after purchase. A first screen 1010 shows a summary of the card 1051 , a source field 1052, with corresponding price from that source 1056 (one row of many possible shown), and an interactive "buy" object 1054, in this case interactive text. The display 1010 may appear as part of a list of cards in a wishlist or in a list of recently received cards, for example, or may be provided by scrolling to a portion of an opened and displayed card. If "buy" input is received, the user interface may display a "payment processing" indicator 1058 and supplemental information 1060. An auxiliary message 1060 may also be provided. Specifying a payment method may be done though the mobile interface or web interface at an earlier time. If the payment clears and the transaction is completed, a next screen 1030 may show a success indicator 1062. Prior to or after a purchase, a card screen 1040 may include a link 1064 for writing a product review or interactive icon 1066 for sharing the card information on a social media site. A sharing screen 1050 may show more detailed sharing options including icons for instant messaging, mail, Twitter™ or other modality.

[0082] Fig. 1 1 is a sequence 1 100 of screenshots illustrating aspects of a user interface of a client device for sharing information form an electronic information card. An initial screen 1 1 10 may show sharing modalities 1 150 and other disposition options 1 152. A next screen 1 120 may be displayed in response to user selection of an instant messaging sharing modality, and may include a summary 1 158 of the card information with a key pad interface 1 156 for adding additional text. For example, at screen 1 130, the user has entered additional text 1 160 and when ready may select at "send" command 1 162 (or cancel the message). Fig. 12A shows a screen 1210 of the user interface for tweeting card information, including an information summary 1252, a user account field 1254, and key pad interface 1256 for adding text. Fig. 12B shows a screen 1220 for an email sharing modality, including an addressee line 1258, a text field 1260, image 1260 and keypad 1256 for additional text. Thus, sharing is convenient within the mobile user interface.

[0083] For further example, Fig. 13 shows operations of a method 1300 by a client device for synchronizing and display of an interactive GUI-enabled data object on the client device to external events detected. Figs. 14-17 illustrate optional operations or aspects 1400, 1500, 1600 or 1700 of the method of claim 13, any one or combination of which may be combined with the operations of method 1300, in any operative order. The method 1300 may include generating 1310, by a client device, a signal sample based on at least one of an audio or video signal output by a player device that is playing a recorded audio-video work, wherein the player device is distinct from the client device. The client device may be, for example, a mobile smart phone with an interactive touch screen, microphone, camera, transceiver for a wireless phone network, processor, and memory, among other things. The player device may be any other machine, computerized or not, that produces audio or video output from a digital or analog recording, including output from streaming audio or video content. The sample may be generated by recording audio output from the player device using the microphone of the client device, or by some other method.

[0084] The method 1300 may include obtaining 1320, using the client device, information relating to an object that is imaged in a portion of the recorded audio- video work from which the signal sample is generated, based on the signal sample. Referring to Figs. 13-14, the obtaining 1320 may include, for example, identifying 1410, using the client device and/or the server, the recorded audio- video work and the portion thereof from which the signal sample was generated, based on the signal sample, and locating the information based on an identity of the recorded audio-video work and the portion. In the alternative, or in addition, the method 1300 may include transmitting 1420 the signal or a representation of the signal to a database application operating on a server over a computer network, obtaining the information, and receiving the data record from the server. The object may be imaged in a visible portion of output from the recorded audio- video work. In the alternative, or in addition, the object, or metadata about the object, may be imaged in a non-visible portion of the recorded audio-video work. The audio-video work may be recorded in a non-transitory medium, for example, an electronic memory storage device, a magnetic computer-readable medium, or an optical computer-readable medium.

[0085] The method 1300 may further include outputting 1330 the information in an interactive user interface of the client device, in response to obtaining the data record, while the audio-video work is playing on the player device. Referring to Fig. 15, outputting the information may include displaying 1510 an interactive GUI-enabled data object containing the information on a display screen of the client device. The GUI-enabled data object, sometime referred to herein or in the accompanying materials as a "KUBE-iT"™ card, or "card" for short, may comprise a user interface object that includes information about the imaged object, links to further information about the object, and command objects or links to a command interface for user-controlled disposition of the KUBE-iT™ card. Disposition may include, for example, saving the card in a default category of a user database, saving the card in a user-specified category of a user database, sharing the card with a designated user, adding a card to a wish list, expressing interest in a card, expressing disinterest in a card, discarding a card, requesting further information about an object described by a card, a combination of the foregoing or other disposition. Data regarding cards received by the client device and any related dispositions may be stored in a memory related to a user account, for example, in a cloud-based storage collection of records associated with a particular user name.

[0086] Absent receipt of a user command directing a specific disposition of a card, the method may include removing 1520 the interactive GUI-enabled data object containing the information from the display screen upon the earlier of: expiration of a designated period of time since initiating the displaying, or receiving later information relating to a different object that is imaged in a subsequent portion of the recorded audio-video work. [0087] The method may further include saving 1530, in a computer memory, an identifier for the information in association with a user identity of a user of the client device. For example, the method may include saving 1540 an indication of affirmative user interest in association with the information, in response to receiving user input via the interactive GUI-enabled data object. In another aspect, the saving may include sending the information and category location to the computer memory located in a network location in association with an account identifier enabling access to the information and category using other client devices.

[0088] In another aspect, the method may include tracking 1550 indications of affirmative user interest in additional interactive GUI-enabled data objects for different objects, and crediting a user account, for example with reward points, based on at least one of a number or a frequency that the user provides the indications of affirmative user interest.

[0089] Referring to Fig. 16, the method 1300 may further include displaying 1610 a user interface screen for purchase of the object by the user, in response to user selection of an interface control object for initiating a purchase transaction. The method may include, for example, associating 1620 the identifier with a category indicator indicated by the user.

[0090] Referring to Fig. 17, the method 1300 may include displaying 1710 a list of eligible information categories on the display screen configured for enabling user selection of the category indicator. The method may include providing 1720 a user interface enabling the user to designate at least one of the information categories for publication to a designated social group as a category of special interest (e.g., a wish list) to the user. The method may include, in response to user selection of a category on the list, obtaining and displaying 1730 a collection of card like data objects each previously categorized under the category by the user. The method may include, in response to user selection of a category on the list, obtaining and displaying a collection of card like data objects each previously categorized under the category by other users who have designated the category as shared with the user. [0091] In another aspect, KUBE-iT™ card data may be provided based on geo-location or other location-based trigger such as, for example, receiving a wireless signal from a signaling device placed in a retail location. Accordingly, the method may include obtaining, using the client device, object information related to an object selected based on a current location of the client device, and outputting the information at least in part by displaying another interactive GUI- enabled data object containing the object information on the display screen. This provides the convenience of the KUBE-iT™ card information management in contexts other than viewing recorded audio-video content.

[0092] With reference to Fig. 18, there is provided an exemplary apparatus 1800 that may be configured as a client device including at least one sensor, display screen, user input device, processor, memory and wireless transceiver, or as a processor or similar device for use within the client device. As depicted, the apparatus 1800 may include functional blocks that can represent functions implemented by a processor, software, or combination thereof (e.g., firmware).

[0093] As illustrated in Fig. 18, the apparatus 1800 may comprise an electrical component or the module 1802 for generating a signal sample based on at least one of an audio or video signal output by a distinct player device that is playing a recorded audio-video work. The apparatus 1800 may comprise an electrical component or the module 1804 for obtaining information relating to an object that is imaged in a portion of the recorded audio-video work from which the signal sample is generated, based on the signal sample. The apparatus 1800 may comprise an electrical component or the module 1806 for outputting the information in an interactive user interface of the client device, in response to obtaining the data record, while the audio-video work is playing on the player device.

[0094] With reference to Fig. 18, the apparatus 1800 may optionally include a processor module 1818 having at least one processor, in the case of the apparatus 1800 configured as a smart phone device with camera and microphone. The processor 1818, in such case, may be in operative communication with the modules 1802-1806 via a bus 1812 or similar communication coupling. The processor 1818 may effect initiation and scheduling of the processes or functions performed by electrical components 1802-1806.

[0095] In related aspects, the apparatus 1800 may include a sensor 1813, for example a microphone or camera, capable of detecting an audio signal or an electromagnetic signal. The apparatus 1800 may also include a wireless transceiver module 1814. A stand alone receiver and/or stand alone transmitter may be used in lieu of or in conjunction with the transceiver 1814. In further related aspects, the apparatus 1800 may optionally include a module for storing information, such as, for example, a memory device/module 1816. The computer readable medium or the memory module 1816 may be operatively coupled to the other components of the apparatus 1800 via the bus 1812 or the like. The memory module 1816 may be adapted to store computer readable instructions and data for effecting the processes and behavior of the modules 1802-1806, and subcomponents thereof, or the processor 1818, or the method 1300 and similar client-side methods disclosed herein. The memory module 1816 may retain instructions for executing functions associated with the modules 1802- 1806. While shown as being external to the memory 1816, it is to be understood that the modules 1802-1806 can exist within the memory 1816.

[0096] Fig. 19 is a flow chart illustrating aspects of a method 1900 by a server that services the synchronizing and display of an interactive GUI-enabled data object on a the client-side device. The method 1900 method may include receiving 1910, via a user interface of a server enabling user input, information related to an object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location. For example, the server may transmit a user interface object to a terminal, in response to a receiving a request direct to a Uniform Resource Locator (URL) of the server. The user interface object may include code that executes on the terminal in response to user selection of a user interface element of a web page that is accessed on the terminal via a web browser. The code may include one or more modules for describing or identifying the object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location in response to user input to the terminal, and relating the object to a portion (e.g., time line segment) of a particular audio-video work or to a geographic location.

[0097] The server-side method 1900 may further include recording 1920, in a data structure accessible over a computer network, the information in association with an identifier for the portion of the recorded audio-video work or for the geographic location. For example, the server may transmit the information and an identifier for the audio-video work or location received from the terminal to a database server, with instructions for saving as related records in a database. The recording operation may further include associating the information with a category indicator based on a signal received from a client device, which is interpreted as indicating user selection of a category to which the card should be assigned.

[0098] The method 1900 may further include receiving, via the user interface, further information for controlling generation of an interactive GUI-enabled data object containing the information for display on a display screen. The information may include, for example, format information, image data including an image of the object, text data describing the object, and a set of user interface controls enabling a user to control disposition of the object.

[0099] The server-side method 1900 may include tracking 1930 requests for the information from client devices that identify the portion of the recorded audio- video work or the geographic location in connection with the requests. The tracking may include, for example, saving, in a computer memory, user identities each associated with ones of the requests. The method may further include tracking additional information requests from the client devices that indicate a user interest in at least one of purchasing the object or obtaining further information about the object. The method may further include comprising tracking distribution of information based on one or more category indicators designated by end consumers of the information, or tracking distribution of information based on one or more social groups designated by end consumers of the information. The method may include crediting user accounts, for example with reward points, based on at least one of a number or a frequency that of the requests per user. The method may include determining an advertising value for distribution of the information based on a quantity of at least one of the requests or the additional information requests within a define time period.

[00100] Fig. 20 is a block diagram illustrating aspects of a server-side apparatus for performing a method as shown in Fig. 19, or variations there on. The apparatus 2000 may comprise an electrical component or module 2002 for receiving information related to an object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location. The apparatus 2000 may comprise an electrical component or module 2004 for recording the information in association with an identifier for the portion of the recorded audio-video work or for the geographic location. The apparatus 2000 may comprise an electrical component or module 2006 for tracking requests for the information from client devices that identify the portion of the recorded audio- video work or the geographic location in connection with the requests.

[00101] With reference to Fig. 20, the apparatus 2000 may optionally include a processor module 2020 having at least one processor, in the case of the apparatus 2000 configured as server core. The processor 2020, in such case, may be in operative communication with the modules 2002-2006 via a bus 2012 or similar communication coupling. The processor 2020 may effect initiation and scheduling of the processes or functions performed by electrical components 2002-2006.

[00102] In related aspects, the apparatus 2000 may include a network interface 2014, in communication with a sensor data analysis process 2015 that may be operating on the processor 2010 or in a separate server. The process 2015 may receive raw or processed sensor data from at least one sensor 2013, for example a microphone. In further related aspects, the apparatus 2000 may optionally include a module for storing information, such as, for example, a memory device/module 2016. The computer readable medium or the memory module 2016 may be operatively coupled to the other components of the apparatus 2000 via the bus 2012 or the like. The memory module 2016 may be adapted to store computer readable instructions and data for effecting the processes and behavior of the modules 2002-2006, and subcomponents thereof, or the processor 2020, or the method 1900 or similar server-side methods disclosed herein. The memory module 2016 may retain instructions for executing functions associated with the modules 2002-2006. While shown as being external to the memory 2016, it is to be understood that the modules 2002-2006 can exist within the memory 2016.

[00103] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[00104] As used in this application, the terms "component", "module", "system", and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.

[00105] Various aspects will be presented in terms of systems that may include a number of components, modules, and the like. It is to be understood and appreciated that the various systems may include additional components, modules, etc. and/or may not include all of the components, modules, etc. discussed in connection with the figures. A combination of these approaches may also be used. The various aspects disclosed herein can be performed on electrical devices including devices that utilize touch screen display technologies and/or mouse-and-keyboard type interfaces. Examples of such devices include computers (desktop and mobile), smart phones, personal digital assistants (PDAs), and other electronic devices both wired and wireless.

[00106] In addition, the various illustrative logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[00107] Furthermore, the one or more versions may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed aspects. The term "article of manufacture" (or alternatively, "computer program product") as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. For example, non-transitory computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips...), optical disks (e.g., compact disk (CD), digital versatile disk (DVD)...), smart cards, and flash memory devices (e.g., card, stick). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope of the disclosed aspects.

[00108] The steps of a method or algorithm described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

[00109] The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[00110] In view of the exemplary systems described supra, methodologies that may be implemented in accordance with the disclosed subject matter have been described with reference to several flow diagrams. While for purposes of simplicity of explanation, the methodologies are shown and described as a series of blocks, it is to be understood and appreciated that the claimed subject matter is not limited by the order of the blocks, as some blocks may occur in different orders and/or concurrently with other blocks from what is depicted and described herein. Moreover, not all illustrated blocks may be required to implement the methodologies described herein. Additionally, it should be further appreciated that the methodologies disclosed herein are capable of being stored on an article of manufacture to facilitate transporting and transferring such methodologies to computers. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device, carrier, or media.

Claims

1 . A method for providing information to a user, comprising:

generating, by a client device, a signal sample based on at least one of an audio or video signal output by a player device that is playing a recorded audio-video work, wherein the player device is distinct from the client device;

obtaining, using the client device, information relating to an object that is imaged in a portion of the recorded audio-video work from which the signal sample is generated, based on the signal sample; and

outputting the information in an interactive user interface of the client device, in response to obtaining the data record, while the audio-video work is playing on the player device.

2. The method of claim 1 , wherein obtaining the information further comprises identifying, using the client device, the recorded audio-video work and the portion thereof from which the signal sample was generated, based on the signal sample, and locating the information based on an identity of the recorded audio-video work and the portion.

3. The method of claim 1 , wherein obtaining the information comprises sending a representation of the signal sample to a server over a computer network, and receiving the data record from the server.

4. The method of claim 1 , wherein outputting the information further comprises displaying an interactive GUI-enabled data object containing the information on a display screen of the client device.

5. The method of claim 4, further comprising removing the interactive GUI-enabled data object containing the information from the display screen upon the earlier of: expiration of a designated period of time since initiating the displaying, or receiving later information relating to a different object that is imaged in a subsequent portion of the recorded audio-video work.

6. The method of claim 4, further comprising saving, in a computer memory, an identifier for the information in association with a user identity of a user of the client device.

7. The method of claim 6, further comprising saving an indication of affirmative user interest in association with the information, in response to receiving user input via the interactive GUI-enabled data object.

8. The method of claim 7, further comprising tracking indications of affirmative user interest in additional interactive GUI-enabled data objects for different objects, and crediting a user account based on at least one of a number or a frequency that the user provides the indications of affirmative user interest.

9. The method of claim 6, further comprising displaying a user interface screen for purchase of the object by the user, in response to user selection of an interface control object for initiating a purchase transaction.

10. The method of claim 6, wherein the saving further comprises associating the identifier with a category indicator indicated by the user.

1 1 . The method of claim 10, further comprising displaying a list of eligible information categories on the display screen configured for enabling user selection of the category indicator.

12. The method of claim 1 1 , further comprising providing a user interface enabling the user to designate at least one of the information categories for publication to a designated social group as a category of special interest to the user.

13. The method of claim 10, further comprising, in response to user selection of a category on the list, obtaining and displaying a collection of card like data objects each previously categorized under the category by the user.

14. The method of claim 10, further comprising, in response to user selection of a category on the list, obtaining and displaying a collection of card like data objects each previously categorized under the category by other users who have designated the category as shared with the user.

15. The method of claim 10, further comprising displaying a ranking of categories based on activity across a group of users.

16. The method of claim 10, wherein the saving further comprises sending the information and category location to the computer memory located in a network location in association with an account identifier enabling access to the information and category using other client devices.

17. The method of claim 6, further comprising obtaining, using the client device, object information related to an object selected based on a current location of the client device, and outputting the information at least in part by displaying another interactive GUI-enabled data object containing the object information on the display screen.

18. An apparatus, comprising:

a processor, a memory coupled to the processor, and a signal sampler coupled to the processor, wherein the memory holds instructions that when executed by the processor, cause the apparatus perform the operations of:

generating a signal sample based on at least one of an audio or video signal output by a player device that is playing a recorded audio-video work, wherein the player device is distinct from the client device;

obtaining information relating to an object that is imaged in a portion of the recorded audio-video work from which the signal sample is generated, based on the signal sample; and

outputting the information in an interactive user interface, in response to obtaining the data record, while the audio-video work is playing on the player device.

19. The apparatus of claim 18, wherein the memory holds further instruction for obtaining the information at least in part by identifying the recorded audio-video work and the portion thereof from which the signal sample was generated, based on the signal sample, and locating the information based on an identity of the recorded audio-video work and the portion.

20. The apparatus of claim 18, wherein the memory holds further instruction for obtaining the information at least in part by sending a representation of the signal sample to a server over a computer network, and receiving the data record from the server.

21 . The apparatus of claim 18, wherein the memory holds further instruction for outputting the information at least in part by displaying an interactive GUI-enabled data object containing the information on a display screen of the client device.

22. The apparatus of claim 21 , wherein the memory holds further instruction for removing the interactive GUI-enabled data object containing the information from the display screen upon the earlier of: expiration of a designated period of time since initiating the displaying, or receiving later information relating to a different object that is imaged in a subsequent portion of the recorded audio- video work.

23. The apparatus of claim 21 , wherein the memory holds further instruction for saving, in a computer memory, an identifier for the information in association with a user identity of a user of the client device.

24. The apparatus of claim 23, wherein the memory holds further instruction for saving an indication of affirmative user interest in association with the information, in response to receiving user input via the interactive GUI- enabled data object.

25. The apparatus of claim 24, wherein the memory holds further instruction for comprising tracking indications of affirmative user interest in additional interactive GUI-enabled data objects for different objects, and crediting a user account based on at least one of a number or a frequency that the user provides the indications of affirmative user interest.

26. The apparatus of claim 23, wherein the memory holds further instruction for displaying a user interface screen for purchase of the object by the user, in response to user selection of an interface control object for initiating a purchase transaction.

27. The apparatus of claim 18, wherein the memory holds further instruction for the saving at least in part by associating the identifier with a category indicator indicated by the user.

28. The apparatus of claim 27, wherein the memory holds further instruction for displaying a list of eligible information categories on the display screen configured for enabling user selection of the category indicator.

29. The apparatus of claim 28, wherein the memory holds further instruction for providing a user interface enabling the user to designate at least one of the information categories for publication to a designated social group as a category of special interest to the user.

30. The apparatus of claim 27, wherein the memory holds further instruction for obtaining and displaying a collection of card like data objects each previously categorized under the category by the user, in response to user selection of a category on the list.

31 . The apparatus of claim 27, wherein the memory holds further instruction for obtaining and displaying a collection of card like data objects each previously categorized under the category by other users who have designated the category as shared with the user, in response to user selection of a category on the list.

32. The apparatus of claim 27, wherein the memory holds further instruction for displaying a ranking of categories based on activity across a group of users.

33. The apparatus of claim 27, wherein the memory holds further instruction for the saving at least in part by sending the information and category location to the computer memory located in a network location in association with an account identifier enabling access to the information and category using other client devices.

34. The apparatus of claim 23, wherein the memory holds further instruction for obtaining object information related to an object selected based on a current location of the client device, and outputting the information at least in part by displaying another interactive GUI-enabled data object containing the object information on the display screen.

35. A computer-readable medium, encoded with instructions that, when executed by a processor, cause a client device to perform the operations of :

36. An apparatus comprising:

means for generating a signal sample based on at least one of an audio or video signal output by a player device that is playing a recorded audio- video work, wherein the player device is distinct from the client device;

means for obtaining information relating to an object that is imaged in a portion of the recorded audio-video work from which the signal sample is generated, based on the signal sample; and

means for outputting the information in an interactive user interface, in response to obtaining the data record, while the audio-video work is playing on the player device.

37. A method for enabling content-related provision of information, comprising:

receiving, via a user interface of a server enabling user input, information related to an object that is imaged in a portion of a recorded audio- video work or that is related to an identified geographic location;

recording, in a data structure accessible over a computer network, the information in association with an identifier for the portion of the recorded audio-video work or for the geographic location; and

tracking requests for the information from client devices that identify the portion of the recorded audio-video work or the geographic location in connection with the requests.

38. The method of claim 37, further comprising receiving, via the user interface, further information for controlling generation of an interactive GUI- enabled data object containing the information for display on a display screen.

39. The method of claim 37, wherein the tracking further comprises saving, in a computer memory, user identities each associated with ones of the requests.

40. The method of claim 37, further comprising crediting user accounts based on at least one of a number or a frequency of the requests per user.

41 . The method of claim 37, further comprising tracking additional information requests from the client devices that indicate a user interest in at least one of purchasing the object or obtaining further information about the object.

42. The method of claim 41 , further comprising tracking an advertising value for distribution of the information based on at least one of the requests or the additional information requests.

43. The method of claim 37, wherein the recording further comprises associating the information with a category indicator.

44. The method of claim 43, further comprising tracking distribution of information based on one or more category indicators designated by end consumers of the information.

45. The method of claim 44, further comprising tracking distribution of information based on one or more social groups designated by end consumers of the information.

46. An apparatus, comprising:

a processor, a memory coupled to the processor, and a network interface coupled to the processor, wherein the memory holds instructions that when executed by the processor, cause the apparatus perform the operations of:

47. The apparatus of claim 46, wherein the memory holds further instruction for receiving, via the user interface, further information for controlling generation of an interactive GUI-enabled data object containing the information for display on a display screen.

48. The apparatus of claim 46, wherein the memory holds further instruction for the tracking at least in part by saving, in a computer memory, user identities each associated with ones of the requests.

49. The apparatus of claim 46, wherein the memory holds further instruction for crediting user accounts based on at least one of a number or a frequency of the requests per user.

50. The apparatus of claim 46, wherein the memory holds further instruction for tracking additional information requests from the client devices that indicate a user interest in at least one of purchasing the object or obtaining further information about the object.

51 . The apparatus of claim 50, wherein the memory holds further instruction for tracking an advertising value for distribution of the information based on at least one of the requests or the additional information requests.

52. The apparatus of claim 46, wherein the memory holds further instruction for the recording further comprises associating the information with a category indicator.

53. The apparatus of claim 52, wherein the memory holds further instruction for tracking distribution of information based on one or more category indicators designated by end consumers of the information.

54. The apparatus of claim 53, wherein the memory holds further instruction for tracking distribution of information based on one or more social groups designated by end consumers of the information.

55. A computer-readable medium, encoded with instructions that, when executed by a processor, cause a client device to perform:

56. An apparatus, comprising:

means for receiving, via a user interface of a server enabling user input, information related to an object that is imaged in a portion of a recorded audio-video work or that is related to an identified geographic location;

means for recording, in a data structure accessible over a computer network, the information in association with an identifier for the portion of the recorded audio-video work or for the geographic location; and

means for tracking requests for the information from client devices that identify the portion of the recorded audio-video work or the geographic location in connection with the requests.