WO2015100070A1 - Présentation d'informations sur la base d'une vidéo - Google Patents
Présentation d'informations sur la base d'une vidéo Download PDFInfo
- Publication number
- WO2015100070A1 WO2015100070A1 PCT/US2014/070580 US2014070580W WO2015100070A1 WO 2015100070 A1 WO2015100070 A1 WO 2015100070A1 US 2014070580 W US2014070580 W US 2014070580W WO 2015100070 A1 WO2015100070 A1 WO 2015100070A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video
- information
- feature information
- prompt
- database
- Prior art date
Links
- 230000004044 response Effects 0.000 claims abstract description 24
- 238000003860 storage Methods 0.000 claims description 66
- 238000000034 method Methods 0.000 claims description 60
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 41
- 238000010586 diagram Methods 0.000 description 19
- 238000000605 extraction Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 7
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000004557 technical material Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2665—Gathering content from different sources, e.g. Internet and satellite
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
Definitions
- the present application relates to smart television technology. More specifically, the present application relates to presenting information based on a video playing at a smart television.
- smart television sets are being designed with greater capabilities.
- smart televisions are configured to have Internet features and are also sometimes capable of cross-platform searches between a television, the Internet, and computer programs. Users can now access information they need via a smart television.
- conventional smart television is unable to generate prompts in real-time for a user and/or provide recommendation information relating to the content that is currently playing at the television, yet this information may be precisely the information that the user is interested in.
- FIG. 1 is a diagram showing an embodiment of a system for presenting information based on a video.
- FIG. 2 is a flow diagram showing an embodiment of a process for presenting information based on a video.
- FIG. 3 is a flow diagram showing an example of a process for presenting
- FIG. 4 is a flow diagram showing an example of a process for presenting
- FIG. 5 is a flow diagram showing an example of a process for presenting
- FIG. 6 is a diagram showing an embodiment of a system for presenting information based on a video.
- FIG. 7 is a diagram showing an embodiment of a system for presenting information based on a video.
- FIG. 8 is a diagram showing an embodiment of a system for presenting information based on a video.
- FIG. 9 is a functional diagram illustrating an embodiment of a programmed computer system for implementing presenting information based on a video.
- the invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor.
- these implementations, or any other form that the invention may take, may be referred to as techniques.
- the order of the steps of disclosed processes may be altered within the scope of the invention.
- a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task.
- the term 'processor' refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
- a video is currently playing at a smart television.
- a "smart television” comprises a television set that is configured to communicate over a network (e.g., the Internet).
- a set of feature information is extracted from one or more images associated with the currently playing video.
- the set of feature information is determined to match a set of video feature information stored at a video database.
- the set of video feature information corresponds to a set of identifying information associated with a video.
- a prompt is generated based at least in part on the set of identifying information associated with the video that
- the search results may comprise merchandise information, such as product information.
- FIG. 1 is a diagram showing an embodiment of a system for presenting information based on a video.
- system 100 includes smart television 102, device 104, network 106, and cloud server 108.
- Network 106 includes one or more of high-speed data networks and/or telecommunications networks.
- Cloud server 108 is configured to access video storage 110, video database 112, and information database 114.
- video storage 110, video database 112, and information database 114 may be configured as one or more storages.
- Smart television 102 is configured to communicate to other entities over network
- a video is currently playing at smart television 102.
- the video playing at smart television 102 is also stored at video storage 110.
- Smart television 102 is configured to enable more information to be provided based on the content of the video by first capturing one or more frames of the video and then extracting a set of feature information from the captured video frames.
- the set of feature information comprises values corresponding to predetermined features or attributes of the video frames.
- smart television 102 is configured to query cloud server 108 to compare the set of feature information to sets of video feature information stored at video database 112. For example, each set of video feature information stored at video database 112 corresponds to a video storage stored at video storage 110.
- smart television 102 is configured to generate a prompt determined based on a set of identifying information corresponding to the matching set of video feature information.
- the set of identifying information corresponding to the matching set of video feature information is also stored at video database 112 and is configured to include information that corresponds to the video stored at video storage 110 from which the matching set of video feature information was extracted.
- device 104 may use device 104 to interact with the prompt displayed at smart television 102 in various manners.
- device 104 comprises a remote control device that is configured to transmit data to smart television 102 and may or may not be configured to also communicate over network 106.
- device 104 comprises a mobile device that includes a camera function and is also configured to communicate over network 106.
- a user may use device 104 to communicate to smart television 102 by inputting data at device 104 to cause device 104 to send a response associated with the prompt displayed at smart television 102, which will cause smart television 102 to query cloud server 108 to perform a search based on the set of identifying information stored at information database 114.
- information database 114 stores (e.g., merchandise) information that corresponds to each video stored at video storage 110. The search result can then be displayed at smart television 102.
- FIG. 2 is a flow diagram showing an embodiment of a process for presenting information based on a video.
- process 200 is implemented at system 100 of FIG. 1.
- a set of feature information is extracted from one or more images associated with a currently playing video.
- One or more frames or images from a video that is currently playing are extracted.
- the video is currently playing at a smart television.
- the video frames are captured through a (e.g., high-definition) video capturing card.
- the captured video frames are pre-processed before feature values are extracted from them.
- the captured video frames may be resized and/or cropped during preprocessing.
- feature extraction is a technique that maps input information to a reduced set of information (i.e., features, which can be represented by mathematical vectors, for example) such that the input information can be accurately recognized or classified based on the reduced representation of features.
- a feature is a variable that is used to represent a characteristic of the input information.
- Features are selected and defined by designers of a feature extraction and are processed to help decode/classify the input information, distinguish/disambiguate the input information, and/or accurately map the input information to the output values.
- feature extraction is used to extract information from the one or more frames captured from a currently playing video that can be used to identify the video frames and/or the video itself.
- a set of features is predetermined.
- the predetermined set of features includes visual information, audio information, or a combination of visual and audio information.
- the set of features may include one or more of the following: color features, position features, binary image features, speeded up robust features (SURF), and audio features.
- a set of feature information e.g., values
- Any type of appropriate feature extraction technique may be used to extract the feature values.
- the feature extraction of step 202 is triggered by an event.
- the video being paused may be an event that triggers features to be extracted from the frame on which the video was paused.
- advertisement video may be an event that triggers features to be extracted from the currently playing advertisement video.
- the feature extraction of step 202 is performed periodically (e.g., every 15 minutes of video playback).
- the set of feature information extracted from the one or more frames is feature information that can be used to identify the video frames.
- the set of feature information is used to determine whether the one or more frames are from a video that is stored at a video storage.
- the video storage may comprise a storage for advertisement videos.
- the set of feature information is determined to match a set of video feature information stored at a video database, wherein the set of video feature information corresponds to a set of identifying information associated with a video.
- a video database is maintained.
- the video database stores a set of video feature information extracted from, and therefore corresponding to, each video stored in a video storage.
- the video storage is also maintained.
- the video storage includes various video files from which features have been extracted and added to the video database.
- the video files of the video storage may be provided by one or more content providers.
- the set of video feature information corresponding to each video includes feature values corresponding to at least some of the same predetermined set of features that were extracted from the video frames of step 202.
- the set of video feature information corresponding to each video includes feature information extracted from a potentially larger number of video frames than the number of frames from which feature information was extracted in step 202.
- the video database is configured in a cloud server.
- a set of identifying information corresponding to the video may be stored as well in the video database.
- the set of identifying information corresponding to each video may include keywords that are associated with the video, the title/name of the video, and/or other metadata associated with the video.
- an information database is also maintained.
- the information database stores at least a set of information corresponding to each video stored in the video storage.
- At least one of the video storage, the video database, and the information database is associated with a cloud server.
- Table 1 is an example of the type of content that is stored at each of a video storage, a video database, and an information database.
- the video storage stores video files
- the video database stores a set of video feature information and a set of identifying information corresponding to each video file in the video storage
- the information database stores a set of information
- the video storage comprises an advertising video storage that stores advertising videos.
- the video database comprises an advertising video database that stores a set of video feature information that is extracted from each
- the information database comprises an advertising information database that stores a set of
- the set of merchandise information may include information associated with products and/or links to webpages associated with products that are related to the content of the corresponding video.
- Table 2 is an example of the type of content that is stored at each of an advertising video storage, an advertising video database, and an advertising information database.
- the advertising video storage stores advertising video files
- the advertising video database stores a set of video feature information and a set of identifying information corresponding to each advertising video file in the advertising video storage
- the advertising information database stores a set of merchandise information corresponding to each advertising video file in the advertising video storage.
- the set of feature information extracted from the video frames is compared against the sets of video feature information that are stored at the video database (e.g., the advertising video database). It is determined whether the set of feature information extracted from the video frames matches a set of video feature information that is stored at the video database.
- the video database e.g., the advertising video database
- the set of feature information extracted from the video frames can be matched to a set of video feature information that is stored at the video database through either fuzzy matching or exact matching.
- the set of identifying information associated with the matching set of video feature information is obtained from the video database.
- each set of video feature information that is stored at the video database corresponds to a video file stored at the video storage and also a set of identifying information corresponding to that same video file.
- the set of identifying information identifies the video that is currently playing.
- process 200 ends.
- a prompt is generated based at least in part on the set of identifying information associated with the video that corresponds to the set of video feature information stored at the video database.
- a prompt is generated based on the set of identifying information from the video database corresponding to the matching set of video feature information.
- the prompt includes the set of identifying information associated with the video.
- the prompt may include text and/or images, for example.
- the prompt may include text that asks a user whether he or she would like to receive further information associated with the video that is currently playing and/or the set of identifying information.
- the prompt includes at least a first control that the user may select to receive further information.
- the prompt includes a second control that the user may select to dismiss the prompt.
- the prompt does not include a control and comprises a code (e.g., a Quick Response (QR) code) that is configured with information associated with the set of identifying information associated with the video.
- a code e.g., a Quick Response (QR) code
- QR Quick Response
- the prompt may be displayed at the same screen at which the video is currently playing. In various embodiments, the prompt is displayed at the smart television. In some embodiments, the prompt may be displayed at a different screen than the screen at which the video is currently playing.
- a search is performed at an information database based at least in part on the set of identifying information associated with the video.
- a search is performed at the information database based at least in part on the set of identifying information associated with the video.
- the information database comprises a merchandise information database and a set of merchandise information is found based on the set of identifying information associated with the video.
- process 200 ends.
- a selection associated with the prompt to receive more information is performed using a device.
- the prompt may be displayed at a smart television display and the selection may be made by a user using a remote device and/or a mobile device.
- the results of the search based on the set of identifying information associated with the video are presented.
- the search results are displayed at the same screen at which the video is currently playing.
- the search results may be displayed at a different screen than the screen at which the video is currently playing.
- an advertising video may be currently playing at a smart television.
- One or more frames of the currently playing advertising video are captured and a set of feature information is extracted from the video frames by the smart television. If it is determined that the set of feature information associated with the video frames matches a set of video feature information stored at an advertising video database, then the set of identifying information associated with the corresponding advertising video file is obtained from the advertising video database.
- a prompt can be generated based on the set of identifying information and displayed at the smart television screen.
- the prompt includes a control (e.g., button) that a user can select to receive more information.
- the user may respond to this prompt by selecting the button that is associated with the prompt using a remote control device that is configured to transmit information to the smart television.
- the smart television will search for the advertising information database corresponding to the set of identifying information in an advertising information database and display the search results at the display screen of the smart television.
- each search result may correspond to a product that matches the set of identifying information associated with the video.
- the user may continue to engage in data exchange with the smart television using the remote control device and may even purchase a displayed product.
- the user may scroll through the search results that are presented at the display screen of the television by using the (e.g., hard) buttons of the remote control device. The user may be prompted by the smart television to log into his or account at a shopping platform associated with at least some products of the presented search results.
- the user may select one or more search results presented at the smart television via the remote control device to add the products associated with the search results into a shopping cart associated with the user's account at the shopping platform.
- the user may follow through with purchasing the products in the shopping cart associated with his or her account at the shopping platform at a later time (e.g., using a mobile device or desktop device).
- the user may select one or more search results presented at the smart television via the remote control device to directly purchase the products associated with the search results.
- the user may directly purchase the products by inputting his or her credit card information through one or more purchase interfaces presented at the smart television using the remote control device.
- the user may respond to this prompt by using a mobile device to capture the information associated with the prompt information.
- prompt information can be displayed in the form of a QR code and the user may select the prompt by scanning the QR code with a mobile device with a scanning or camera function.
- an application executing at the mobile device that is configured to read QR codes may use the content of the QR code (the set of identifying information associated with the video) to perform a search at the advertising information database and display search results at the display screen of the mobile device.
- the user may select a search result that is displayed at the display screen of the mobile device via a touchscreen or other input mechanism of the mobile device.
- the user may be prompted by the smart television to log into his or account at a shopping platform associated with at least some products of the presented search results.
- the user may select one or more search results presented at the mobile device via an input mechanism of the mobile device to add the products associated with the search results into a shopping cart associated with the user's account at the shopping platform.
- the user may follow through with purchasing the products in the shopping cart associated with his or her account at the shopping platform at a later time (e.g., using the mobile device or desktop device).
- the user may select one or more search results presented at the mobile device via an input mechanism of the mobile device to directly purchase the products associated with the search results.
- the user may directly purchase the products by inputting his or her credit card information through one or more purchase interfaces presented at the mobile device.
- the user may select one or more search results presented at the mobile device via an input mechanism of the mobile device to view additional information associated with the products associated with the search results.
- a smart television e.g., a smart television and a separate device (e.g., a remote television control or a mobile device).
- a separate device e.g., a remote television control or a mobile device.
- FIG. 3 is a flow diagram showing an example of a process for presenting information based on a video.
- process 300 is implemented at system 100 of FIG. 1.
- process 200 of FIG. 2 is implemented at least in part by process 300.
- Process 300 is implemented by a smart television such as smart television 102 of system 100 of FIG. 1.
- a set of feature information is extracted by a smart television from one or more images associated with a currently playing video.
- the video is currently playing at the smart television.
- the set of feature information is compared by the smart television to sets of video feature information stored at a video database.
- the set of feature information that is extracted from the video frames is compared to the sets of video feature information stored at a video database. It is determined whether the set of feature information matches any set of video feature information that is stored at the video database.
- the video database is associated with a cloud server.
- At 306 it is determined by the smart television whether the set of feature information has successfully matched a set of video feature information stored at the video database, wherein the set of video feature information corresponds to a set of identifying information associated with a video. In the event that a set of video feature information stored at the video database has been determined to successfully match the set of feature information extracted from the video frames, control is transferred to 308. Otherwise, in the event that a set of video feature information stored at the video database has not been determined to successfully match the set of feature information extracted from the video frames, process 300 ends.
- a prompt is generated by the smart television based at least in part on the set of identifying information associated with the video that corresponds to the set of video feature information stored at the video database.
- the prompt is displayed on the smart television.
- the prompt can appear in the form of text, numbers, pictures, or a combination thereof on the smart television screen.
- the prompt is displayed at the smart television.
- the device may comprise a remote control device that is configured to transmit data to the smart television. For example, a user may make a selection associated with (e.g., a control displayed in) the prompt that is displayed at the smart television screen by pressing a button on the remote control device.
- a search is performed by the smart television at an information database based at least in part on the set of identifying information associated with the video, wherein the selection is received from a device.
- the information database is associated with a cloud server.
- search results are displayed at the smart television.
- the smart television displays the search results on its screen, which may comprise merchandise information that is determined to relate to the set of identifying information associated with the currently playing video. If the user is interested in the displayed merchandise information, he or she may use the remote control device to further interact with the merchandise information, such as by requesting more information on a product, purchasing a product, and/or adding a product to a shopping cart.
- the user may interact with the search results and select one or more search results by transmitting information to the smart television via the remote control device. For example, the user may scroll through the search results that are presented at the display screen of the television by using the (e.g., hard) buttons of the remote control device.
- the user prior to or subsequent to a user selecting a search result that is presented at the smart television, the user is prompted by a login screen displayed at the smart television to log into his or her account associated with a shopping platform associated with selling at least some of the products among the presented search results.
- the user may select one or more search results presented at the smart television via the remote control device to add the products associated with the search results into a shopping cart associated with the user's account at the shopping platform.
- the user may follow through with purchasing the products in the shopping cart associated with his or her account at the shopping platform at a later time (e.g., using a mobile device or desktop device).
- the user may select one or more search results presented at the smart television via the remote control device to directly purchase the products associated with the search results.
- the user may directly purchase the products by inputting his or her credit card information through one or more purchase interfaces presented at the smart television using the remote control device.
- the user may select one or more search results presented at the smart television via the remote control device to view additional information associated with the products associated with the search results.
- FIG. 4 is a flow diagram showing an example of a process for presenting
- process 400 is implemented at system 100 of FIG. 1.
- process 200 of FIG. 2 is implemented at least in part by process 400.
- Process 400 is implemented by a smart television such as smart television 102 of system 100 of FIG. 1 and also a separate device such as device 104 of system 100 of FIG. 1.
- the device comprises a mobile device that is configured to access the Internet and includes a camera function.
- Examples of a mobile device include a smart phone, a tablet device, or any other computing device.
- a set of feature information is extracted by a smart television from one or more images associated with a currently playing video.
- the video is currently playing at the smart television.
- the set of feature information is compared by the smart television to sets of video feature information stored at a video database.
- the set of feature information that is extracted from the video frames is compared to the sets of video feature information stored at a video database. It is determined whether the set of feature information matches any set of video feature information that is stored at the video database.
- the video database is associated with a cloud server.
- the smart television determines by the smart television whether the set of feature information has successfully matched a set of video feature information stored at the video database, wherein the set of video feature information corresponds to a set of identifying information associated with a video.
- control is transferred to 408. Otherwise, in the event that a set of video feature information stored at the video database has not been determined to successfully match the set of feature information extracted from the video frames, process 400 ends.
- a prompt is generated by the smart television based at least in part on the set of identifying information associated with the video that corresponds to the set of video feature information stored at the video database.
- the prompt is displayed on the smart television.
- the prompt can appear in the form of text, numbers, pictures, or a combination thereof on the smart television screen.
- the prompt comprises a QR code.
- the prompt is presented at the smart television.
- the device may comprise a mobile device that is configured to access the Internet and includes a camera function.
- the set of identifying information associated with the video is obtained by a device based at least in part on the prompt.
- the device can obtain the set of identifying information associated with the video by taking a photo of and/or making a scan of the QR code displayed at the smart television screen.
- a search is performed by the device at an information database based at least in part on the set of identifying information associated with the video.
- An application executing at the mobile device can read the scanned QR code and determine the set of identifying information associated with the video. Furthermore, the application executing at the mobile device can also perform a search using the set of identifying information associated with the video at an information database.
- the information database is associated with a cloud server.
- search results are presented at the device.
- the search results obtained by the mobile device can be displayed at a screen of the device itself. If the user is interested in the displayed merchandise information, he or she can use the device to further interact with the merchandise information, such as by requesting more information on a product, purchasing a product, and/or adding a product to a shopping cart. In some embodiments, the user may respond to this prompt by using a mobile device to capture the information associated with the prompt information.
- prompt information can be displayed in the form of a QR code and the user may select the prompt by scanning the QR code with a mobile device with a scanning or camera function.
- an application executing at the mobile device that is configured to read QR codes may use the content of the QR code (the set of identifying information associated with the video) to perform a search at the advertising information database and display search results at the display screen of the mobile device.
- the user may select a search result that is displayed at the display screen of the mobile device via a touchscreen or other input mechanism of the mobile device.
- the user may be prompted by the smart television to log into his or account at a shopping platform associated with at least some products of the presented search results.
- the user may select one or more search results presented at the mobile device via an input mechanism of the mobile device to add the products associated with the search results into a shopping cart associated with the user's account at the shopping platform. The user may follow through with purchasing the products in the shopping cart associated with his or her account at the shopping platform at a later time (e.g., using the mobile device or desktop device).
- the user may select one or more search results presented at the mobile device via an input mechanism of the mobile device to directly purchase the products associated with the search results. The user may directly purchase the products by inputting his or her credit card information through one or more purchase interfaces presented at the mobile device.
- FIG. 5 is a flow diagram showing an example of a process for presenting
- process 500 is implemented at system 100 of FIG. 1.
- process 200 of FIG. 2 is implemented at least in part by process 500.
- Process 500 is implemented by a smart television such as smart television 102 of system 100 of FIG. 1 and also a separate device such as device 104 of system 100 of FIG. 1.
- the device comprises a mobile device that is configured to access the Internet and includes a camera function.
- Examples of a mobile device include a smart phone, a tablet device, or any other computing device.
- process 500 after a first set of feature information extracted by the smart television from a first set of video frames is determined to match a set of video feature information stored in the video database, the device performs its own extraction of a second set of feature information from a second set of video frames and performs its own comparison of the second set of feature information with sets of video feature information stored at the video database.
- a first set of feature information is extracted by a smart television from a first set of images associated with a currently playing video.
- the video is currently playing at the smart television.
- the first set of feature information is compared by the smart television to sets of video feature information stored at a video database.
- the set of feature information that is extracted by the smart television from the video frames captured by the smart television is compared to the sets of video feature information stored at a video database. It is determined whether the set of feature information extracted by the smart television matches any set of video feature information that is stored at the video database.
- the video database is associated with a cloud server.
- a prompt is generated by the smart television based at least in part on the first set of identifying information associated with the video that corresponds to the first set of video feature information stored at the video database.
- the prompt is displayed on the smart television.
- the prompt can appear in the form of text, numbers, pictures, or a combination thereof on the smart television screen.
- a second set of feature information is extracted by a device from a second set of images associated with the currently playing video.
- the device may comprise a mobile device that is configured to access the Internet and includes a camera function.
- the prompt comprises a set of instructions that instructs the user to take a photo or video of the video that is currently playing at the smart television.
- the mobile device that is separate from the smart television is configured to extract its own set of feature information from the frames of the currently playing video that were captured by the mobile device itself.
- the one or more frames of the video that were captured by the mobile device may differ from the one or more frames of the video that were captured earlier by the smart television because the mobile device may have captured its frames at a later point in the playback of the video than the smart television.
- the set of feature information extracted by the mobile device from the video frames that the mobile device captured may differ from the set of feature information extracted by the smart television from the video frames that the smart television had captured.
- the second set of feature information is compared by the device to the sets of video feature information stored at the video database.
- the set of feature information that is extracted by the mobile device from the video frames captured by the mobile device is compared to the sets of video feature information stored at the video database.
- the video database used in the comparison by the mobile device is the same video database that was used in the comparison by the smart television. It is determined whether the set of feature information extracted by the mobile device matches any set of video feature information that is stored at the video database.
- the video database is associated with a cloud server.
- control may be transferred to 502, so that the smart television can again capture a new set of video frames and extract a set of feature information from this new set of video frames.
- both sets of feature information may be determined to match the same set of video feature information and therefore the same set of identifying information associated with a video that is stored at the video database.
- the set of video feature information from the video database that is matched by the set of feature information extracted by the smart television may be different from the set of video feature information from the video database that is matched by the set of feature information extracted by the mobile device, in which case the set of identifying information associated with a video as determined by the smart television would differ from the set of identifying information associated with a video as determined by the mobile device.
- the mobile device extract and compare its own set of feature information from the video currently playing at the smart television after the smart television has extracted and compared its respective set of feature information is that the extraction and/or matching techniques used by the mobile device may be updated and improved more frequently than the extraction and/or matching techniques used by the smart television (e.g., due to the different availability of opportunities to update firmware and software at the smart television and mobile device).
- a search is performed by the device at an information database based at least in part on the second set of identifying information associated with the video.
- An application executing at the mobile device can also perform a search using the set of identifying information associated with the video that was determined by the mobile device at the information database.
- search results are presented at the device.
- FIG. 6 is a diagram showing an embodiment of a system for presenting information based on a video.
- system 600 includes storage module 630, cloud database 650, extracting module 610, match processing module 620, and displaying module 640.
- system 600 is associated with and/or a part of a smart television.
- the modules can be implemented as software components executing on one or more processors, as hardware such as programmable logic devices, and/or Application Specific
- Integrated Circuits designed to elements can be embodied by a form of software products which can be stored in a nonvolatile storage medium (such as optical disk, flash storage device, mobile hard disk, etc.), including a number of instructions for making a computer device (such as personal computers, servers, network equipment, etc.) implement the methods described in the embodiments of the present invention.
- the modules may be implemented on a single device or distributed across multiple devices.
- Extracting module 610 is configured to extract a set of feature information from a currently playing video.
- the set of feature information may include audio feature information, video feature information, or audio and video feature information.
- Match processing module 620 is connected to extracting module 610 and is configured to compare the set of feature information to sets of video feature information stored at cloud database 650. If a matching set of video feature information can be found in cloud database 650, then match processing module 620 is configured to generate a prompt based on a set of identifying information associated with a video corresponding to the matching set of video feature information. In some embodiments, the set of identifying information associated with the video is also stored at cloud database 650.
- Displaying module 640 is for displaying the prompts generated by match processing module 620.
- Storage module 630 is configured to store (e.g., cache) at least some of the sets of video feature information and respective corresponding sets of identifying information that were previously obtained from cloud database 650.
- FIG. 7 is a diagram showing an embodiment of a system for presenting information based on a video.
- system 700 includes system 600 of FIG. 6. The modules of system 600 of FIG. 6 are not shown again in the diagram of FIG. 7.
- System 700 also includes receiving module 760 and looking up module 770.
- system 700 is associated with and/or a part of a smart television
- Receiving module 760 is configured to receive a selection from a device in response to a presentation of a prompt.
- the device comprises a remote control device.
- Searching module 770 is configured to search at an information database (e.g., that is part of cloud database 650 of system 600 of FIG. 6) for search results that match the set of identifying information associated with the video. For example, the search results can be displayed (e.g., that is part of displaying module 640 of system 600 of FIG. 6).
- searching module 770 is configured to present a login user interface associated with a shopping platform. For example, searching module 770 is configured to receive login credentials (e.g., username and password) input by a user (e.g., using a device) and send the login credentials to a server associated with the shopping platform. In some embodiments, searching module 770 is configured to receive a selection of a displayed search result.
- login credentials e.g., username and password
- searching module 770 is configured to receive a selection of a displayed search result.
- searching module 770 is configured to send an indication to the server associated with the shopping platform to add a product associated with the selected search result in a shopping cart associated with the user's logged in account at the shopping platform.
- searching module 770 is configured to present a payment information receiving interface associated with the shopping platform.
- searching module 770 is configured to receive payment information (e.g., credit card information) input by a user (e.g., using a device) and send the payment information to the server associated with the shopping platform to complete the purchase of the product associated with the selected search result.
- searching module 770 is configured to present additional information associated with a product associated with the selected search result.
- FIG. 8 is a diagram showing an embodiment of a system for presenting information based on a video.
- system 800 includes smart television 802 and device 804.
- smart television 802 can be implemented using system 600 of FIG. 6 or system 700 of FIG. 7 and will not be described further.
- Device 804 is configured to obtain a set of identifying information associated with a video that was determined by smart television 802 based on a prompt presented by smart television 802.
- Device 804 is configured to search in an information database (e.g., that is part of cloud database 650 of system 600 of FIG. 6) for information based on the set of identifying information and also display the found search results.
- device 804 is capable of accessing the Internet and also includes a camera function.
- device 804 is configured to capture a set of images from the video currently playing at smart television 802, extract a set of feature information from that set of images, determine a set of identifying information associated with a video based at least in part on comparing that set of feature information to sets of video feature information stored at a video database (e.g., that is part of cloud database 650 of system 600 of FIG. 6), and then perform a search at the information database based on that set of identifying information.
- a video database e.g., that is part of cloud database 650 of system 600 of FIG. 6
- device 804 is configured to present the found search results at screen associated with the mobile device.
- device 804 is configured to present a login user interface associated with a shopping platform.
- device 804 is configured to receive login credentials (e.g., username and password) input by a user (e.g., using a device) and send the login credentials to a server associated with the shopping platform.
- device 804 is configured to receive a selection of a displayed search result.
- device 804 is configured to send an indication to the server associated with the shopping platform to add a product associated with the selected search result in a shopping cart associated with the user's logged in account at the shopping platform.
- receiving the selection of the displayed search result in some embodiments, in some
- device 804 is configured to present a payment information receiving interface associated with the shopping platform.
- device 804 is configured to receive payment information (e.g., credit card information) input by a user (e.g., using a device) and send the payment information to the server associated with the shopping platform to complete the purchase of the product associated with the selected search result.
- device 804 is configured to present additional information associated with a product associated with the selected search result.
- FIG. 9 is a functional diagram illustrating an embodiment of a programmed computer system for implementing presenting information based on a video.
- Computer system 900 which includes various subsystems as described below, includes at least one microprocessor subsystem (also referred to as a processor or a central processing unit (CPU)) 902.
- processor 902 can be implemented by a single-chip processor or by multiple processors.
- processor 902 is a general purpose digital processor that controls the operation of the computer system 900.
- processor 902 controls the reception and manipulation of input data, and the output and display of data on output devices (e.g., display 918).
- processor 902 includes and/or is used to provide the presentation of information based on a video.
- Processor 902 is coupled bi-directionally with memory 910, which can include a first primary storage area, typically a random access memory (RAM), and a second primary storage area, typically a read-only memory (ROM).
- primary storage can be used as a general storage area and as scratch-pad memory, and can also be used to store input data and processed data.
- Primary storage can also store programming instructions and data, in the form of data objects and text objects, in addition to other data and instructions for processes operating on processor 902.
- primary storage typically includes basic operating instructions, program code, data, and objects used by the processor 902 to perform its functions (e.g., programmed instructions).
- memory 910 can include any suitable computer readable storage media, described below, depending on whether, for example, data access needs to be bi-directional or uni-directional.
- processor 902 can also directly and very rapidly retrieve and store frequently needed data in a cache memory (not shown).
- a removable mass storage device 912 provides additional data storage capacity for the computer system 900 and is coupled either bi-directionally (read/write) or uni-directionally (read only) to processor 902.
- storage 912 can also include computer readable media such as magnetic tape, flash memory, PC-CARDS, portable mass storage devices, holographic storage devices, and other storage devices.
- a fixed mass storage 920 can also, for example, provide additional data storage capacity. The most common example of fixed mass storage 920 is a hard disk drive.
- Mass storage 912, 920 generally store additional programming instructions, data, and the like that typically are not in active use by the processor 902. It will be appreciated that the information retained within mass storages 912 and 920 can be incorporated, if needed, in standard fashion as part of memory 910 (e.g., RAM) as virtual memory.
- bus 914 can also be used to provide access to other subsystems and devices. As shown, these can include a display 918, a network interface 916, a keyboard 904, and a pointing device 908, as well as an auxiliary input/output device interface, a sound card, speakers, and other subsystems as needed.
- the pointing device 908 can be a mouse, stylus, track ball, or tablet, and is useful for interacting with a graphical user interface.
- the network interface 916 allows processor 902 to be coupled to another computer, computer network, or telecommunications network using a network connection as shown.
- the processor 902 can receive information (e.g., data objects or program instructions) from another network or output information to another network in the course of performing method/process steps.
- Information often represented as a sequence of instructions to be executed on a processor, can be received from and outputted to another network.
- An interface card or similar device and appropriate software implemented by e.g.,
- processor 902 can be used to connect the computer system 900 to an external network and transfer data according to standard protocols. For example, various process embodiments disclosed herein can be executed on processor 902, or can be performed across a network such as the Internet, intranet networks, or local area networks, in conjunction with a remote processor that shares a portion of the processing. Additional mass storage devices (not shown) can also be connected to processor 902 through network interface 916.
- auxiliary I/O device interface (not shown) can be used in conjunction with computer system 900.
- the auxiliary I/O device interface can include general and customized interfaces that allow the processor 902 to send and, more typically, receive data from other devices such as microphones, touch-sensitive displays, transducer card readers, tape readers, voice or handwriting recognizers, biometrics readers, cameras, portable mass storage devices, and other computers.
- the computation equipment comprises one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
- processors CPUs
- input/output interfaces network interfaces
- memory volatile and non-volatile memory
- Memory may include such forms as volatile storage devices in computer-readable media, random access memory (RAM), and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
- RAM random access memory
- ROM read-only memory
- flash RAM flash memory
- Computer-readable media including permanent and non-permanent and removable and non-removable media, may achieve information storage by any method or technology.
- Computer-readable commands can be computer-readable commands, data structures, program modules, or other data.
- Examples of computer storage media include but are not limited to phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digit multifunction disc (DVD) or other optical storage, magnetic cassettes, magnetic tape or magnetic disc storage, or other magnetic storage equipment or any other non- transmission media that can be used to store information that is accessible to computers.
- computer-readable media does not include temporary computer-readable media, (transitory media), such as modulated data signals and carrier waves.
- the embodiment of the present application can be provided as methods, systems, or computer program products. Therefore, the present application may take the form of complete hardware embodiments, complete software embodiments, or embodiments that combine software and hardware. In addition, the present application can take the form of computer program products implemented on one or more computer-operable storage media (including but not limited to magnetic disk storage devices, CD- ROMs, and optical storage devices) containing computer operable program codes.
- computer-operable storage media including but not limited to magnetic disk storage devices, CD- ROMs, and optical storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- General Physics & Mathematics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
L'invention concerne la présentation d'informations sur la base d'une vidéo, consistant : à extraire un ensemble d'informations d'attributs d'une ou plusieurs images associées à une vidéo actuellement en lecture ; à déterminer que l'ensemble d'informations d'attributs est apparié à un ensemble d'informations d'attributs vidéo contenus dans une base de données vidéo, l'ensemble d'informations d'attributs vidéo correspondant à un ensemble d'informations d'identification associées à une vidéo ; à générer une invite sur la base au moins en partie de l'ensemble d'informations d'identification associées à la vidéo qui correspond à l'ensemble d'informations d'attributs vidéo contenues dans la base de données ; à présenter l'invite ; et en réponse à une sélection associée à l'invite de recevoir plus d'informations, à effectuer une recherche dans une base de données d'informations sur la base au moins en partie de l'ensemble d'informations d'identification associées à la vidéo.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310741071.2 | 2013-12-27 | ||
CN201310741071.2A CN104754377A (zh) | 2013-12-27 | 2013-12-27 | 智能电视的数据处理方法、智能电视以及智能电视系统 |
US14/570,604 US20150189384A1 (en) | 2013-12-27 | 2014-12-15 | Presenting information based on a video |
US14/570,604 | 2014-12-15 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015100070A1 true WO2015100070A1 (fr) | 2015-07-02 |
Family
ID=52293244
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2014/070580 WO2015100070A1 (fr) | 2013-12-27 | 2014-12-16 | Présentation d'informations sur la base d'une vidéo |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2015100070A1 (fr) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105183162A (zh) * | 2015-09-02 | 2015-12-23 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN108752808A (zh) * | 2018-06-27 | 2018-11-06 | 东台市富安合成材料有限公司 | 一种钢性载体嵌入式生态革及其制备方法 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN202998337U (zh) * | 2012-11-07 | 2013-06-12 | 深圳新感易搜网络科技有限公司 | 视频节目识别系统 |
US20130198768A1 (en) * | 2011-08-05 | 2013-08-01 | Sony Corporation | Receiving device, receiving method, program, and information processing system |
EP2645732A2 (fr) * | 2010-11-24 | 2013-10-02 | LG Electronics Inc. | Procédé de réception de service amélioré et appareil d'affichage pour celui-ci |
-
2014
- 2014-12-16 WO PCT/US2014/070580 patent/WO2015100070A1/fr active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2645732A2 (fr) * | 2010-11-24 | 2013-10-02 | LG Electronics Inc. | Procédé de réception de service amélioré et appareil d'affichage pour celui-ci |
US20130198768A1 (en) * | 2011-08-05 | 2013-08-01 | Sony Corporation | Receiving device, receiving method, program, and information processing system |
CN202998337U (zh) * | 2012-11-07 | 2013-06-12 | 深圳新感易搜网络科技有限公司 | 视频节目识别系统 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105183162A (zh) * | 2015-09-02 | 2015-12-23 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN108752808A (zh) * | 2018-06-27 | 2018-11-06 | 东台市富安合成材料有限公司 | 一种钢性载体嵌入式生态革及其制备方法 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20150189384A1 (en) | Presenting information based on a video | |
JP7009769B2 (ja) | 推奨生成方法、プログラム、及びサーバ装置 | |
US9332189B2 (en) | User-guided object identification | |
US20190138815A1 (en) | Method, Apparatus, User Terminal, Electronic Equipment, and Server for Video Recognition | |
US12061642B2 (en) | Methods to present search keywords for image-based queries | |
US8935259B2 (en) | Text suggestions for images | |
US8553981B2 (en) | Gesture-based visual search | |
US9547938B2 (en) | Augmenting a live view | |
EP2567536B1 (fr) | Génération d'une image combinée à partir d'images multiples | |
US10078621B2 (en) | Method, apparatus, and system for displaying order information | |
US9691000B1 (en) | Orientation-assisted object recognition | |
JP2020504475A (ja) | ビデオデータ再生中の関連オブジェクトの提供 | |
US9729792B2 (en) | Dynamic image selection | |
US20190325497A1 (en) | Server apparatus, terminal apparatus, and information processing method | |
US10248306B1 (en) | Systems and methods for end-users to link objects from images with digital content | |
CN113869063A (zh) | 数据推荐方法、装置、电子设备及存储介质 | |
US11356728B2 (en) | Interfacing a television with a second device | |
WO2015100070A1 (fr) | Présentation d'informations sur la base d'une vidéo | |
US20180189602A1 (en) | Method of and system for determining and selecting media representing event diversity | |
US10354176B1 (en) | Fingerprint-based experience generation | |
AU2013205544A1 (en) | Method, apparatus and system for displaying augmented content |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14824682 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14824682 Country of ref document: EP Kind code of ref document: A1 |