WO2015118563A1

WO2015118563A1 - A method and system for providing information on one or more frames selected from a video by a user

Info

Publication number: WO2015118563A1
Application number: PCT/IN2015/000075
Authority: WO
Inventors: Singh Bindra GURBRINDER
Original assignee: Gurbrinder Singh Bindra
Priority date: 2014-02-06
Filing date: 2015-02-06
Publication date: 2015-08-13
Also published as: WO2015118563A8

Abstract

The present invention discloses a method and system for providing information/similar advertisement on an object or a complete frame encapsulating all the objects within the frame. The method comprises capturing one or more frames from the video running on a display device based on at least one user input, identifying one or more objects in the one or more captured frames based the at least one user input.storing at least one appropriate frame for the one or more captured frames in a database, retrieving the information associated with the one or more objects in the one or more captured frames, from a server, and displaying the information for user's consumption. In one embodiment, the at least one appropriate frame is selected from the video based on analysis of one or more factors such as level of clarity, appealing posture etc.

Description

A METHOD AND SYSTEM FOR PROVIDING INFORMATION ON ONE OR MORE FRAMES SELECTED FROM A VIDEO BY A USER

RELATED APPLICATION

Benefit is claimed to Indian Provisional Application No.553/CHE/2014titled "VISUALLY SIMILAR ADVERTISING NETWORK FOR ELECTRONIC VISUAL MEDIA" filed on 06 February 2014, which is herein incorporated in its entirety by reference for all purposes.

FIELD OF INVENTION

The present invention related to multimedia files and more particularly related to a method and system of providing information/advertisement on one or more objects selected from a video by a user.

BACKGROUND OF THE INVENTION

As visual media have evolved, so have the ways in which marketers and advertisers have used visual media to sell everything. When movies and televisions becamecommonplace, so did advertisements using product placement and video clips inaddition to text and graphics. Likewise, as Internet-connected computers and mobile phones became householdappliances (and vice versa), so did computer- based advertising.

Marketers use amyriad of ways, via computers to present products and services to users. Webpagesuse banner ads, pay-for-placement ads and pop-up ads to generate revenue throughadvertising. Today, with different wireless networks all around, and a multitude ofportable devices capable of connecting and interconnecting via those myriad wirelessnetworks, the possible ways of receiving l advertisements seems endless. However, all of these historic and more modern types of advertising rely on 'pushing'the text, graphics and/or video specifically for the marketed product or service to thetargeted audience. This type of advertising is very intrusive and interferes with viewingpleasure of the audience.

Hence, there arises a need of a method of providing advertisement or information related to all the objects present in a video.

SUMMARY

An embodiment of the present invention describes a method of providing information/similar advertisement on an object or a complete frame encapsulating all theobjects within the frame. The method comprises capturing one or more frames from the video running on a display device based on at least one user input, identifying one or more objects in the one or more captured frames based the at least one user input.storing at least one appropriate frame for the one or more captured frames in a database, retrieving the information associated with the one or more objects in the one or more captured frames, from a server, and displaying the information for user's consumption. In one embodiment, theat least one appropriate frame is selected from the video based on analysis/processing of one or more factors such as level of clarity, appealing posture etc.

In an embodiment, the method further comprisesdisplayingat least one appropriate frame on the display device for enabling the user to select and access the information associated with the one or more captured frames.

In one embodiment, the information associated with one or more objects in the one or more captured frames comprises at least one of a textual and visual content associated with the one or more captured frames. In another embodiment, the information associated with one or more objects in the one or more captured frames comprises at least one visually matching image. Retrieving the information according to one embodiment of present invention includes analyzing information associated with the one or more objectspresent in the one or more captured frame, comparing analyzed information for identifying visually similar matches andproviding an optimal identified match corresponding to the one or more objects.

In one embodiment, the video is at least one of a pre-processed video and unprocessed video.

In yet another embodiment, the method further comprising pre-processing the video which comprisesinputting a video into to a processing module for pre-processing the video, identifying one or more objects from the one or more frames of the video, andassociating information including all metatagscorresponding to the one or more objects of the one or more frames.

.In one embodiment, the steps for retrieving the information associated with the one or more objects in the one or more captured frames comprisesanalyzing information associated with one or more objects, comparing analyzed information for identifying best visually similar matches, andproviding the best identified match corresponding to one or more objects.

In one embodiment, the user input is provided through a device comprises a keyboard, a mouse, a touch pad, a joystick, a video touch screen, a remote controller, a game controller, a voice command receiver, a motion detector, buttons, and knobs.

In one embodiment, the display device is selected from a group comprises a television, a monitor, a computer, laptop screen, a smart phone screen, a touch screen, and a projector. Another embodiment of the present invention describes a system for providing information/similar advertisement on one or more frames selected from a video by a user.The system comprises means for capturing one or more frames from the video running on a display device based on at least one user input, means for identifying one or more objects in the one or more captured frames based at least one user input.means for storing at least one appropriate frame for the one or more captured frames in a database, andmeans for fetching the information associated with the one or more objects in the one or more captured frames, from a server.

BRIEF DESRIPTION OF THE ACCOMPANYING DRAWINGS

The aforementioned aspects and other features of the present invention will be explained in the following description, taken in conjunction with the accompanying drawings, wherein:

Figure 1 illustrates a block diagram of a system for providing information on one or more objects in one or more frames in a video, according to one embodiment of present invention.

Figure 2A isa schematic representation illustrating method steps and the visual effect of a method of providing information on one or more objects in one or more frames in a video, according to another embodiment of the present invention.

Figure 2B isa schematic representation illustrating method steps and the visual effect of a method of providing information on one or more objects in one or more frames in a video, according to another embodiment of the present invention. Figure 3 is a flow diagram illustrating a method of providing information on one or more frames selected from a video by a user, according to another embodiment of the present invention. Figure 4 is a flow diagram illustrating a method of pre-processing a video, according to one embodiment of present invention.

DETAILED DESCRIPTION OF THE INVENTION The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments. The present invention can be modified in various forms. Thus, the embodiments of the present invention are only provided to explain more clearly the present invention to the ordinarily skilled in the art of the present invention. In the accompanying drawings, like reference numerals are used to indicate like components.

The specification may refer to "an", "one" or "some" embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms "includes", "comprises", "including" and/or "comprising" when used in this specification, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Furthermore, "connected" or "coupled" as used herein may include operatively connected or coupled. As used herein, the term "and/or" includes any and all combinations and arrangements of one or more of the associated listed items.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. The present invention provides any existingor purposefully-created visual media to facilitate providing information which is 'pull' based asagainst 'push'. The information may be about the advertisement of any products present in the video, information on the location where the video is taken, or persons present in the video etc. Advertising using the techniques disclosed herein may be for productsand/or services shown within or somehow related to the visual media, and/or somehowrelated to the particular viewer and/or creator of the visual media. Once a video is created, for example using any kind of a video-recording device such as smart phone, camera, camcorder, webcam, or visual media creation process such as simulation, animation, presentation, video game creation, etc., the video stream must be somehow presented to a user for viewing. With internet connectivity, the created video presentation often involves digitally transmitting the video to the user over the Internet and/or using some wired or wireless capability. Videos may also be presented in an analog manner via wired or wireless methods. Additionally, videos may be transmitted to the user for view via some kind of storage device (e.g., compact disc, thumb drive, mini-disc, etc.). Once transmitted, a coded video stream is decoded and presented to a user for viewing. The user may be on a wired or wireless network, or may be off any network viewing the content from pre- downloaded material or from a loaded (or pre-loaded) storage device. The device being used may be virtually any type of equipment capable of visually presenting the video stream to the user. These devices may include televisions, monitors, set- top/cable boxes, digital video recorders, video disc players, computers, laptops, netbooks, smart books, personal digital assistants, personal video players, electronic book readers, smart phones, tablets, cell phones, set top boxes and any similar type of equipment. Throughout this application, reference will be made to a frame of a video/ video stream. However, those skilled in the art will understand that the techniques disclosed herein are equally applicable to a single image (e.g., JPEG, GIF, TIF, BMP, etc.) and/or any multi-image compilation (e.g., a slideshow, gallery, album, etc.), as well as to individual frames and sets of frames of a video stream. As used herein, either alone or in combination with other words, the term "frame" may refer to a picture, a frame, a field or a slice thereof.

Figure 1 illustrates a block diagram of a system for providing information/advertisement on one or more objects in one or more frames in a video, according to one embodiment of present invention. According to present invention, the system comprises a user device 101 , one or more server 102, and a network 103. The user device101 is connected to one or more servers 102 (such as 102A,

102B.102C, 102N)through anetwork 103. The network includes wire or wireless medium. In one embodiment, the user device has a display where the display is a user interactive display.

In one embodiment of present invention, the user interacts with the user device 101 in order to provide input and receiving output through a display of the user device 101. In another embodiment, the input/output capabilities of user may be automated. The user device 101 includes any type of device compatible to receive instructions, executeand/or initialize commands. In one embodiment, the user device compatible to receive inputs through input component which includes but not limited, a keyboard, a mouse, a touch pad, a joystick, a video touch screen, a remote controller, a game controller, a voice command receiver, a motion detector, buttons or knobs, and the like. The user receives output from (and may provide input to) the user device 101 via a visual media display device. In one embodiment, the visual media displays hot spots which allow users to select the hot-spotted images by touching, clicking or using any other gesture or otherwise pointing device including mouse. The display of the user device facilitates visual interaction with user, which includes but not limited to a television, a monitor, a computer or laptop screen, a smart phone screen, a touch screen, a projector, and the like.

In one embodiment, the server 102 is apromotional information server in general can include computing capabilities, using one or more processing units, connected to internal or external memory. In another embodiment, the server includes a computing device which is capable of supplying visual media information for displaying on the user device 101 also is capable of receiving input from the user through the user device 101 for processing the input. Further, the computing device is capable of managing and controlling what, when and how visual media are supplied to the use device. For example, the computing device may be capable of decoding video streams. The computing device also may be capable of storing visual media information, such as programmed input prompts and/or responses to such prompts, and running stored programs relating to the visual media. In one embodiment, the computing device is connected to a promotional information server, via a wired or wireless connection network, such as a LAN, WAN WiFi, WiMAX, Internet, Intranet, 4G/5G Broadband, DSL, dial-up, cable, satellite, USB, Ethernet, etc..

In another embodiment, the server 102 includes a promotional information calculator, which is capable of performing promotional calculations to facilitate the visually- similar advertising disclosed within this application. The promotional information calculator can execute programs, software code or modules, including machine instructions for a programmable processor, which may be implemented in - one or more high-level programming languages and stored in memory for the promotional information calculator to use.

Figure 2A is a schematic representation illustrating method steps and the visual effect of a method of providing information/advertisement on one or more objects in one or more frames in a video, according to an exemplary embodiment of the present invention. Consider that the video is running in the user device. The user is enabled to interact with the display. Once the user selects one or more objects in a frame, or the complete one or more frame of the video stream, then the selected frames are captured by the user device. The objects present in each of the captured frame are identified. In one embodiment, the best image is stored/ bookmarked for the identified object, based on one or more factors such as level of clarity or more appealing posture etc. The information/advertisement related to the identified objects are collected from different databases based on the nature of the object, location of the user, search history of the user and user interests. For instance, in figure 2A(1) the user selects a frame using touch. The selected frame contains an actress and other objects. This frame is processed to determine an object i.e. the actress. This object is captured and bookmarked/stored in the user device. The user can revisit the information by selecting the object that is selected by him. Figure 2A(2), shows actress clearly and the outfits that she wear. Further, a small icon/discovery cardprovides the information about outfit. This is generated based on analyzing the user interest and previous search history. In one exemplary embodiment, the object is the contents of the frame which include but not limited to, personnel, characters, furniture, vehicles, plants, animals, structures etc.

According to another embodiment of present invention, the user is enabled to revisit the objects that are selected. Upon each selection, the user interests are captured and stored in the database. This in turn enables the user to access the updated the information/advertisement corresponding to the selected objects. For instance, 2A(3) and 2A(4) of the figure 2A displays the updated information/advertisement in respect of the outfit of the actress. Figure 2B is a schematic representation illustrating method steps and the visual effect of a method of providing information on one or more objects in one or more frames in a video, according to another exemplary embodiment of the present invention. In the given example, the information related to the selected frame is given based on the location. For instance, the frame contains an actress in a ceremony. The user selects the location tab, which gives the information about the location of the scene where it has been shot.

Figure 3 illustrates a flow chart of a method 300 of providing information on one or more frames selected from a video by a user according to an embodiment of the present invention. According to Figure 3, at step 301 , the user input on the running video is detected by the user device/display device. At step 302, one or more frames are captured from the video stream running on the display device based on at least one user input. At step 303, one or more objects are identified in the one or more captured frames based the at least one user input. At step 304, appropriate frames for the one or more captured frames in a database are stored in the data base. The appropriate frames are selected from the video stream based on one or more factors such as clarity of the objects present in the frame, straight posture of the object, more appealing posture of the objects present in the frame compared to other frames in the video stream etc. Additionally, the appropriate frame is selected based on user interest and user selection. Further at step 304, the information/advertisement associated or similar to the one or more objects/images in the one or more captured frames are fetched by a server. When theuser access the stored/bookmarked appropriate frame, the display device displays the appropriate frame and enables the user to select and access the information/advertisement associated with the one or more objects/images as indicated at step 306.

In one embodiment, the video is pre-processed in order to provide similar information/advertisement to the user. In another embodiment, the video is processed on real-time in order to provide similar information/advertisement to the user.

For example, a piece of information may be used to present advertising to the viewer of the media. Images within the information may be targeted, highlighted and identified for any type of advertisement. In one exemplary embodiment, the frame of the video stream is an electronic visual media. A method for advertising for electronic visual media in an electronic device includes capturing the electronic visual media, identifying the electronic visual media information associated with the electronic visual media, communicating the electronic visual media information, analyzing and comparing the electronic visual media information for association with the visual-similar advertising, and presenting the visually matched advertising together with the electronic visual media on an electronic device.

Figure 4 is a flow diagram illustrating a method 400 of pre-processing a video stream, according to one embodiment of present invention. At step 401 , a video stream is inputted into a processing module. At step 402, one or more objects from the one or more frames of the video stream are identified. At step 403, a plurality of information existing over the internet/network of advertisersis processed according the user requirement.Further, at step 404, information corresponding to the one or more objects of the one or more frames is associated.

In one embodiment, a system for visually similar advertising for electronic visual media is presented, which can include a media player or a specific property of a media player for capturing the selection made by a viewer using any kind of pointing device such as finger/hand/gesture/mouse and storing the selections thereof or bookmarking the selected electronic visual media information associated with the electronic visual media, information server for receiving the selected /bookmarked electronic visual media information, and information calculator for analyzing the electronic visual media information for association with the visually similar advertising, wherein the media player presents the visually similar advertising together with the electronic visual media on an electronic device.

In another embodiment, an apparatus for visually-similar advertising for electronic visual media is presented, which can include means for capturing the electronic visual media, means for identifying the electronic visual media information associated with the electronic visual media, means for communicating the electronic visual media information, means for analyzing the electronic visual media information for association with the visually- similar advertising, and means for presenting the visually-similar advertising together with the electronic visual media on an electronic device. In yet another embodiment, a computer-program storage apparatus for visually similar advertising for electronic visual media is presented, which can include at least one memory that can have one or more software modules stored thereon, the one or more software modules can be executable by , one or more processors and the one or more software modules can include code for hot spotting the electronic visual media whether the hot spot is displayed or remains hidden so as not to intrude into video watching experience, code for storing or bookmarking the selected electronic visual media information associated with the electronic visual media, code for communicating the electronic visual media information, code for analyzing the electronic visual media information including comparing the subject bookmarked image with images from network of advertisers for association with visually-similar advertising, and code for presenting the visually- similar advertising together with the electronic visual media on an electronic device. In further embodiment, a method for presenting the visually-similar advertising for electronic visual media is presented, which can include selecting a frame of the electronic visual media, finding a visually-similar image to be associated with the frame of the electronic visual media, and presenting the image from the network of advertisers for further exploration.

Claims

Claims:

1. A method of providing information on one or more frames selected from a video by a user, the method comprising: capturing one or more frames from the video running on a display device based on at least one user input; identifying one or more objects in the one or more captured frames based on the user input; storing at least one appropriate frame for the one or more captured frames in a database; retrieving the information associated with the one or more objects in the one or more captured frames, from a server based on user requirement; and displaying the retrieved information for user's consumption.

2. The method as claimed in claim 1 , further comprising: determiningat least one appropriate frame from the one or more frames of the video, where themost appropriate frame is selected from the video based on analysis of one or more factors.

3. The method as claimed in claim 1 further comprising displaying at least one most appropriate frame on the display device for enabling the user to select and access the information associated with the one or more objects in the one or more captured frames.

The method as claimed in claim 1 , wherein the information associated with one or more objects in the one or more captured frames comprises at least one of a textual and visual content associated with the one or more captured frames.

The method as claimed in claim 1, wherein the information associated with one or more objects in the one or more captured frames comprises at least one visually matching image.

The method as claimed in claim 1 , wherein the video is at least one of a pre- processed video and a real time processed video.

The method as claimed in claim land 6, further comprising a method of processing the video, wherein the method comprises: identifying one or more objects from the one or more frames of the input video; processing a plurality of information existing over the internet according the user requirement; and associating information corresponding to the one or more objects of the one or more frames based on processing.

8. The method as claimed in claim 1 , wherein retrieving the information associated with the one or more objects in the one or more captured frames comprises: analyzinginformation associated with the one or more objectspresent in the one or more captured frame; comparinganalyzed information for identifying visually similar matches; and providing an optimalidentified match corresponding tothe one or more objects.

9. The method as claimed in claim 1 , wherein the display device comprises a user interaction enabled display.

10. A method of providing similar advertisement information for one or more objects from one or more selected framesfrom a video by a user, the method comprising: capturing one or more framesfrom the video running on a display device based on at least one user input; identifying one or more objects in the one or more captured framesbased on at least one user input; storing at least one appropriate frames for the one or more captured frames in a database; and retrieving the advertisement information associated with the one or more objects in the one or more captured frames from a server based on user requirement.

1 1. The method as claimed in claim 10, wherein, theappropriate frame is selected from the video based on analysis of one or more factors.

12. The method as claimed in claim 10 further comprising displaying at least one appropriate frame on the display device for enabling the user to select and access the advertisement information associated with the one or more objects in the one or more captured frames.

13. The method as claimed in claim 10, wherein the advertisement information associated with one or more objects in the one or more captured frames comprises at least one of a textual content and visual content associated with the one or more captured frames.

14. The method as claimed in claim 10, wherein the video is at least one of a pre-processed video and a real time processed video.

15. The method as claimed in claim 11 and 14 further comprising a method of processing the video, wherein the method comprises identifying one or more objects from the one or more frames of the input video; processing a plurality of information existing over the internet according the user requirement; and associating information corresponding to the one or more objects of the one or more frames based on processing.

16. A system for providing information on one or more frames selected from a video by a user, the system comprising: means for capturing one or more frames from the video running on a display device based on at least one user input; means for identifying one or more objects in the one or more captured frames based on at least one user input; means for storing at least one appropriate frame for the one or more captured frames in a database; means for fetching the information associated with the one or more objects in the one or more captured frames, from a server;and meansfor displaying the information for user's consumption.