Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
  • G06Q30/02—Marketing; Price estimation or determination; Fundraising
  • G06Q30/0241—Advertisements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
  • G06Q30/02—Marketing; Price estimation or determination; Fundraising
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/81—Monomedia components thereof
  • H04N21/812—Monomedia components thereof involving advertisement data
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/76—Television signal recording
  • H04N5/91—Television signal processing therefor

Definitions

• search engines In data-searching systems preceding the Web, and on the Web since its inception, search engines have employed a variety of tools to aid in organizing and presenting advertisements in tandem with search results and other online content, such as digital images and streaming video. These tools are also leveraged to optimize the revenue received by the search engine, where optimizing revenue may be facilitated by selecting advertisements that are relevant to a user and by placing the selected advertisements in a noticeable location.
• companies that advertise strive to develop advertisements that are attention-capturing, frequently selected by the search engines for display, and, upon being displayed, readily perceived by users of those search engines. If these three objectives are achieved, a company is likely to be successful in selling a particular item or a particular service.
• an eye-catching advertisement placed in a top-center banner position on a web page will likely receive more attention from a user and, thus, likely generate more revenue for the search engine and the company, as opposed to a bland advertisement positioned in a lower portion of the web page. That is, because the advertisement is noticed by the user, the likelihood that the user will take action (e.g., visit a website of the advertiser) based on the advertisement is increased.
• Embodiments of the present invention generally relate to computer-readable media and computerized methods for identifying and tracking an object within video content of a media file (e.g., digital video) such that an awareness of characteristics of the video content is developed. This awareness can then be used for manipulating how and when an advertisement is overlaid on the video content. For instance, the advertisement may be manipulated to interact within the identified object.
• a media file e.g., digital video
• the step of developing the awareness of video-content characteristics is carried out by an offline authoring process.
• This offline authoring process is implemented to identify an object within the video content with which an advertisement will visually interact.
• Tracking may include the steps of targeting a patch within the object appearing in the video content of the media file and tracking the movement of the patch over a sequence of frames within the media file.
• a "patch" generally refers to a prominent set of pixels within the object that exhibits an identifiable texture (e.g., an eye of a person or animal).
• locations of the patch within the sequence of frames are written to a trajectory.
• a trajectory includes a list of patch locations, configured as X and Y coordinates, that are each associated with a particular frame in the sequence of frames.
• the step of manipulating how and when an advertisement is overlaid on the video content is performed by the online rendering process.
• the online rendering process is carried out upon initiating play of the media file. Accordingly, several steps are typically performed before the online rendering process is invoked, such as receiving a plurality of advertisements that are each designed with consideration of the trajectory and choosing one of the received advertisements for rendering based on a selection scheme (e.g., revenue optimizing, rotational, and the like).
• a selection scheme e.g., revenue optimizing, rotational, and the like.
• the online rendering process Upon choosing an advertisement and receiving an indication (e.g., user-initiated selection of a representation of the media file on a web page) to invoke the online rendering process, the online rendering process conducts the following procedures: generating an ad-overlay that accommodates a container to hold the video advertisement; positioning the container within the ad-overlay according to the trajectory; and inserting the chosen advertisement into the container. Accordingly, the ad-overlay is rendered on top of the video content when playing the media file such that the advertisement appears to visually interact with the object or other video content.
• FIG. 1 is a block diagram of an exemplary computing environment suitable for use in implementing embodiments of the present invention
• FIG. 2 is a block diagram illustrating a distributed computing environment, suitable for use in implementing embodiments of the present invention, that is configured to utilize awareness of video content within a media file to select and place an advertisement;
• FIG. 3 is a diagrammatic view of a sequence of frames of a media file with an object being tracked therein, in accordance with an embodiment of the present invention
• FIG. 4 is a diagrammatic view of a user interface (UI) display illustrating an object, within the video content, with a patch selected thereon, in accordance with an embodiment of the present invention
• FIG. 5 depicts progressive UI displays illustrating stages of placing an advertisement on top of the video content within a sequence of frames, in accordance with an embodiment of the present invention
• FIG. 6 is a diagrammatic view of animation of an advertisement being incorporated into video content via an ad-overlay, in accordance with an embodiment of the present invention
• FIG. 7 is a flow diagram illustrating an overall method for performing an offline authoring process to generate a trajectory, in accordance with an embodiment of the present invention.
• FIG. 8 is a flow diagram illustrating an overall method for performing an online rendering process upon initiating play of the media file, in accordance with an embodiment of the present invention.
• the present invention relates to computer- executable instructions, embodied on one or more computer-readable media, that perform a method for dynamically placing an advertisement on top of video content in a media file, based on movement of an object therein.
• the method involves performing an offline authoring process for generating a trajectory.
• the offline authoring process includes the steps of targeting a patch within the object appearing in the video content of the media file, tracking the movement of the patch over a sequence of frames within the media file, and, based on the tracked movement of the patch, writing locations of the patch within the sequence of frames to the trajectory.
• the term "patch" is not meant to be limiting but may encompass any segment of the object that can be consistently identified within a predefined sequence of frames within the media file.
• the term patch may refer to a prominent set of pixels (e.g., eyes) within the object (e.g., bear) that exhibits an identifiable texture. See FIGS. 4 and 6 for a more detailed explanation of how the eyes of a bear may be utilized as a patch to establish a trajectory.
• the term patch may broadly refer to any feature within any sequence of frames in the media file that it appears in a substantial number of the frames of the sequence of frames.
• the method involves performing an online rendering process upon initiating play of the media file.
• the online rendering process includes the steps of automatically selecting the advertisement and, while the media file is playing, dynamically placing the selected advertisement on top of the video content as a function of the locations within the trajectory. Accordingly, the advertisement and media file are rendered in a synchronized manner such that the advertisement appears to visually interact within the object, or at least some portion of the video content.
• aspects of the present invention involve a computerized method, implemented at one or more processing units, for utilizing an awareness of video content within a media file to select and to place a video advertisement therein.
• the method includes abstracting one or more coordinate locations of an object appearing in the video content of the media file.
• object is not meant to be limiting, but may encompass an expansive scope of items, elements, lines, points, figures, or other aspects of the video content being presented upon playing the media file.
• the object represents a most impressive figure or item within the video content.
• the object may be a football.
• FIGS. 3 and 5 the object may be a football.
• the object may be the bear. As such, the thing that initially draws the attention of a viewer of the media file may be selected as the object.
• the object may be determined by monitoring and collecting both the less intrusive and the most intrusive aspects of the video content, and ascertaining the object as the appropriate vehicle within to the video content to be associated with the advertisement. For instance, if the media file is a video clip of a football game and if it is determined that a football being thrown is the most impressive figure, the advertisement may be placed on the football and/or on the jersey of the player receiving the football, which is not as prominent yet still captures the user's attention.
• the computerized method continues with, at least temporarily, storing coordinate locations of the object on a trajectory.
• the coordinate locations are stored in association within a sequence of frames comprising the media file.
• the trajectory is utilized to generate an ad-overlay that accommodates a container that holds the video advertisement.
• the container is positioned within the ad-overlay according to the coordinate locations stored in the trajectory. For instance, the container may be placed on top of the coordinate locations.
• placing the container on top of the coordinate locations involves inserting the video advertisement within the container and positioning the video advertisement on top of a football, which was previously determined to be the object.
• the computerized method includes the steps of receiving the video advertisement, inserting the video advertisement into the container, and rendering the ad-overlay on top of the video content when playing the media file.
• embodiments of the present invention provide for selection and presentation of advertisements, or the video advertisement.
• the term "advertisement" or the phrase "video advertisement” is not meant to be limiting.
• advertisements could relate to a promotional communication between a seller offering goods or services to a prospective purchaser of such goods or services.
• the advertisement could contain any type or amount of data that is capable of being communicated for the purpose of generating interest in, or sale of, goods or services, such as text, animation, executable information, video, audio, and other various forms.
• the advertisement may be configured as a digital image that is published within an advertisement space allocated within a UI display.
• the UI display is rendered by a web browser or other application running on a client device.
• the video advertisement may be specifically designed to visually interact with the object within the video content of the media file.
• the design of the video advertisement may be performed by an administrator associated with the web browser, a third-party advertising company, or any other entity capable of generating video content. Further, the design of the video advertisement may be based on the trajectory, the timestamps associated with locations of the object, a theme of the media file, an identity of the object, or any other useful criteria.
• the video advertisement may be developed in such a way as to visually interact with the video content when played.
• the present invention encompasses a computer system for abstracting information from the video content of a media file and for placing the advertisement within the video content to visually interact therewith.
• the abstracted information allows for developing the visually interactive advertisement, as discussed immediately above.
• the computer system includes a first processing unit and a second processing unit.
• the first processing unit is configured to accomplish at least the following steps: access the media file; track locations of an object appearing in the video content of the media file on a frame -by- frame basis; and write the tracked locations to a trajectory.
• the second processing unit is configured to accomplish the following steps: access the advertisement that is developed utilizing the trajectory; dynamically place the advertisement in a position on top of the video content based on the tracked locations; and render the video content in synchronization with animating the advertisement placed thereon. Accordingly, the animated advertisement appears to visually interact with the video content.
• these steps above may be performed by a single processing unit, on a multitude of processing units that are communicatively coupled (e.g., server cloud), and/or on one processing unit. Further, some of the steps may be carried out in an offline authoring process, while other steps may be carried out in real-time as the video content is streaming online.
• the first processing unit may operate offline while the second processing unit may operate online.
• computing device 100 an exemplary operating environment for implementing embodiments of the present invention is shown and designated generally as computing device 100.
• Computing device 100 is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing device 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.
• the invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device.
• program components including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types.
• Embodiments of the present invention may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, specialty computing devices, etc.
• Embodiments of the invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
• computing device 100 includes a bus 110 that directly or indirectly couples the following devices: memory 112, one or more processors 114, one or more presentation components 116, input/output (I/O) ports 118, I/O components 120, and an illustrative power supply 122.
• Bus 110 represents what may be one or more busses (such as an address bus, data bus, or combination thereof).
• FIG. 1 is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present invention. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “handheld device,” etc., as all are contemplated within the scope of FIG. 1 and reference to "computer” or “computing device.”
• Computing device 100 typically includes a variety of computer-readable media.
• computer-readable media may comprise Random Access Memory (RAM); Read Only Memory (ROM); Electronically Erasable Programmable Read Only Memory (EEPROM); flash memory or other memory technologies; CDROM, digital versatile disks (DVDs), or other optical or holographic media; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices; or any other medium that can be used to encode desired information and be accessed by computing device 100.
• Memory 112 includes computer-storage media in the form of volatile and/or nonvolatile memory.
• the memory may be removable, non removable, or a combination thereof.
• Exemplary hardware devices include solid-state memory, hard drives, optical- disc drives, etc.
• Computing device 100 includes one or more processors that read data from various entities such as memory 112 or I/O components 120.
• Presentation component(s) 116 present data indications to a user or other device.
• Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
• I/O ports 118 allow computing device 100 to be logically coupled to other devices including I/O components 120, some of which may be built in.
• Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
• FIG. 2 is a block diagram illustrating a distributed computing environment 200 suitable for use in implementing embodiments of the present invention.
• the exemplary computing environment 200 includes a first processing unit 210, a second processing unit 220, at least one data store 230, a display device 250, and a network (not shown) that interconnects each of these items.
• Each of the first processing unit 210 and the second processing unit 220, shown in FIG. 2 may take the form of various types of computing devices, such as, for example, the computing device 100 described above with reference to FIG. 1.
• first processing unit 210 and the second processing unit 220 may be a personal computer, desktop computer, laptop computer, consumer electronic device, handheld device (e.g., personal digital assistant), various servers, processing equipment, and the like. It should be noted, however, that the invention is not limited to implementation on such computing devices but may be implemented on any of a variety of different types of computing devices within the scope of embodiments of the present invention.
• each of the first processing unit 210 and the second processing unit 220 includes, or is linked to, some form of a computing unit (e.g., central processing unit, microprocessor, etc.) to support operations of the component(s) running thereon.
• a computing unit e.g., central processing unit, microprocessor, etc.
• the phrase “computing unit” generally refers to a dedicated computing device with processing power and storage memory, which supports operating software that underlies the execution of software, applications, and computer programs thereon.
• the computing unit is configured with tangible hardware elements, or machines, that are integral, or operably coupled, to the first processing unit 210 and the second processing unit 220 in order to enable each device to perform communication-related processes and other operations (e.g., executing an offline authoring process 215 or an online rendering process 225).
• the computing unit may encompass a processor (not shown) coupled to the computer-readable medium accommodated by each of the first processing unit 210 and the second processing unit 220.
• the computer-readable medium includes physical memory that stores, at least temporarily, a plurality of computer software components that are executable by the processor.
• the term "processor” is not meant to be limiting and may encompass any elements of the computing unit that act in a computational capacity. In such capacity, the processor may be configured as a tangible article that processes instructions. In an exemplary embodiment, processing may involve fetching, decoding/interpreting, executing, and writing back instructions.
• the processor may transfer information to and from other resources that are integral to, or disposed on, the first processing unit 210 and the second processing unit 220.
• resources refer to software components or hardware mechanisms that enable the first processing unit 210 and the second processing unit 220 to perform a particular function.
• a resource accommodated by the first processing unit 210 includes a component to conduct the offline authoring process 215, while a resource accommodated by the second processing unit includes a component to conduct the online rendering process 225.
• the second processing unit 220 may be integral to a computer that has a monitor to serve as the display device 250. In these embodiment, the computer may include an input device (not shown).
• the input device is provided to receive input(s) affecting, among other things, a media file 205, such as invoking the play of its video content 290, or altering properties of the video content being surfaced at a graphical user interface (GUI) 260 display.
• GUI graphical user interface
• Illustrative input devices include a mouse, joystick, key pad, microphone, I/O components 120 of FIG. 1, or any other component capable of receiving a user input and communicating an indication of that input to the second processing unit 220.
• the display device 250 is configured to render and/or present the GUI 260 thereon.
• the display device 250 which is operably coupled to an output of the second processing unit 220, may be configured as any presentation component that is capable of presenting information to a user, such as a digital monitor, electronic display panel, touch-screen, analog set-top box, plasma screen, Braille pad, and the like.
• the display device 250 is configured to present rich content, such as the advertisement 270 embedded within video content 290 and/or digital images.
• the display device 250 is capable of rendering other forms of media (e.g., audio signals).
• the data store 230 is generally configured to store information associated with the advertisement 270 and the media file 205 that may be selected for concurrent presentation.
• information may include, without limitation, the advertisement 270, the media file 205, a description file 255 to be passed to an ad- designing entity 240; and a group of advertisements (being a compilation of advertisements developed specifically for presentation in tandem with the media file 205) associated within the media file 205, and a trajectory 265.
• the data store 230 may be configured to be searchable for suitable access to the stored advertisement 270 and the stored media file(s) 205. For instance, the data store 230 may be searchable for one or more of the advertisements within the group that are targeted toward interests of a user, relevant to the video content 290, and/or associated within the media file 205.
• the information stored in the data store 230 may be configurable and may include any information relevant to the storage or, access to, and retrieval of the advertisement 270 for placement within the video content 290 of the media file 205 and for rendering the integrated advertisement 270 and media file 205 on the GUI 260.
• the content and volume of such information are not intended to limit the scope of embodiments of the present invention in any way.
• the data store 230 may, in fact, be a plurality of databases, for instance, a database cluster, portions of which may reside on the first processing unit 210, the second processing unit 220, another external computing device (not shown), and/or any combination thereof.
• This distributed computing environment 200 is but one example of a suitable environment that may be implemented to carry out aspects of the present invention and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the illustrated distributed computing environment 200 be interpreted as having any dependency or requirement relating to any one or combination of the devices 210, 220, and 250, the data store 230, nor the components for carrying out the processes 215 and 225 as illustrated. In some embodiments, the components may be implemented as stand-alone devices. In other embodiments, one or more of the components may be integrated directly into the processing units 210 and 220. It will be appreciated and understood that the components for implementing the processes 215 and 225 are exemplary in nature and in number and should not be construed as limiting.
• any number of components and devices may be employed to achieve the desired functionality within the scope of embodiments of the present invention.
• the various components and devices of FIG. 2 are shown with lines for the sake of clarity, in reality, delineating various components is not so clear, and, metaphorically, the lines would more accurately be grey or fuzzy.
• some components and devices of FIG. 2 are depicted as single blocks, the depictions are exemplary in nature and in number and are not to be construed as limiting (e.g., although individual processing units 210 and 220 are shown, the steps and operations performed by each may be performed by a single processing unit or other type of computing device).
• the devices 210, 220, and 250, and the data store 230, of the exemplary system architecture may be interconnected by any method known in the relevant field. For instance, they may be operably coupled via a distributed computing environment that includes multiple computing devices coupled with one another via one or more networks (not shown).
• the networks may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs).
• LANs local area networks
• WANs wide area networks
• Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. Accordingly, the network is not further described herein.
• the components are designed to perform the offline authoring process 215 and the online rendering process 225.
• the offline authoring process 215 includes a plurality of discrete steps that may include the following: targeting a patch within an object appearing in the video content 290 of the media file 205; tracking movement of the patch over a sequence of frames within the media file 205; based on the tracked movement of the patch, abstracting coordinate locations of the patch within the video content 290; and writing locations of the patch within the sequence of frames to the trajectory 265.
• the phrase "media file” is not meant to be construed as limiting, but may encompass any general structure for time-based multimedia, such as video and audio.
• the media file 205 may be configured with any known file formats (e.g. container formats, MP4, and 3GP) that facilitates interchange, management, editing, and presentation of the video content 290.
• the presentation may be local, via a network, or other streaming delivery mechanism.
• the media file may be a digital video that is configured to play upon receiving a user-initiated selection (during an online computing session) directed thereto.
• the media file 205 may be accessed at a variety of storage locations.
• these storage locations may reside locally on the first processing unit 210, in the possession of a user (e.g., internal folders, CD memory, external flash drives, etc), online space accommodated by remote web servers responsible for managing media, a networking site, or a public database for hosting a media collection.
• the offline authoring process 215 Upon retrieving the media file 205, the offline authoring process 215 abstracts information from the media file 205 to generate a trajectory 265 and/or a description file 255.
• the "trajectory" 265 essentially serves as a vehicle to store the abstracted information in a logical format.
• the trajectory may assume a form of an XML file that stores the locations as metadata.
• the trajectory 265 may be distinct from the media file 205, or may comprise data appended to the media file 205 such that media file 205 includes the abstracted information, yet the video content 290 remains unaltered.
• the trajectory 265 may include timestamps associated with each of the locations of the object abstracted from the media file 205, where the timestamps are potentially utilized for developing the advertisement 270 and for starting and stopping play of the advertisement 270 in a manner that synchronizes the presentation of it and the media file 205. Consequently, in this instance, the trajectory 265 persists a robust set of information for accurately describing a location and timing of the object's existence at the location within the media file 205.
• FIG. 3 One embodiment of abstracting information from the media file 205 is shown in FIG. 3.
• a diagrammatic view 300 of a sequence of frames 301, 302, 303, 311, and 312 of the media file 205 is illustrated with an object 320 being tracked therein, in accordance with an embodiment of the present invention.
• the object 320 is represented as a football. As discussed above, however, the object 320 may be any identifiable item occurring in the video content 290.
• the sequence of frames 301, 302, 303, 311, and 312 is analyzed to find the object 320 within the video content 290.
• analyzing involves selecting key frames, shown at reference numerals 301, 302, and 303, and labeling them as such.
• locations 341, 343, and 345 of positions of the object 320 within the key frames 301, 311, and 312, respectively, are manually ascertained. These locations 341, 340, and 345 may be retained in a listing of locations within the trajectory 265 and may be associated with their respective key frames 310, 302, and 303.
• the locations 341, 343, and 345 of positions of the object 320 are X 335 and Y 330 coordinates of the object 320 relative to the key frames 310, 302, and 303.
• a mechanism is applied to automatically interpolate movement of the object 320 on intermediate frames, shown at reference numerals 311 and 312, that are in- between, the key frames 301, 302, and 303.
• the mechanism may comprise a video or vision computing algorithm (e.g., various research algorithms used to understand the video content 290 and recognize the object 320 therein) to review the locations 341, 343, and 345 of the object 320 in the key frames 310, 302, and 303 and to interpolate predicted locations 342 and 344 for the intermediate frames 311 and 312, respectively.
• Interpolation may be carried out by deducing a difference in location of the object 320 from one frame to the next, and identifying the predicted locations 342 and 344 within the difference, thereby linking the locations 341, 343, and 345 into a continuous movement pattern of the object 320. Accordingly, a semiautomatic procedure is conducted for accurately pulling locations 341, 342, 343, 344, and 345 from the video content 290.
• this semiautomatic procedure is scalable to accommodate abstracting accurate locations from large media files because it is not necessary to manually recognize and record a location of the object therein for each frame of a selected sequence of frames in which the advertisements will be placed.
• an additional algorithm is executed for automatically tuning the predicted locations generated by interpolation.
• the tuning process may involve automatically locating the object 320 using known characteristics of the object 320, such as shape, color, size, and predicted location at a particular frame. Further, known characteristics may include an identifiable texture associated with a patch on the object 320, as discussed more fully below with reference to FIG. 4.
• the predicted locations 342 and 343 may be tuned to correspond with an actual position of the object. These tuned locations are indicated by reference numerals 351 and 352. Accordingly, an accurate arc-shaped path of the object 320, which follows the true movement of a football through the air, is saved to the trajectory 265, thereby correcting deficiencies of the interpolation process.
• FIG. 4 is a diagrammatic view of a user interface (UI) display 400 illustrating an object 405, within video content 415, with a patch 410 selected thereon, in accordance with an embodiment of the present invention.
• UI user interface
• the patch 410 is an area of the object 405 (bear) that is generally easy to indentify and frequently appears within the sequence of frames. Further, it is preferred that the patch 410 be of substantially consistent shape and color, have invariant feature points, and be somewhat prominent. As shown, the patch 410 covers a rectangular area (5 x 5 window of pixels 420) with a distinct texture (contrasting dark and light colors of the eye as compared against the brown fur of the bear) that is targeted within the object 405, or at least associated with the object 405. As such, the window of pixels 420 can be used to manually or automatically identify a location of the object 405, or specific portions thereof.
• the window of pixels 420 can be used to manually or automatically identify a vector 425 established by the window, or set, of pixels 420 that are designated as the patch 410.
• attributes of the identified vector 425 are maintained in the trajectory 265. These attributes may include a radial direction and origin.
• the attributes in the trajectory 265 are employed to render an advertisement at positions within the video content 290 that consistently intersect the identified vector 425.
• the vector 425 is based on a feature of the object 405 that naturally provides a linear subspace. For instance, as illustrated in FIG. 4, identifying attributes of the vector 425 involves ascertaining line-of-sight originating from one or more eyes of the object 405. In operation, employing the attributes in the trajectory 265 to render an advertisement at a position within the video content 290 that consistently intersects the vector 425 involves placing the advertisement in a position that intersects the line-of-sight of the object 405, or bear. As such, because the bear appears to be looking at the advertisement upon placement, the attention of a viewer of the media file 205 would likely gravitate toward the advertisement as well. [0054] Returning to FIG.
• the first processing unit 210 may conduct the offline authoring process 215 that includes generating a description file 255.
• Generating the description file 255 may involve analyzing the video content 290 to determine a theme of the media file 205, and combining the media- file theme with the trajectory 265 to form the description file 255.
• the phrase "description file” is not meant to be limiting, but may encompass a broad range of vehicles for carrying information related to the video content 290 to an ad-designer entity 240 in order to assist in developing the advertisement 270.
• the description file 255 may include some or all data from the trajectory 265, such as coordinate locations and timestamps of positions of an object, as well as a theme or topic of the media file 205 and an identity of the object.
• the ad-designer entity 240 uses some or all information carried by the description file 255 to create the advertisement 270.
• the creation of the advertisement 270 may be based on a concept of a bear in a stream, as illustrated in FIG. 4, such that an appropriate subject of the advertisement 270 may be an animated fish or other water-based animation that is relevant to the bear in a stream. (This example is depicted in FIG.
• the advertisement 270 may be created in a way that visually interacts within the context of the video content 290 and appears more natural or sophisticated.
• ad-designer entity 240 As shown in FIG. 2, only one ad-designer entity 240 is illustrated. However, it should be appreciated that a plurality of advertisement designers may have access to the description file 255 and may create a variety of advertisements that are relevant to the theme of the media file 205 and that can be placed in the video content 290. Accordingly, in embodiments, these relevant advertisements are joined to a group associated with the media file 205. As used herein, the phrase "group" generally refers to a compilation of advertisements developed specifically for being presented in tandem with the media file 205. In operation, the group may be stored on the data store 230 and may be accessible by the second processing unit 220 for gathering a relevant advertisement to be placed on the video content 290 during the online rendering process 225.
• the online rendering process 225 applies a selection scheme that provides rules for choosing one of the relevant advertisements within the group (e.g., on a rotational basis). [0057] Further, the online rendering process 225 carries out a plurality of steps for placing the advertisement 270 on top of the video content 290.
• the trigger for implementing the online rendering process 225 involves a user selection of a representation of the media file 205. This user selection may involve a user-initiated click action directed toward a uniform resource locator (URL) linked to the media file 205. Or, the user selection may involve launching a web browser that is configured to present the media file 205. In yet other embodiments, the user selection involves receiving an indication that a user-initiated selection occurred with respect to a visual representation of the advertisement 270.
• URL uniform resource locator
• the variety of steps performed by the online rendering process 225 include one or more of the following, in no particular order: selecting the advertisement 270; generating an ad-overlay that accommodates a container to hold the advertisement 270, where the container is positioned within the ad-overlay according to the trajectory 265; inserting the advertisement 270 into the container; and rendering the ad-overlay on top of the video content 290 when playing the media file 205.
• a particular embodiment of performing these steps is depicted at FIG. 5. In particular, FIG.
• FIG. 5 depicts progressive UI displays illustrating stages of placing an advertisement 510 (flag waving in the wind) on top of the object 320 (football) within the sequence of frames 302, 312, and 303, in accordance with an embodiment of the present invention.
• the advertisement 510 can be placed on the object 320 in such a manner that the flag remains on the football throughout the movement of the football through the air.
• attention is drawn to the advertisement 510.
• a trajectory associated with the object 320 allows for creation and placement of the advertisement 510 such that it visually interacts with the video content.
• the trajectory provides an advertisement designer with a concept of a path of the object 320 allowing the advertisement designer to animate the advertisement 510 in a meaningful way.
• the flag (advertisement 510) is blowing in a direction as if it were attached to the football (object 320) as it travels through the air.
• the trajectory allows the online rendering process to dynamically place the advertisement 510 on top of the video content by rendering the advertisement 510 at positions within the video content that substantially correspond to the locations of the object 320, or patch, written to the trajectory. Accordingly, the flag may be placed, based on X and Y coordinate locations of the football, along its entire path.
• an interesting map that records locations of significant objects embedded within the video content may be applied.
• the phrase "interesting map” relates to information gathered from the sequence of frames that may be employed for positioning the advertisement 510 (flag) on top of the object 320 (football).
• the interesting map may include information about another object 520 (receiver) within the video content.
• the position of the advertisement 510 may be adjusted by an offset 550 so that it does not obscure the object 520 when being placed.
• the interesting map allows for building freedom into the placement of the advertisement 510 about the locations in the trajectory. This freedom provides the ability to rotate or translate laterally/vertically the advertisement 510 to avoid blocking any significant object (e.g., the object 520) or other critical aspects in the video content.
• FIG. 6 is a diagrammatic view 600 of animation of an advertisement 610 being incorporated into video content 415 via an ad-overlay 620, in accordance with an embodiment of the present invention.
• the advertisement 610 is created to include animation that visually interacts with the video content 415.
• the fish (advertisement 610) is created to swim within a stream (video content 415).
• the ad-overlay 620 is prepared with a container 615.
• the container 615 is placed within the ad-overlay 620 as a function of the locations of the object 405 or a vector 425 originating from the object 405.
• the container 615 is placed at an intersection with the vector 425 using X' and Y' coordinate locations of the vector 425.
• the container 615 may be placed in proximity with X and Y coordinate locations of the object 405 itself. Accordingly, the use of the container 615 to dynamically place the advertisement 610 within the video content 415 provides a suitable mechanism for positioning the advertisement 610 based on the trajectory, thereby generating the visual interaction between the advertisement 610 and the object 405.
• FIG. 7 illustrates a high-level overview of techniques for performing an offline authoring process to generate a trajectory, in accordance with an embodiment of the present invention.
• steps may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
• the exemplary flow diagram 700 commences with targeting a patch within an object appearing in video content of the media file, as indicated at block 710.
• the patch 410 may cover an identifiable texture (e.g., eye) of the object (e.g., bear).
• the movement of the patch is tracked over a sequence of frames (e.g., employing the offline authoring process 215 of FIG. 2), as indicated at block 720.
• tracking the movement of the patch, or object may involve the following steps: selecting key frames within the sequence of frames (see block 730); manually inputting the locations of the patch within the key frames into the trajectory (see block 740); and utilizing the input locations to automatically interpolate movement of the patch on intermediate frames that reside in-between the key frames (see block 750).
• the process of tracking movement of the patch may further include the following steps: partitioning the interpolated movement into predicted locations that are each associated with the intermediate frames, respectively (see block 760); and tuning the predicted locations based on perceived locations of an identifiable texture associated with the patch (see block 770). Based on the tracked movement of the patch, the locations of the patch may be written to a trajectory 780.
• FIG. 8 a flow diagram illustrating an overall method 800 for performing an online rendering process (e.g., online rendering process 225 performed by the second processing unit 220 of FIG. 2) upon initiating play of the media file is shown, in accordance with an embodiment of the present invention.
• the method 800 includes automatically selecting an advertisement (e.g., utilizing a selection scheme), as indicated at block 810.
• the advertisement e.g., utilizing a selection scheme
• the method 800 involves dynamically placing the selected advertisement on top of the video content of the media file.
• the advertisement is dynamically placed as a function of the locations saved to the trajectory.
• the process of dynamically placing includes the following steps: creating an ad-overlay that includes a container that is positioned within the ad-overlay based on the trajectory (see block 830); inserting the selected advertisement into the container (see block 840); and rendering the ad-overlay and the media file in a synchronized manner such that the container is layered on top of the video content (see block 850).
• the process of dynamically placing may further include rendering the selected advertisement at positions within the video content that substantially correspond to the locations of a patch written to the trajectory, as indicated at block 860. As such, the advertisement will appear to visually interact with an object in the video content and draw a user's attention to the advertisement.

Landscapes

• Business, Economics & Management (AREA)
• Engineering & Computer Science (AREA)
• Finance (AREA)
• Accounting & Taxation (AREA)
• Development Economics (AREA)
• Strategic Management (AREA)
• Marketing (AREA)
• Economics (AREA)
• Game Theory and Decision Science (AREA)
• Entrepreneurship & Innovation (AREA)
• Physics & Mathematics (AREA)
• General Business, Economics & Management (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Signal Processing (AREA)
• Multimedia (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)
• Processing Or Creating Images (AREA)
• Information Transfer Between Computers (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Priority Applications (10)

Applications Claiming Priority (4)

Publications (2)

Family

ID=43781716

Family Applications (1)

Country Status (13)

Families Citing this family (50)

Family Cites Families (34)

• 2009
  • 2009-12-08 US US12/633,609 patent/US9111287B2/en active Active
• 2010
  • 2010-08-19 TW TW099127767A patent/TWI521456B/zh not_active IP Right Cessation
  • 2010-08-31 MX MX2012003327A patent/MX2012003327A/es active IP Right Grant
  • 2010-08-31 CA CA2771167A patent/CA2771167C/en active Active
  • 2010-08-31 RU RU2012112228/08A patent/RU2542923C2/ru not_active IP Right Cessation
  • 2010-08-31 SG SG2012007225A patent/SG178220A1/en unknown
  • 2010-08-31 BR BR112012007127A patent/BR112012007127A2/pt not_active IP Right Cessation
  • 2010-08-31 JP JP2012532091A patent/JP5570607B2/ja not_active Expired - Fee Related
  • 2010-08-31 EP EP10820993.3A patent/EP2483858A4/en not_active Ceased
  • 2010-08-31 AU AU2010301005A patent/AU2010301005B2/en not_active Ceased
  • 2010-08-31 WO PCT/US2010/047198 patent/WO2011041056A2/en not_active Ceased
  • 2010-08-31 KR KR1020127008172A patent/KR101760962B1/ko not_active Expired - Fee Related
  • 2010-08-31 CN CN201080044011.XA patent/CN102576441B/zh active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events